kmc-solid: Fix memory ordering in thread operations
Fixes two memory ordering issues in the thread state machine (`ThreadInner::lifecycle`) of the [`*-kmc-solid_*`](https://doc.rust-lang.org/nightly/rustc/platform-support/kmc-solid.html) Tier 3 targets.
1. When detaching a thread that is still running (i.e., the owner updates `lifecycle` first, and the child updates it next), the first update did not synchronize-with the second update, resulting in a data race between the first update and the deallocation of `ThreadInner` by the child thread.
2. When joining on a thread, the joiner has to pass its own task ID to the joinee in order to be woken up later, but in doing so, it did not synchronize-with the read operation, creating possible sequences of execution where the joinee wakes up an incorrect or non-existent task.
Both issue are theoretical and most likely have never manifested in practice because of the stronger guarantees provided by the Arm memory model (particularly due to its barrier-based definition). Compiler optimizations could have subverted this, but the inspection of compiled code did not reveal such optimizations taking place.
Bump master bootstrap compiler
This PR bumps the bootstrap compiler to the beta created earlier this week, cherry-picks the stabilization version number updates, and updates the `cfg(bootstrap)`s.
r? `@Mark-Simulacrum`
adjust message on non-unwinding panic
"thread panicked while panicking" is just plain wrong in case this is a non-unwinding panic, such as
- a panic out of a `nounwind` function
- the sanity checks we have in `mem::uninitialized` and `mem::zeroed`
- the optional debug assertion in various unsafe std library functions
Clarify `catch_unwind` docs about panic hooks
Makes it clear from `catch_unwind` docs that the panic hook will be called before the panic is caught.
Fixes#105432
Make sentinel value configurable in `library/std/src/sys_common/thread_local_key.rs`
This is an excerpt of a changeset for the QNX/Neutrino OS. To make the patch for QNX smaller and easier to review, I've extracted this change (which is OS independent). I would be surprised if no other OS is also affected.
All this patch does is to define a `const` for a sentinel value instead of using it directly at several places.
There are OSs that always return the lowest free value. The algorithm in `lazy_init` always avoids keys with the sentinel value.
In affected OSs, this means that each call to `lazy_init` will always request two keys from the OS and returns/frees the first one (with sentinel value) immediately afterwards.
By making the sentinel value configurable, affected OSs can use a different value than zero to prevent this performance issue.
On QNX/Neutrino, it is planned to use a different sentinel value:
```rust
// Define a sentinel value that is unlikely to be returned
// as a TLS key (but it may be returned).
#[cfg(not(target_os = "nto"))]
const KEY_SENTVAL: usize = 0;
// On QNX/Neutrino, 0 is always returned when currently not in use.
// Using 0 would mean to always create two keys and remote the first
// one (with value of 0) immediately afterwards.
#[cfg(target_os = "nto")]
const KEY_SENTVAL: usize = libc::PTHREAD_KEYS_MAX + 1;
```
It seems like no other OS defines `PTHREAD_KEYS_MAX` in Rusts libc, but `limits.h` on unix systems does.
available_parallelism: Gracefully handle zero value cfs_period_us
There seem to be some scenarios where the cgroup cpu quota field `cpu.cfs_period_us` can contain `0`. This field is used to determine the "amount" of parallelism suggested by the function `std:🧵:available_parallelism`
A zero value of this field cause a panic when `available_parallelism()` is invoked. This issue was detected by the call from binaries built by `cargo test`. I really don't feel like `0` is a good value for `cpu.cfs_period_us`, but I also don't think applications should panic if this value is seen.
This panic started happening with rust 1.64.0.
This case is gracefully handled by other projects which read this information: [num_cpus](e437b9d908/src/linux.rs (L207-L210)), [ninja](https://github.com/ninja-build/ninja/pull/2174/files), [dotnet](c4341d45ac/src/coreclr/pal/src/misc/cgroup.cpp (L481-L483))
Before this change, running `cargo test` in environments configured as described above would trigger this panic:
```
$ RUST_BACKTRACE=1 cargo test
Finished test [unoptimized + debuginfo] target(s) in 3.55s
Running unittests src/main.rs (target/debug/deps/x-9a42e145aca2934d)
thread 'main' panicked at 'attempt to divide by zero', library/std/src/sys/unix/thread.rs:546:70
stack backtrace:
0: rust_begin_unwind
1: core::panicking::panic_fmt
2: core::panicking::panic
3: std::sys::unix:🧵:cgroups::quota
4: std::sys::unix:🧵:available_parallelism
5: std:🧵:available_parallelism
6: test::helpers::concurrency::get_concurrency
7: test::console::run_tests_console
8: test::test_main
9: test::test_main_static
10: x::main
at ./src/main.rs:1:1
11: core::ops::function::FnOnce::call_once
at /tmp/rust-1.64-1.64.0-1/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
error: test failed, to rerun pass '--bin x'
```
I've tested this change in an environment which has the bad (questionable?) setup and rebuilding the test executable against a fixed std library fixes the panic.
Move `ReentrantMutex` to `std::sync`
If I understand #84187 correctly, `sys_common` should not contain platform-independent code, even if it is private.
Fix backoff doc to match implementation
The commit 8dddb22943 in the crossbeam-channel PR (#93563) changed the backoff strategy to be quadratic instead of exponential. This updates the doc to prevent confusion.
Use correct clock in `park_timeout` on Horizon
Horizon does not support using `CLOCK_MONOTONIC` with condition variables, so use the system time instead.
More inference-friendly API for lazy
The signature for new was
```
fn new<F>(f: F) -> Lazy<T, F>
```
Notably, with `F` unconstrained, `T` can be literally anything, and just `let _ = Lazy::new(|| 92)` would not typecheck.
This historiacally was a necessity -- `new` is a `const` function, it couldn't have any bounds. Today though, we can move `new` under the `F: FnOnce() -> T` bound, which gives the compiler enough data to infer the type of T from closure.
Realistic `Path::as_mut_os_str` doctest
With "Implement DerefMut for PathBuf" (#105018) now merged, it's
possible to exercise `Path::as_mut_os_str` (#105002) without going
through `into_boxed_path`.
Use a more efficient `Once` on platforms without threads
The current implementation uses an atomic queue and spins rather than panicking when calling `call_once` recursively. Since concurrency is not supported on platforms like WASM, `Once` can be implemented much more efficiently using just a single non-atomic state variable.
fs: Fix#50619 (again) and add a regression test
Bug #50619 was fixed by adding an end_of_stream flag in #50630.
Unfortunately, that fix only applied to the readdir_r() path. When I
switched Linux to use readdir() in #92778, I inadvertently reintroduced
the bug on that platform. Other platforms that had always used
readdir() were presumably never fixed.
This patch enables end_of_stream for all platforms, and adds a
Linux-specific regression test that should hopefully prevent the bug
from being reintroduced again.
Allow blocking `Command::output`
### Problem
Currently, `Command::output` is internally implemented using `Command::spawn`. This is problematic because some targets (like UEFI) do not actually support multitasking and thus block while the program is executing. This coupling does not make much sense as `Command::output` is supposed to block until the execution is complete anyway and thus does not need to rely on a non-blocking `Child` or any other intermediate.
### Solution
This PR moves the implementation of `Command::output` to `std::sys`. This means targets can choose to implement only `Command::output` without having to implement `Command::spawn`.
### Additional Information
This was originally conceived when working on https://github.com/rust-lang/rust/pull/100316. Currently, the only target I know about that will benefit from this change is UEFI.
This PR can also be used to implement more efficient `Command::output` since the intermediate `Process` is not actually needed anymore, but that is outside the scope of this PR.
Since this is not a public API change, I'm not sure if an RFC is needed or not.
Implement DerefMut for PathBuf
Without this, there's no way to get a `&mut Path` from `PathBuf` without
going through `into_boxed_path`. This is relevant now that #105002 adds
`PathBuf::as_mut_os_string` and `Path::as_mut_os_str`.
Use more LFS functions.
On Linux, use mmap64, open64, openat64, and sendfile64 in place of their non-LFS counterparts.
This is relevant to #94173.
With these changes (together with rust-lang/backtrace-rs#501), the simple binaries I produce with rustc seem to have no non-LFS functions, so maybe #94173 is fixed. But I can't be sure if I've missed something and maybe some non-LFS functions could sneak in somehow.
Bug #50619 was fixed by adding an end_of_stream flag in #50630.
Unfortunately, that fix only applied to the readdir_r() path. When I
switched Linux to use readdir() in #92778, I inadvertently reintroduced
the bug on that platform. Other platforms that had always used
readdir() were presumably never fixed.
This patch enables end_of_stream for all platforms, and adds a
Linux-specific regression test that should hopefully prevent the bug
from being reintroduced again.
Better documentation for env::home_dir()'s broken behaviour
This improves the documentation to say *why* it was deprecated. The reason was because it reads `HOME` on Windows which is meaningless there. Note that the PR that deprecated it stated that returning an empty string if `HOME` is set to an empty string was a problem, however I can find no evidence that this is the case. `cd` handles it fine whereas if `HOME` is unset it gives an explicit `HOME not set` error.
* Original deprecation reason: https://internals.rust-lang.org/t/deprecate-or-break-fix-std-env-home-dir/7315
* Original deprecation PR: https://github.com/rust-lang/rust/pull/51656
See #71684
This allows decoupling `Command::spawn` and `Command::output`. This is
useful for targets which do support launching programs in blocking mode
but do not support multitasking (Eg: UEFI).
This was originally conceived when working on https://github.com/rust-lang/rust/pull/100316
Signed-off-by: Ayush Singh <ayushsingh1325@gmail.com>
Avoid heap allocation when truncating thread names
Ensure that heap allocation does not occur in a thread until `std::thread` is ready. This fixes issues with custom allocators that call `std:🧵:current()`, since doing so prematurely initializes `THREAD_INFO` and causes the following `thread_info::set()` to fail.
Reimplement std's thread parker on top of events on SGX
Mutex and Condvar are being replaced by more efficient implementations, which need thread parking themselves (see #93740). Therefore, the generic `Parker` needs to be replaced on all platforms where the new lock implementation will be used.
SGX enclaves have a per-thread event state, which allows waiting for and setting specific bits. This is already used by the current mutex implementation. The thread parker can however be much more efficient, as it only needs to store the `TCS` address of one thread. This address is stored in a state variable, which can also be set to indicate the thread was already notified.
`park_timeout` does not guard against spurious wakeups like the current condition variable does. This is allowed by the API of `Parker`, and I think it is better to let users handle these wakeups themselves as the guarding is quite expensive and might not be necessary.
`@jethrogb` as you wrote the initial SGX support for `std`, I assume you are the target maintainer? Could you help me test this, please? Lacking a x86_64 chip, I can't run SGX.
Add `read_to_end` method for `sys::{target}::pipe::AnonPipe`. This allows
having a more optimized version of `read_to_end` for ChildStdout.
Signed-off-by: Ayush Singh <ayushsingh1325@gmail.com>
kmc-solid: `std::sys` code maintenance
Includes a set of changes to fix the [`*-kmc-solid_*`](https://doc.rust-lang.org/nightly/rustc/platform-support/kmc-solid.html) Tier 3 targets and make some other improvements.
- Address `fuzzy_provenance_casts` by using `expose_addr` and `from_exposed_addr` for pointer-integer casts
- Add a stub implementation of `is_terminal` (#98070)
- Address `unused_imports` and `unused_unsafe`
- Stop doing `Box::from_raw(&*(x: Box<T>) as *const T as *mut T)`
Ensure that heap allocation does not occur in a thread until std::thread
is ready. This fixes issues with custom allocators that call
std:🧵:current(), since doing so prematurely initializes
THREAD_INFO and causes the following thread_info::set() to fail.
On Linux, use mmap64, open64, openat64, and sendfile64 in place of their
non-LFS counterparts.
This is relevant to #94173.
With these changes (together with rust-lang/backtrace-rs#501), the
simple binaries I produce with rustc seem to have no non-LFS functions,
so maybe #94173 is fixed. But I can't be sure if I've missed something
and maybe some non-LFS functions could sneak in somehow.
Pass on null handle values to child process
Fixes#101645
In Windows, stdio handles are (semantically speaking) `Option<Handle>` where `Handle` is a non-zero value. When spawning a process with `Stdio::Inherit`, Rust currently turns zero values into `-1` values. This has the unfortunate effect of breaking console subprocesses (which typically need stdio) that are spawned from gui applications (that lack stdio by default) because the console process won't be assigned handles from the newly created console (as they usually would in that situation). Worse, `-1` is actually [a valid handle](https://doc.rust-lang.org/std/os/windows/io/struct.OwnedHandle.html) which means "the current process". So if a console process, for example, waits on stdin and it has a `-1` value then the process will end up waiting on itself.
This PR fixes it by propagating the nulls instead of converting them to `-1`.
While I think the current behaviour is a mistake, changing it (however justified) is an API change so I think this PR should at least have some input from t-libs-api. So choosing at random...
r? `@joshtriplett`
