Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for QNX Neutrino to standard library #106673

Merged
merged 5 commits into from
Mar 2, 2023

Conversation

flba-eb
Copy link
Contributor

@flba-eb flba-eb commented Jan 10, 2023

This change:

  • adds standard library support for QNX Neutrino (7.1).
  • upgrades libc to version 0.2.139 which supports QNX Neutrino

@gh-tr

⚠️ Backtraces on QNX require rust-lang/backtrace-rs#507 which is not yet merged! (But everything else works without these changes) ⚠️

Tested mainly with a x86_64 virtual machine (see qnx-nto.md) and partially with an aarch64 hardware (some tests fail due to constrained resources).

@rustbot
Copy link
Collaborator

rustbot commented Jan 10, 2023

r? @m-ou-se

(rustbot has picked a reviewer for you, use r? to override)

@rustbot rustbot added T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Jan 10, 2023
@flba-eb flba-eb marked this pull request as ready for review January 10, 2023 09:57
@rustbot
Copy link
Collaborator

rustbot commented Jan 10, 2023

Hey! It looks like you've submitted a new PR for the library teams!

If this PR contains changes to any rust-lang/rust public library APIs then please comment with @rustbot label +T-libs-api -T-libs to tag it appropriately. If this PR contains changes to any unstable APIs please edit the PR description to add a link to the relevant API Change Proposal or create one if you haven't already. If you're unsure where your change falls no worries, just leave it as is and the reviewer will take a look and make a decision to forward on if necessary.

Examples of T-libs-api changes:

  • Stabilizing library features
  • Introducing insta-stable changes such as new implementations of existing stable traits on existing stable types
  • Introducing new or changing existing unstable library APIs (excluding permanently unstable features / features without a tracking issue)
  • Changing public documentation in ways that create new stability guarantees
  • Changing observable runtime behavior of library APIs

@flba-eb flba-eb marked this pull request as draft January 10, 2023 09:57
@flba-eb flba-eb changed the title Add QNX Neutrino standard library support Add support for QNX Neutrino to standard library Jan 10, 2023
@flba-eb flba-eb marked this pull request as ready for review January 10, 2023 12:42
@flba-eb flba-eb force-pushed the add_qnx_nto_stdlib branch 2 times, most recently from a8a2332 to b1191fd Compare January 10, 2023 15:37
Copy link
Member

@joboet joboet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the synchronization primitives aside from Mutex, I think it would be better to implement futex support using pthread_sleepon_* and share the Linux code, so that they do not need to use lazy allocation. See #93740 for context.

library/std/src/sys/unix/os.rs Outdated Show resolved Hide resolved
library/std/src/sys/unix/thread.rs Outdated Show resolved Hide resolved
library/std/src/sys/unix/thread.rs Outdated Show resolved Hide resolved
// based on stable time.
let now = Instant::now();

let timeout = SystemTime::now()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This timeout uses the wrong clock, the condvar is configured to use CLOCK_MONOTONIC (which Instant::now uses) above to avoid issues when changing the system time.

Copy link
Contributor Author

@flba-eb flba-eb Jan 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will discuss possible solutions with @gh-tr and fix the new merge conflicts tomorrow.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should now be fixed as well, thanks for spotting this bug. In pthread.rs the correct time source has already been used, now pthread_condvcar.rs uses the same implementation (and almost the same as e.g. Linux which is why I've merged the two wait_timeout functions). Also compared it with the QNX documentation and example. @gh-tr please also have another look just to be sure. Thanks!

library/std/src/sys/unix/time.rs Outdated Show resolved Hide resolved
src/test/ui/thread-local/tls.rs Outdated Show resolved Hide resolved
library/std/src/net/udp/tests.rs Outdated Show resolved Hide resolved
@gh-tr
Copy link

gh-tr commented Jan 11, 2023

For the synchronization primitives aside from Mutex, I think it would be better to implement futex support using pthread_sleepon_* and share the Linux code, so that they do not need to use lazy allocation. See #93740 for context.

Would it be possible to elaborate on what you're suggesting here? I don't see any references to pthread_sleepon_* being used when searching the nightly builds and it's unclear to me how this can be used to avoid lazy allocation. I haven't looked in the kernel to see if it's safe to move mutex's around memory when not in a locked state, but I don't think this is what you're suggesting.

As a side note, in Neutrino, mutex locking/unlocking are predominately handled in user space and only hit the kernel under contention.

If there is room for optimisation, perhaps it makes sense to push it as a separate pull request?

@flba-eb flba-eb force-pushed the add_qnx_nto_stdlib branch 2 times, most recently from ca6e61a to 4843dee Compare January 11, 2023 16:06
@joboet
Copy link
Member

joboet commented Jan 11, 2023

For the synchronization primitives aside from Mutex, I think it would be better to implement futex support using pthread_sleepon_* and share the Linux code, so that they do not need to use lazy allocation. See #93740 for context.

Would it be possible to elaborate on what you're suggesting here? I don't see any references to pthread_sleepon_* being used when searching the nightly builds and it's unclear to me how this can be used to avoid lazy allocation. I haven't looked in the kernel to see if it's safe to move mutex's around memory when not in a locked state, but I don't think this is what you're suggesting.

As a side note, in Neutrino, mutex locking/unlocking are predominately handled in user space and only hit the kernel under contention.

If there is room for optimisation, perhaps it makes sense to push it as a separate pull request?

On Linux and some other platforms, std implements its own versions synchronization primitives instead of using the versions provided by pthread, as the POSIX specification does not guarantee that they are movable when not in use. If I understand correctly, pthread_sleepon_* allows emulating the futex syscall used in these implementations like so:

fn futex_wait(addr: &AtomicU32, val: u32) {
    pthread_sleepon_lock();
    if addr.load(Relaxed) == val {
        pthread_sleepon_wait(addr.as_mut_ptr().cast());
    }
    pthread_sleepon_unlock();
}

fn futex_wake(addr: &AtomicU32) {
    pthread_sleepon_lock();
    pthread_sleepon_signal(addr.as_mut_ptr().cast());
    pthread_sleepon_unlock();
}

Of course, priority inheritance is not provided by these implementations, so Mutex still needs to use the pthread_mutex, but for RwLock, Condvar, Once and the Parker struct used for thread::park the futex-based variants are more flexible and maybe even faster.

@flba-eb flba-eb force-pushed the add_qnx_nto_stdlib branch 2 times, most recently from 125478e to e5b3d1b Compare January 12, 2023 09:18
@flba-eb
Copy link
Contributor Author

flba-eb commented Jan 12, 2023

If there is room for optimisation, perhaps it makes sense to push it as a separate pull request?

On Linux and some other platforms, std implements its own versions synchronization primitives instead of using the versions provided by pthread, as the POSIX specification does not guarantee that they are movable when not in use. If I understand correctly, pthread_sleepon_* allows emulating the futex syscall used in these implementations like so:

fn futex_wait(addr: &AtomicU32, val: u32) {
    pthread_sleepon_lock();
    if addr.load(Relaxed) == val {
        pthread_sleepon_wait(addr.as_mut_ptr().cast());
    }
    pthread_sleepon_unlock();
}

fn futex_wake(addr: &AtomicU32) {
    pthread_sleepon_lock();
    pthread_sleepon_signal(addr.as_mut_ptr().cast());
    pthread_sleepon_unlock();
}

Of course, priority inheritance is not provided by these implementations, so Mutex still needs to use the pthread_mutex, but for RwLock, Condvar, Once and the Parker struct used for thread::park the futex-based variants are more flexible and maybe even faster.

Thanks for the good explanation, @joboet !

As far as I understand them, pthread_sleepon_lock is using one global lock (one per process?). As long as the Rust stdlib is the only user of these functions it should be fine, but there could also be some usage outside Rust code (C/C++ code in the same process).

At the moment, the pthread_sleepon_* functions are not available in the libc crate (and they are implemented as simple macros). It should not be a big deal to introduce them but it will take some time.

Considering also the risk of getting a priority inversion (at least I don't have a good enough overview of the std yet to evaluate this) I would prefer to delay this optimization and to discuss with QNX Neutrino kernel experts how the best solution would be. We had this topic on our wish list anyway...

Would this be okay for you, @joboet ?

@flba-eb
Copy link
Contributor Author

flba-eb commented Jan 14, 2023

fb025ee includes a minor change in qnx-nto.md and is rebased to current master.

@albertlarsan68

This comment was marked as off-topic.

@rustbot rustbot removed the T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) label Jan 18, 2023
@flba-eb
Copy link
Contributor Author

flba-eb commented Jan 18, 2023

r? @thomcc
Assigning to Thom as discussed. Thanks for considering to review it! 😄 👍
I guess we should not wait for backtrace-rs as it has some CI issues at the moment (rust-lang/backtrace-rs#511, rust-lang/backtrace-rs#509).

Copy link
Member

@workingjubilee workingjubilee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was a bunch of out-of-band chatter, and the upshot of it is that the tests don't quite pass due to the Neutrino OS on the CPUs in question only being able to handle limited concurrency, even with a retry loop, unless the retry loop is extended to a very large number of cycles.

This is currently expected to be unlikely to affect actual user code that does not strain the resources of the actual target devices that Neutrino will run on. That is because Neutrino is an RTOS, which are often intended for relatively lower-power devices. So, I think that's completely fair, and while a minimal retry loop seems good since it makes most of the tests pass, for the last few holdouts it seems better to just skip the tests right now than to add another 100 retries. That seems like it risks introducing hard-to-debug long delays in code on this OS, because we tried to "fix" things. And this is a tier 3 target.

With that, I have only one last nit.

library/std/src/os/nto/mod.rs Outdated Show resolved Hide resolved
Co-authored-by: Jubilee <46493976+workingjubilee@users.noreply.github.com>
@workingjubilee
Copy link
Member

@bors r+

@bors
Copy link
Contributor

bors commented Mar 1, 2023

📌 Commit a510715 has been approved by workingjubilee

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Mar 1, 2023
@bors
Copy link
Contributor

bors commented Mar 1, 2023

⌛ Testing commit a510715 with merge 955007b771439aabf76ac82f58736241b2cf8707...

@bors
Copy link
Contributor

bors commented Mar 1, 2023

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Mar 1, 2023
@workingjubilee
Copy link
Member

FAILED: lib/libLLVMGlobalISel.a
cmd.exe /C "cd . && "C:\Program Files\CMake\bin\cmake.exe" -E rm -f lib\libLLVMGlobalISel.a && D:\a\rust\rust\mingw32\bin\ar.exe qc lib\libLLVMGlobalISel.a lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/CSEInfo.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/GISelKnownBits.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/CSEMIRBuilder.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/CallLowering.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/GlobalISel.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/Combiner.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/CombinerHelper.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/GISelChangeObserver.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/IRTranslator.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/InlineAsmLowering.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/InstructionSelect.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/InstructionSelector.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/LegalityPredicates.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/LegalizeMutations.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/Legalizer.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/LegalizerHelper.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/LegalizerInfo.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/LegacyLegalizerInfo.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/LoadStoreOpt.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/Localizer.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/LostDebugLocObserver.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/MachineIRBuilder.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/RegBankSelect.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/Utils.cpp.obj && D:\a\rust\rust\mingw32\bin\ranlib.exe lib\libLLVMGlobalISel.a && cd ."
D:\a\rust\rust\mingw32\bin\ar.exe: could not create temporary file whilst writing archive: no more archived files

Spurious LLVM failure?
@bors retry

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 1, 2023
@bors
Copy link
Contributor

bors commented Mar 2, 2023

⌛ Testing commit a510715 with merge 864b625...

@bors
Copy link
Contributor

bors commented Mar 2, 2023

☀️ Test successful - checks-actions
Approved by: workingjubilee
Pushing 864b625 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Mar 2, 2023
@bors bors merged commit 864b625 into rust-lang:master Mar 2, 2023
@rustbot rustbot added this to the 1.69.0 milestone Mar 2, 2023
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (864b625): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
3.3% [2.4%, 3.8%] 3
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-2.0% [-2.0%, -2.0%] 1
All ❌✅ (primary) - - 0

Cycles

This benchmark run did not return any relevant results for this metric.

@rust-log-analyzer
Copy link
Collaborator

The job i686-mingw-2 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
[2604/3021] Linking CXX static library lib\libLLVMSelectionDAG.a
[2605/3021] Linking CXX static library lib\libLLVMWebAssemblyUtils.a
[2606/3021] Linking CXX static library lib\libLLVMAsmPrinter.a
[2607/3021] Linking CXX static library lib\libLLVMGlobalISel.a
FAILED: lib/libLLVMGlobalISel.a 
cmd.exe /C "cd . && "C:\Program Files\CMake\bin\cmake.exe" -E rm -f lib\libLLVMGlobalISel.a && D:\a\rust\rust\mingw32\bin\ar.exe qc lib\libLLVMGlobalISel.a  lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/CSEInfo.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/GISelKnownBits.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/CSEMIRBuilder.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/CallLowering.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/GlobalISel.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/Combiner.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/CombinerHelper.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/GISelChangeObserver.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/IRTranslator.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/InlineAsmLowering.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/InstructionSelect.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/InstructionSelector.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/LegalityPredicates.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/LegalizeMutations.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/Legalizer.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/LegalizerHelper.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/LegalizerInfo.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/LegacyLegalizerInfo.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/LoadStoreOpt.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/Localizer.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/LostDebugLocObserver.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/MachineIRBuilder.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/RegBankSelect.cpp.obj lib/CodeGen/GlobalISel/CMakeFiles/LLVMGlobalISel.dir/Utils.cpp.obj && D:\a\rust\rust\mingw32\bin\ranlib.exe lib\libLLVMGlobalISel.a && cd ."
D:\a\rust\rust\mingw32\bin\ar.exe: could not create temporary file whilst writing archive: no more archived files
[2608/3021] Linking CXX static library lib\libLLVMWebAssemblyAsmParser.a
[2609/3021] Linking CXX static library lib\libLLVMDWARFLinker.a
[2610/3021] Building CXX object tools/dsymutil/CMakeFiles/dsymutil.dir/SymbolMap.cpp.obj
[2611/3021] Building CXX object tools/dsymutil/CMakeFiles/dsymutil.dir/MachOUtils.cpp.obj
[2611/3021] Building CXX object tools/dsymutil/CMakeFiles/dsymutil.dir/MachOUtils.cpp.obj
[2612/3021] Building CXX object tools/dsymutil/CMakeFiles/dsymutil.dir/dsymutil.cpp.obj
[2613/3021] Linking CXX executable bin\llvm-profdata.exe
[2614/3021] Linking CXX static library lib\libLLVMPasses.a
ninja: build stopped: subcommand failed.
command did not execute successfully, got: exit code: 1


build script failed, must exit now', C:\Users\runneradmin\.cargo\registry\src\index.crates.io-1cd66030c949c28d\cmake-0.1.48\src\lib.rs:975:5
 finished in 303.885 seconds
Build completed unsuccessfully in 0:06:55
Build completed unsuccessfully in 0:06:55
make: *** [Makefile:80: ci-mingw-subset-2] Error 1

@flba-eb flba-eb deleted the add_qnx_nto_stdlib branch March 7, 2023 10:50
jyn514 added a commit to jyn514/rust that referenced this pull request Apr 26, 2023
…jyn514

Replace `yes` command by `while-echo` in test `tests/ui/process/process-sigpipe.rs`

The `yes` command is not available on all platforms.

Fixes rust-lang#108596.

Inviting `@mvf` as he contributed to this patch. Thanks! This issue has been discussed in rust-lang#106673 but was moved to rust-lang#108596 to get going.

CC `@gh-tr`

r? `@workingjubilee`
`@rustbot` label +O-neutrino

Notes about the comments rust-lang#106673 (comment):
- The `echo` command is `/proc/boot/echo` (not built-in)
- `/bin/sh` is a symlink to `/proc/boot/ksh`
```sh
# ls -l /bin/sh /proc/boot/ksh /proc/boot/echo
lrwxrwxrwx   1 root      root             14 Mar 20 07:52 /bin/sh -> /proc/boot/ksh
-r-xr-xr-x   1 root      root           9390 Sep 12  2022 /proc/boot/echo
-r-xr-xr-x   1 root      root         308114 Sep 12  2022 /proc/boot/ksh
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. O-neutrino OS: QNX Neutrino, a POSIX-compatible real-time operating system S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.