Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Supress niches in coroutines to avoid aliasing violations #129313

Merged
merged 2 commits into from
Sep 8, 2024

Conversation

RalfJung
Copy link
Member

@RalfJung RalfJung commented Aug 20, 2024

As mentioned here, using niches in fields of coroutines that are referenced by other fields is unsound: the discriminant accesses violate the aliasing requirements of the reference pointing to the relevant field. This issue causes Miri errors in practice.

The "obvious" fix for this is to suppress niches in coroutines. That's what this PR does. However, we have several tests explicitly ensuring that we do use niches in coroutines. So I see two options:

  • We guard this behavior behind a -Z flag (that Miri will set by default). There is no known case of these aliasing violations causing miscompilations. But absence of evidence is not evidence of absence...
  • (What this PR does right now.) We temporarily adjust the coroutine layout logic and the associated tests until the proper fix lands. The "proper fix" here is to wrap fields that other fields can point to in UnsafePinned and make UnsafePinned suppress niches; that would then still permit using niches of other fields (those that never get borrowed). However, I know that coroutine sizes are already a problem, so I am not sure if this temporary size regression is acceptable.

@compiler-errors any opinion? Also who else should be Cc'd here?

@rustbot
Copy link
Collaborator

rustbot commented Aug 20, 2024

r? @wesleywiser

rustbot has assigned @wesleywiser.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Aug 20, 2024
@rustbot
Copy link
Collaborator

rustbot commented Aug 20, 2024

The Miri subtree was changed

cc @rust-lang/miri

@RalfJung RalfJung changed the title supress niches in coroutines Supress niches in coroutines to avoid aliasing violations Aug 20, 2024
@rust-log-analyzer

This comment has been minimized.

@compiler-errors
Copy link
Member

I'm of the opinion that we do the latter (fix the tests, fix it the right way later when we can). But also it would be nice if we actually knew how bad this would affect layout sizes in practice... I have no idea how to gather that information, though.

@RalfJung
Copy link
Member Author

RalfJung commented Aug 20, 2024

I blessed the failing tests. async-drop for some reason seems quite badly affected. But that is an unstable feature.

@RalfJung
Copy link
Member Author

RalfJung commented Sep 3, 2024

Since we haven't heard from our reviewer, let's re-roll.
r? compiler

Also maybe it helps to get more people to take a look -- this is primarily about async fn, so Cc @rust-lang/wg-async

@rustbot rustbot assigned fee1-dead and unassigned wesleywiser Sep 3, 2024
@compiler-errors
Copy link
Member

I wanna think about this so

r? compiler-errors

please bother me if i dont review this in a few days!!

@rustbot rustbot assigned compiler-errors and unassigned fee1-dead Sep 3, 2024
Copy link
Member

@compiler-errors compiler-errors left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this seems necessary to do until we implement UnsafePinned correctly. I guess if people complain about this, we could gate this behind miri or something.

@compiler-errors
Copy link
Member

@bors r+ rollup=never

@bors
Copy link
Contributor

bors commented Sep 3, 2024

📌 Commit 12cda6e has been approved by compiler-errors

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 3, 2024
bors added a commit to rust-lang-ci/rust that referenced this pull request Sep 4, 2024
…r-errors

Supress niches in coroutines to avoid aliasing violations

As mentioned [here](rust-lang#63818 (comment)), using niches in fields of coroutines that are referenced by other fields is unsound: the discriminant accesses violate the aliasing requirements of the reference pointing to the relevant field. This issue causes [Miri errors in practice](rust-lang/miri#3780).

The "obvious" fix for this is to suppress niches in coroutines. That's what this PR does. However, we have several tests explicitly ensuring that we *do* use niches in coroutines. So I see two options:
- We guard this behavior behind a `-Z` flag (that Miri will set by default). There is no known case of these aliasing violations causing miscompilations. But absence of evidence is not evidence of absence...
- (What this PR does right now.) We temporarily adjust the coroutine layout logic and the associated tests until the proper fix lands. The "proper fix" here is to wrap fields that other fields can point to in [`UnsafePinned`](rust-lang#125735) and make `UnsafePinned` suppress niches; that would then still permit using niches of *other* fields (those that never get borrowed). However, I know that coroutine sizes are already a problem, so I am not sure if this temporary size regression is acceptable.

`@compiler-errors` any opinion? Also who else should be Cc'd here?
@bors
Copy link
Contributor

bors commented Sep 4, 2024

⌛ Testing commit 12cda6e with merge 15287f5...

@rust-log-analyzer
Copy link
Collaborator

The job x86_64-msvc-ext failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
[RUSTC-TIMING] miri test:false 4.437
error: failed to remove file `C:\a\rust\rust\build\x86_64-pc-windows-msvc\stage1-tools\x86_64-pc-windows-msvc\release\miri.exe`

Caused by:
  Access is denied. (os error 5)
Command has failed. Rerun with -v to see more details.
  local time: Wed, Sep  4, 2024  2:53:57 AM
  network time: Wed, 04 Sep 2024 02:53:58 GMT
##[error]Process completed with exit code 1.
Post job cleanup.

@bors
Copy link
Contributor

bors commented Sep 4, 2024

💔 Test failed - checks-actions

@bors bors removed the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Sep 4, 2024
bors added a commit to rust-lang-ci/rust that referenced this pull request Sep 6, 2024
…r-errors

Supress niches in coroutines to avoid aliasing violations

As mentioned [here](rust-lang#63818 (comment)), using niches in fields of coroutines that are referenced by other fields is unsound: the discriminant accesses violate the aliasing requirements of the reference pointing to the relevant field. This issue causes [Miri errors in practice](rust-lang/miri#3780).

The "obvious" fix for this is to suppress niches in coroutines. That's what this PR does. However, we have several tests explicitly ensuring that we *do* use niches in coroutines. So I see two options:
- We guard this behavior behind a `-Z` flag (that Miri will set by default). There is no known case of these aliasing violations causing miscompilations. But absence of evidence is not evidence of absence...
- (What this PR does right now.) We temporarily adjust the coroutine layout logic and the associated tests until the proper fix lands. The "proper fix" here is to wrap fields that other fields can point to in [`UnsafePinned`](rust-lang#125735) and make `UnsafePinned` suppress niches; that would then still permit using niches of *other* fields (those that never get borrowed). However, I know that coroutine sizes are already a problem, so I am not sure if this temporary size regression is acceptable.

`@compiler-errors` any opinion? Also who else should be Cc'd here?
@bors
Copy link
Contributor

bors commented Sep 6, 2024

⌛ Testing commit 12cda6e with merge 7e24610...

@rust-log-analyzer
Copy link
Collaborator

The job x86_64-msvc-ext failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
[RUSTC-TIMING] miri test:false 4.468
error: failed to remove file `C:\a\rust\rust\build\x86_64-pc-windows-msvc\stage1-tools\x86_64-pc-windows-msvc\release\miri.exe`

Caused by:
  Access is denied. (os error 5)
Command has failed. Rerun with -v to see more details.
  local time: Sat, Sep  7, 2024  1:35:54 AM
  network time: Sat, 07 Sep 2024 01:35:54 GMT
##[error]Process completed with exit code 1.
Post job cleanup.

@bors
Copy link
Contributor

bors commented Sep 7, 2024

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Sep 7, 2024
@lqd
Copy link
Member

lqd commented Sep 7, 2024

@bors retry #127883

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 7, 2024
@bors
Copy link
Contributor

bors commented Sep 7, 2024

⌛ Testing commit 12cda6e with merge b0cfc07...

bors added a commit to rust-lang-ci/rust that referenced this pull request Sep 7, 2024
…r-errors

Supress niches in coroutines to avoid aliasing violations

As mentioned [here](rust-lang#63818 (comment)), using niches in fields of coroutines that are referenced by other fields is unsound: the discriminant accesses violate the aliasing requirements of the reference pointing to the relevant field. This issue causes [Miri errors in practice](rust-lang/miri#3780).

The "obvious" fix for this is to suppress niches in coroutines. That's what this PR does. However, we have several tests explicitly ensuring that we *do* use niches in coroutines. So I see two options:
- We guard this behavior behind a `-Z` flag (that Miri will set by default). There is no known case of these aliasing violations causing miscompilations. But absence of evidence is not evidence of absence...
- (What this PR does right now.) We temporarily adjust the coroutine layout logic and the associated tests until the proper fix lands. The "proper fix" here is to wrap fields that other fields can point to in [`UnsafePinned`](rust-lang#125735) and make `UnsafePinned` suppress niches; that would then still permit using niches of *other* fields (those that never get borrowed). However, I know that coroutine sizes are already a problem, so I am not sure if this temporary size regression is acceptable.

`@compiler-errors` any opinion? Also who else should be Cc'd here?
@rust-log-analyzer
Copy link
Collaborator

The job x86_64-msvc-ext failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
[RUSTC-TIMING] miri test:false 4.461
error: failed to remove file `C:\a\rust\rust\build\x86_64-pc-windows-msvc\stage1-tools\x86_64-pc-windows-msvc\release\miri.exe`

Caused by:
  Access is denied. (os error 5)
Command has failed. Rerun with -v to see more details.
  local time: Sat, Sep  7, 2024  5:48:03 AM
  network time: Sat, 07 Sep 2024 05:48:03 GMT
##[error]Process completed with exit code 1.
Post job cleanup.

@bors
Copy link
Contributor

bors commented Sep 7, 2024

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Sep 7, 2024
@RalfJung
Copy link
Member Author

RalfJung commented Sep 7, 2024 via email

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 7, 2024
@bors
Copy link
Contributor

bors commented Sep 8, 2024

⌛ Testing commit 12cda6e with merge 7f4b270...

@bors
Copy link
Contributor

bors commented Sep 8, 2024

☀️ Test successful - checks-actions
Approved by: compiler-errors
Pushing 7f4b270 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Sep 8, 2024
@bors bors merged commit 7f4b270 into rust-lang:master Sep 8, 2024
7 checks passed
@rustbot rustbot added this to the 1.83.0 milestone Sep 8, 2024
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (7f4b270): comparison URL.

Overall result: ❌ regressions - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
0.5% [0.5%, 0.5%] 1
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) - - 0

Max RSS (memory usage)

Results (primary -0.9%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
2.6% [2.6%, 2.6%] 1
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-4.4% [-4.4%, -4.4%] 1
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) -0.9% [-4.4%, 2.6%] 2

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 757.221s -> 754.658s (-0.34%)
Artifact size: 341.00 MiB -> 341.04 MiB (0.01%)

@RalfJung RalfJung deleted the coroutine-niches branch September 9, 2024 08:27
github-merge-queue bot pushed a commit to model-checking/kani that referenced this pull request Sep 17, 2024
Relevant upstream PR:

rust-lang/rust#129313: Supress niches in
coroutines to avoid aliasing violations #129313

Resolves #3512

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 and MIT licenses.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants