resolve: Implement prelude search for macro paths, implement tool attributes #52841

petrochenkov · 2018-07-30T00:52:26Z

When identifier is macro path is resolved in scopes (i.e. the first path segment - foo in foo::mac!() or foo!()), scopes are searched in the same order as for non-macro paths - items in modules, extern prelude, tool prelude (see later), standard library prelude, language prelude, but with some extra shadowing restrictions (names from globs and macro expansions cannot shadow names from outer scopes). See the comment in fn resolve_lexical_macro_path_segment for more details.

"Tool prelude" currently contains two "tool modules" rustfmt and clippy, and is searched immediately after extern prelude.
This makes the possible long-term solution for tool attributes exactly equivalent to the existing extern prelude scheme, except that --extern=my_crate making crate names available in scope is replaced with something like --tool=my_tool making tool names available in scope.

The tool_attributes feature is still unstable and #![feature(tool_attributes)] now implicitly enables #![feature(use_extern_macros)]. use_extern_macros is a prerequisite for tool_attributes, so their stabilization will happen in the same order.
If use_extern_macros is not enabled, then tool attributes are treated as custom attributes (this is temporary, anyway).

Fixes #52576
Fixes #52512
Fixes #51277
cc #52269

rust-highfive · 2018-07-30T00:52:29Z

r? @cramertj

(rust_highfive has picked a reviewer for you, use r? to override)

petrochenkov · 2018-07-30T00:54:17Z

r? @alexcrichton
cc @nrc @djrenren

petrochenkov · 2018-07-30T00:56:52Z

src/librustc_resolve/macros.rs

+                        Err(Determinacy::Determined)
+                    }
+                }
+                WhereToResolve::StdLibPrelude => {


~~FIXME: account for no_implicit_prelude.~~
Done.

rust-highfive · 2018-07-30T02:12:23Z

The job x86_64-gnu-llvm-5.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[01:18:25] 
[01:18:25] failures:
[01:18:25] 
[01:18:25] ---- /checkout/src/doc/unstable-book/src/language-features/tool-attributes.md - tool_attributes::An_example (line 17) stdout ----
[01:18:25] error[E0658]: The attribute `rustfmt::skip` is currently unknown to the compiler and may have meaning added to it in the future (see issue #29642)
[01:18:25]   |
[01:18:25] 4 | #[rustfmt::skip]
[01:18:25]   | ^^^^^^^^^^^^^^^^
[01:18:25]   |
[01:18:25]   |
[01:18:25]   = help: add #![feature(custom_attribute)] to the crate attributes to enable
[01:18:25] thread '/checkout/src/doc/unstable-book/src/language-features/tool-attributes.md - tool_attributes::An_example (line 17)' panicked at 'couldn't compile the test', librustdoc/test.rs:332:13
[01:18:25] note: Run with `RUST_BACKTRACE=1` for a backtrace.
[01:18:25] 
[01:18:25]

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

alexcrichton · 2018-07-30T14:26:06Z

Looks great to me, thanks for the detailed comments!

It looks like tool attributes like #[rustfmt::skip] are still gated by default, and otherwise it looks like this is just fixing bugs, so r=me!

petrochenkov · 2018-07-30T22:01:59Z

@bors r=alexcrichton

bors · 2018-07-30T22:02:00Z

📌 Commit dd93535e028ece7f01491fd25c8369d375026f69 has been approved by alexcrichton

Mark-Simulacrum · 2018-07-31T17:59:04Z

@bors p=1 edition critical

bors · 2018-08-01T01:24:20Z

🔒 Merge conflict

This pull request and the master branch diverged in a way that cannot be automatically merged. Please rebase on top of the latest master branch, and let the reviewer approve again.

How do I rebase?

Assuming self is your fork and upstream is this repository, you can resolve the conflict following these steps:

git checkout premacro (switch to your branch)
git fetch upstream master (retrieve the latest master)
git rebase upstream/master -p (rebase on top of it)
Follow the on-screen instruction to resolve conflicts (check git status if you got lost).
git push self premacro --force-with-lease (update this PR)

You may also read Git Rebasing to Resolve Conflicts by Drew Blessing for a short tutorial.

Please avoid the "Resolve conflicts" button on GitHub. It uses git merge instead of git rebase which makes the PR commit history more difficult to read.

Sometimes step 4 will complete without asking for resolution. This is usually due to difference between how Cargo.lock conflict is handled during merge and rebase. This is normal, and you should still perform step 5 to update this PR.

Error message

Auto-merging src/test/ui/macro-path-prelude-pass.rs
Auto-merging src/test/ui/macro-path-prelude-fail-1.rs
Auto-merging src/librustc_resolve/macros.rs
CONFLICT (content): Merge conflict in src/librustc_resolve/macros.rs
Auto-merging src/librustc_resolve/lib.rs
Auto-merging src/librustc_resolve/build_reduced_graph.rs
Automatic merge failed; fix conflicts and then commit the result.

pietroalbini · 2018-08-01T08:48:26Z

This is also blocking edition crater runs.

resolve/expansion: Implement tool attributes

petrochenkov · 2018-08-01T10:15:43Z

@bors r=alexcrichton

bors · 2018-08-01T10:15:43Z

📌 Commit c3e5421 has been approved by alexcrichton

bors · 2018-08-01T10:42:29Z

⌛ Testing commit c3e5421 with merge e53bca8baf30e0686c3b2b82af3db0be0d592a64...

pietroalbini · 2018-08-02T19:37:48Z

@bors r=alexcrichton

bors · 2018-08-02T19:37:49Z

📌 Commit c3e5421 has been approved by alexcrichton

pietroalbini · 2018-08-02T19:38:20Z

@bors p=1

alexcrichton · 2018-08-02T21:21:43Z

@bors: rollup-

bors · 2018-08-02T21:39:20Z

⌛ Testing commit c3e5421 with merge 40e4b6e...

resolve: Implement prelude search for macro paths, implement tool attributes When identifier is macro path is resolved in scopes (i.e. the first path segment - `foo` in `foo::mac!()` or `foo!()`), scopes are searched in the same order as for non-macro paths - items in modules, extern prelude, tool prelude (see later), standard library prelude, language prelude, but with some extra shadowing restrictions (names from globs and macro expansions cannot shadow names from outer scopes). See the comment in `fn resolve_lexical_macro_path_segment` for more details. "Tool prelude" currently contains two "tool modules" `rustfmt` and `clippy`, and is searched immediately after extern prelude. This makes the [possible long-term solution](https://github.com/rust-lang/rfcs/blob/master/text/2103-tool-attributes.md#long-term-solution) for tool attributes exactly equivalent to the existing extern prelude scheme, except that `--extern=my_crate` making crate names available in scope is replaced with something like `--tool=my_tool` making tool names available in scope. The `tool_attributes` feature is still unstable and `#![feature(tool_attributes)]` now implicitly enables `#![feature(use_extern_macros)]`. `use_extern_macros` is a prerequisite for `tool_attributes`, so their stabilization will happen in the same order. If `use_extern_macros` is not enabled, then tool attributes are treated as custom attributes (this is temporary, anyway). Fixes #52576 Fixes #52512 Fixes #51277 cc #52269

bors · 2018-08-02T23:43:12Z

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing 40e4b6e to master...

Manishearth · 2018-08-03T20:08:41Z

This breaks Servo in a weird way. I'm unable to reproduce this out of tree

https://travis-ci.org/servo/servo-with-rust-nightly/jobs/411851355

error[E0659]: `warn` is ambiguous
   --> ports/servo/glutin_app/window.rs:733:21
    |
733 |                     warn!("Failed to swap window buffers ({}).", err);
    |                     ^^^^
    |
note: `warn` could refer to the name imported here
   --> ports/servo/non_android_main.rs:12:1
    |
12  | #[macro_use] extern crate log;
    | ^^^^^^^^^^^^
note: `warn` could also refer to the name defined here
   --> ports/servo/glutin_app/window.rs:733:21
    |
733 |                     warn!("Failed to swap window buffers ({}).", err);
    |                     ^^^^
    = note: macro-expanded macro imports do not shadow

any idea what's going on? seems like a regression

Manishearth · 2018-08-03T20:09:23Z

cc @pietroalbini might want to be careful about that regression before cutting a beta ^^

petrochenkov · 2018-08-03T20:54:33Z

I'll look what happens.

petrochenkov · 2018-08-05T13:30:34Z

@Manishearth
This is an expected breakage.
The general rule sounds like "names from globs and macro expansions cannot shadow names from outer scopes when macro paths are resolved".
Without this rule macro expansion would almost always get stuck in scopes with unexpanded macros or globs without being able to proceed.

In this case macro expanded name (macro_rules macro warn from #[macro_use] extern crate log; inside include!() expansion) shadows a name from outer scope (built-in attribute warn from the language prelude), and this happens during resolution of the macro path warn in warn!("Failed to swap window buffers ({}).", err);.

Minimized reproduction:

macro_rules! my_include {() => {
    #[macro_use] extern crate log;
}}

my_include!();

fn main() {
    warn!("");
}

In this case, since breakage affects stable channel, built-in attributes can be somehow special-cased to avoid the error, or perhaps some more general solution can be found, but I'd prefer to see crater results before proceeding with a fix.

petrochenkov · 2018-08-05T13:34:17Z

Since #53072 doesn't include this PR, we need a separate crater run for it.

Mark-Simulacrum · 2018-08-05T13:38:40Z

Queued a crater run for this PR (check only) here: #53089

nnethercote · 2018-08-17T05:26:34Z

Good news! This PR improved compile speed on a few benchmarks, the best by 3.7%:
https://perf.rust-lang.org/compare.html?start=40cb4478a3f550bf12d81cdcf08cc9ef3985ed41&end=40e4b6ee3dd70a05007915bd1c15c150a7b7899f&stat=instructions:u

Stabilize a few secondary macro features - `tool_attributes` - closes rust-lang#44690 - `proc_macro_path_invoc` - this feature was created due to issues with tool attributes (rust-lang#51277), those issues are now fixed (rust-lang#52841) - partially `proc_macro_gen` - this feature was created due to issue rust-lang#50504, the issue is now fixed (rust-lang#51952), so proc macros can generate modules. They still can't generate `macro_rules` items though due to unclear hygiene interactions.

Stabilize a few secondary macro features - `tool_attributes` - closes #44690 - `proc_macro_path_invoc` - this feature was created due to issues with tool attributes (#51277), those issues are now fixed (#52841) - partially `proc_macro_gen` - this feature was created due to issue #50504, the issue is now fixed (#51952), so proc macros can generate modules. They still can't generate `macro_rules` items though due to unclear hygiene interactions.

Resolves rustc error E0659 in the base.rs example caused by [rust-lang/rust](https://github.com/rust-lang/rust) pull request [#52841](rust-lang/rust#52841) "resolve: Implement prelude search for macro paths, implement tool attributes."

Resolves nightly rustc error E0659 in the base.rs example caused by [rust-lang/rust](https://github.com/rust-lang/rust) pull request [#52841](rust-lang/rust#52841) "resolve: Implement prelude search for macro paths, implement tool attributes."

rust-highfive assigned cramertj Jul 30, 2018

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 30, 2018

rust-highfive assigned alexcrichton and unassigned cramertj Jul 30, 2018

petrochenkov mentioned this pull request Jul 30, 2018

tool_attributes feature stops working when rust_2018_preview feature is enabled #51277

Closed

petrochenkov commented Jul 30, 2018

View reviewed changes

mati865 mentioned this pull request Jul 30, 2018

Replace cfg_attr(rustfmt... with rustfmt::skip rust-lang/rust-clippy#2823

Merged

petrochenkov force-pushed the premacro branch from d28b1c6 to dd93535 Compare July 30, 2018 22:00

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 30, 2018

petrochenkov mentioned this pull request Jul 31, 2018

#[feature(uniform_paths)]: allow use x::y; to resolve through self::x, not just ::x. #52923

Merged

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 1, 2018

resolve: Implement prelude search for macro paths

c3e5421

resolve/expansion: Implement tool attributes

petrochenkov force-pushed the premacro branch from dd93535 to c3e5421 Compare August 1, 2018 10:15

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 1, 2018

bors merged commit c3e5421 into rust-lang:master Aug 2, 2018

alexcrichton added a commit to rust-lang/rust-clippy that referenced this pull request Aug 3, 2018

Fix fallout from rust-lang/rust#52841

8ef759e

petrochenkov mentioned this pull request Aug 5, 2018

1.29 first Crater run #53072

Closed

Mark-Simulacrum added the beta-nominated Nominated for backporting to the compiler in the beta channel. label Aug 5, 2018

petrochenkov removed the beta-nominated Nominated for backporting to the compiler in the beta channel. label Aug 5, 2018

mati865 mentioned this pull request Aug 8, 2018

Stabilize use_extern_macros #50911

Merged

SimonSapin mentioned this pull request Aug 8, 2018

Regression importing log’s warn! macro inside of include!()ed file #53205

Closed

petrochenkov mentioned this pull request Aug 17, 2018

Stabilize a few secondary macro features #53459

Merged

dtolnay mentioned this pull request Jan 2, 2019

Allow shadowing of procedural macro generated macros #57283

Closed

ehuss mentioned this pull request Apr 21, 2019

Document name resolution. rust-lang/reference#568

Open

29 tasks

petrochenkov deleted the premacro branch June 5, 2019 16:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

resolve: Implement prelude search for macro paths, implement tool attributes #52841

resolve: Implement prelude search for macro paths, implement tool attributes #52841

petrochenkov commented Jul 30, 2018 •

edited

Loading

rust-highfive commented Jul 30, 2018

petrochenkov commented Jul 30, 2018

petrochenkov Jul 30, 2018 •

edited

Loading

rust-highfive commented Jul 30, 2018

alexcrichton commented Jul 30, 2018

petrochenkov commented Jul 30, 2018

bors commented Jul 30, 2018

Mark-Simulacrum commented Jul 31, 2018

bors commented Aug 1, 2018

pietroalbini commented Aug 1, 2018

petrochenkov commented Aug 1, 2018

bors commented Aug 1, 2018

bors commented Aug 1, 2018

pietroalbini commented Aug 2, 2018

bors commented Aug 2, 2018

pietroalbini commented Aug 2, 2018

alexcrichton commented Aug 2, 2018

bors commented Aug 2, 2018

bors commented Aug 2, 2018

Manishearth commented Aug 3, 2018

Manishearth commented Aug 3, 2018

petrochenkov commented Aug 3, 2018

petrochenkov commented Aug 5, 2018

petrochenkov commented Aug 5, 2018

Mark-Simulacrum commented Aug 5, 2018

nnethercote commented Aug 17, 2018

resolve: Implement prelude search for macro paths, implement tool attributes #52841

resolve: Implement prelude search for macro paths, implement tool attributes #52841

Conversation

petrochenkov commented Jul 30, 2018 • edited Loading

rust-highfive commented Jul 30, 2018

petrochenkov commented Jul 30, 2018

petrochenkov Jul 30, 2018 • edited Loading

Choose a reason for hiding this comment

rust-highfive commented Jul 30, 2018

alexcrichton commented Jul 30, 2018

petrochenkov commented Jul 30, 2018

bors commented Jul 30, 2018

Mark-Simulacrum commented Jul 31, 2018

bors commented Aug 1, 2018

pietroalbini commented Aug 1, 2018

petrochenkov commented Aug 1, 2018

bors commented Aug 1, 2018

bors commented Aug 1, 2018

pietroalbini commented Aug 2, 2018

bors commented Aug 2, 2018

pietroalbini commented Aug 2, 2018

alexcrichton commented Aug 2, 2018

bors commented Aug 2, 2018

bors commented Aug 2, 2018

Manishearth commented Aug 3, 2018

Manishearth commented Aug 3, 2018

petrochenkov commented Aug 3, 2018

petrochenkov commented Aug 5, 2018

petrochenkov commented Aug 5, 2018

Mark-Simulacrum commented Aug 5, 2018

nnethercote commented Aug 17, 2018

petrochenkov commented Jul 30, 2018 •

edited

Loading

petrochenkov Jul 30, 2018 •

edited

Loading