Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Internal compiler docs is broken on master #44629

Closed
Michael-F-Bryan opened this issue Sep 16, 2017 · 7 comments
Closed

Internal compiler docs is broken on master #44629

Michael-F-Bryan opened this issue Sep 16, 2017 · 7 comments
Labels
C-bug Category: This is a bug. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.

Comments

@Michael-F-Bryan
Copy link

I don't think this has already been reported, but it looks like generating documentation for internal rustc crates is currently broken. This is probably because of an error in bootstrap or rustbuild where the wrong -L flags are being passed to rustdoc, because build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libproc_macro-6210e4b46662ec28.so (the file which it says is missing) already exists on my system.

According to my rustc-internal-docs tool, this breaks documentation generation for 32 of the ~60 internal crates in the compiler.


Steps to Reproduce

  1. Check out the latest version of the rustc repo
  2. Set up your config.toml as described in document internal rustc items
  3. Run ./x.py doc (possibly adding --stage 1 so you don't compile stage 2 things unnecessarily)

Output:

$ ./x.py doc --stage 1
...
 Documenting rustc_errors v0.0.0 (file:///srv/github-backups/rust-lang/rust/src/librustc_errors)
   Compiling string_cache v0.6.2
   Compiling markup5ever v0.3.2
   Compiling selectors v0.18.0
 Documenting cssparser v0.13.7
error: libproc_macro-6210e4b46662ec28.so: cannot open shared object file: No such file or directory
  --> /home/michael/.cargo/registry/src/github.com-1ecc6299db9ec823/cssparser-0.13.7/src/lib.rs:71:14
   |
71 | #[macro_use] extern crate cssparser_macros;
   |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: Could not compile `cssparser`.
warning: build failed, waiting for other jobs to finish...
error: build failed


command did not execute successfully: "/srv/github-backups/rust-lang/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "doc" "-j" "4" "--target" "x86_64-unknown-linux-gnu" "--release" "--features" " jemalloc llvm" "--manifest-path" "/srv/github-backups/rust-lang/rust/src/rustc/Cargo.toml" "-p" "rustc_driver" "-p" "rustdoc"
expected success, got: exit code: 101


failed to run: /srv/github-backups/rust-lang/rust/build/bootstrap/debug/bootstrap doc --stage 1
Build completed unsuccessfully in 0:28:03
@steveklabnik steveklabnik added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Sep 18, 2017
@Mark-Simulacrum Mark-Simulacrum added C-bug Category: This is a bug. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Sep 18, 2017
@Manishearth
Copy link
Member

Yeah I kept hitting this too, but at this stage the compiler doc build is more often broken than not so I didn't look too deeply.

IIRC proc macros don't work on stage1. Being able to skip to stage 2 and document that would be enough.

@kennytm
Copy link
Member

kennytm commented Oct 14, 2017

I tried to reproduce it on macOS. The error emitted was:

Documenting stage2 std (x86_64-apple-darwin)
    Finished release [optimized] target(s) in 0.0 secs
   Compiling cssparser v0.13.7
 Documenting rustc v0.0.0 (file:///./src/librustc)
error: dlopen(./build/x86_64-apple-darwin/stage1-rustc/release/deps/libcssparser_macros-7faf70d30b03767d.dylib, 1): Library not loaded: @rpath/libproc_macro-4a190002959d1e04.dylib
  Referenced from: ./build/x86_64-apple-darwin/stage1-rustc/release/deps/libcssparser_macros-7faf70d30b03767d.dylib
  Reason: image not found
  --> ~/.cargo/registry/src/github.com-1ecc6299db9ec823/cssparser-0.13.7/src/lib.rs:71:14
   |
71 | #[macro_use] extern crate cssparser_macros;
   |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: Could not compile `cssparser`.
warning: build failed, waiting for other jobs to finish...

The libproc_macro-4a190002959d1e04.dylib can be found in

./build/x86_64-apple-darwin/stage1/lib/rustlib/x86_64-apple-darwin/lib/libproc_macro-4a190002959d1e04.dylib
./build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/libproc_macro-4a190002959d1e04.dylib
./build/x86_64-apple-darwin/stage2/lib/libproc_macro-4a190002959d1e04.dylib
./build/x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/libproc_macro-4a190002959d1e04.dylib

so on macOS it should be something fixable by adjusting the rpath.

I'm not sure about the treatment on other OS.

@Manishearth
Copy link
Member

The problem is basically that rustc doesn't really handle the --extern thing well when dealing with dylibs. Clippy has the same problem; if you do a build and then run cargo test the tests sometimes fail for this reason.

@kennytm
Copy link
Member

kennytm commented Oct 14, 2017

I'm trying to see if just adding DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:«sysroot»/lib/rustlib/«target»/lib before the parsing phase will help.

EDIT: Currently I'm applying #13857 to all OS.

@ollie27
Copy link
Member

ollie27 commented Oct 14, 2017

So the issue here as I understand it is that since #41991 rustdoc can't be built at stage 1. Due to

.arg("-p").arg("rustdoc");
rustbuild is trying to build the rustdoc docs with the rustc docs but that hasn't made sense since #43482 split up rustdoc and rustc.

@Michael-F-Bryan
Copy link
Author

I just read #45366 and it sounds like the internal docs should build again. Did I get that right, @ollie27?

I'll run my rustc-internal-docs program again when #45366 gets merged to see how it goes.

@ollie27
Copy link
Member

ollie27 commented Oct 21, 2017

@Michael-F-Bryan yeah, the compiler docs are building fine for me with #45366.

bors added a commit that referenced this issue Oct 21, 2017
rustbuild: Don't try to build rustdoc API docs with compiler docs

rustdoc is built separately to rustc now so the docs would need to be
generated separately as well. Also rustdoc doesn't build at stage 1
which prevented the compiler docs being built at stage 1.

Fixes: #44629
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Category: This is a bug. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

6 participants