Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add warnings as errors for ex_doc #8596

Merged
merged 5 commits into from
Jun 25, 2024
Merged

Add warnings as errors for ex_doc #8596

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
3f2487d
Fix ex_doc warnings
garazdawi Jun 19, 2024
8d18ce9
otp: Add support for warnings as errors for ex_doc
garazdawi Jun 19, 2024
3c3d26b
Use escript to launch ex_doc for nix to work
garazdawi Jun 19, 2024
0a5025e
Read ex_doc from environment if set
garazdawi Jun 20, 2024
75913d7
Update gitignore
garazdawi Jun 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ JAVADOC-GENERATED
/doc

/make/output.mk
/make/emd2exml
/make/ex_doc_wrapper
/make/make_emakefile
/make/install_dir_data.sh

Expand Down
6 changes: 5 additions & 1 deletion erts/configure
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7331,7 +7331,6 @@ fi
done

if test -z "$EX_DOC"; then
EX_DOC="$ERL_TOP/make/ex_doc_wrapper"
{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: WARNING: No 'ex_doc' command found: it will (optionally) be downloaded when building the HTML documentation " >&5
printf "%s\n" "$as_me: WARNING: No 'ex_doc' command found: it will (optionally) be downloaded when building the HTML documentation " >&2;}
fi
Expand Down Expand Up @@ -26903,6 +26902,9 @@ ac_config_files="$ac_config_files emulator/$host/Makefile:emulator/Makefile.in e
ac_config_files="$ac_config_files ../make/make_emakefile:../make/make_emakefile.in"


ac_config_files="$ac_config_files ../make/ex_doc_wrapper:../make/ex_doc_wrapper.in"


ac_config_files="$ac_config_files ../lib/os_mon/c_src/$host/Makefile:../lib/os_mon/c_src/Makefile.in ../lib/runtime_tools/c_src/$host/Makefile:../lib/runtime_tools/c_src/Makefile.in"


Expand Down Expand Up @@ -27616,6 +27618,7 @@ do
"lib_src/$host/Makefile") CONFIG_FILES="$CONFIG_FILES lib_src/$host/Makefile:lib_src/Makefile.in" ;;
"../make/$host/otp.mk") CONFIG_FILES="$CONFIG_FILES ../make/$host/otp.mk:../make/otp.mk.in" ;;
"../make/make_emakefile") CONFIG_FILES="$CONFIG_FILES ../make/make_emakefile:../make/make_emakefile.in" ;;
"../make/ex_doc_wrapper") CONFIG_FILES="$CONFIG_FILES ../make/ex_doc_wrapper:../make/ex_doc_wrapper.in" ;;
"../lib/os_mon/c_src/$host/Makefile") CONFIG_FILES="$CONFIG_FILES ../lib/os_mon/c_src/$host/Makefile:../lib/os_mon/c_src/Makefile.in" ;;
"../lib/runtime_tools/c_src/$host/Makefile") CONFIG_FILES="$CONFIG_FILES ../lib/runtime_tools/c_src/$host/Makefile:../lib/runtime_tools/c_src/Makefile.in" ;;
"../make/install_dir_data.sh") CONFIG_FILES="$CONFIG_FILES ../make/install_dir_data.sh:../make/install_dir_data.sh.in" ;;
Expand Down Expand Up @@ -28172,6 +28175,7 @@ printf "%s\n" "$as_me: $ac_file is unchanged" >&6;}

case $ac_file$ac_mode in
"../make/make_emakefile":F) chmod +x ../make/make_emakefile ;;
"../make/ex_doc_wrapper":F) chmod +x ../make/ex_doc_wrapper ;;
"../make/install_dir_data.sh":F) chmod +x ../make/install_dir_data.sh ;;

esac
Expand Down
4 changes: 3 additions & 1 deletion erts/configure.ac
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -474,7 +474,6 @@ fi

AC_CHECK_PROGS(EX_DOC, ex_doc)
if test -z "$EX_DOC"; then
EX_DOC="$ERL_TOP/make/ex_doc_wrapper"
AC_MSG_WARN([No 'ex_doc' command found: it will (optionally) be downloaded when building the HTML documentation ])
fi

Expand Down Expand Up @@ -3677,6 +3676,9 @@ AC_CONFIG_FILES([
AC_CONFIG_FILES([../make/make_emakefile:../make/make_emakefile.in],
[chmod +x ../make/make_emakefile])

AC_CONFIG_FILES([../make/ex_doc_wrapper:../make/ex_doc_wrapper.in],
[chmod +x ../make/ex_doc_wrapper])

dnl
dnl The ones below should be moved to their respective lib
dnl
Expand Down
1 change: 1 addition & 0 deletions lib/common_test/doc/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ include $(ERL_TOP)/make/$(TARGET)/otp.mk
include ../vsn.mk
VSN=$(COMMON_TEST_VSN)
APPLICATION=common_test
EX_DOC_WARNINGS_AS_ERRORS=false

# ----------------------------------------------------
# Release Target
Expand Down
2 changes: 1 addition & 1 deletion lib/tftp/src/tftp.erl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ This is a complete implementation of the following IETF standards:
The only feature that not is implemented in this release is
the "netascii" transfer mode.

The [start/](`start/3`)function starts a daemon process which, listens
The [start](`start/1`) function starts a daemon process which, listens
for UDP packets on a port. When it receives a request for read or
write it spawns a temporary server process which handles the actual
transfer of the file. On the client side the
Expand Down
8 changes: 5 additions & 3 deletions make/doc.mk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,15 +52,17 @@ else
DOC_TARGETS?=html
endif

EX_DOC_WARNINGS_AS_ERRORS?=true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last question, should this be false by default?

I immediately had to turn it off in the nix build. Which worked by the way! Here is the simplification I was able to do with this https://github.com/chiroptical/nixpkgs/pull/4/files.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No I want it to be on by default as there should never be any warnings if you use the correct ExDoc vsn.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get this from the build with that set to true.

warning: the VM is running with native name encoding of latin1 which may cause Elixir to malfunction as it expects utf8. Please ensure your locale is set to UTF-8 (which can be verified by running "locale" in your shell) or set the ELIXIR_ERL_OPTIONS="+fnu" environment variable

I guess you may want to set this in the wrapper then?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be something that needs fixing in nix I think. The Erlang VM thinks that filenames should be encoded in latin1 because the locale is not set to utf8.

The elixir nix package seems to set LANG and LC_TYPE to C.UTF-8 which should do the trick.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess so! Really appreciate you looking through all this.


docs: $(DOC_TARGETS)

chunks:

HTML_DEPS?=$(wildcard $(APP_EBIN_DIR)/*.beam) $(wildcard *.md) $(wildcard */*.md) $(wildcard assets/*)

$(HTMLDIR)/index.html: $(HTML_DEPS) docs.exs
$(gen_verbose)ERL_FLAGS="-pz $(ERL_TOP)/erts/ebin" \
$(EX_DOC) $(EX_DOC_FORMATS) --homepage-url "$(INDEX_DIR)/index.html" "$(APPLICATION)" $(VSN) $(APP_EBIN_DIR) -o "$(HTMLDIR)" -c $(ERL_TOP)/make/ex_doc.exs
$(HTMLDIR)/index.html: $(HTML_DEPS) docs.exs $(ERL_TOP)/make/ex_doc.exs
$(gen_verbose)EX_DOC_WARNINGS_AS_ERRORS=$(EX_DOC_WARNINGS_AS_ERRORS) ERL_FLAGS="-pz $(ERL_TOP)/erts/ebin" \
$(ERL_TOP)/make/ex_doc_wrapper $(EX_DOC_FORMATS) --homepage-url "$(INDEX_DIR)/index.html" "$(APPLICATION)" $(VSN) $(APP_EBIN_DIR) -o "$(HTMLDIR)" -c $(ERL_TOP)/make/ex_doc.exs

html: $(HTMLDIR)/index.html

Expand Down
2 changes: 1 addition & 1 deletion make/ex_doc.exs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ current_datetime = System.os_time() |> DateTime.from_unix!(:native)
config = [
proglang: :erlang,
source_url_pattern: source_url_pattern,
assets: Path.join(cwd, "/assets"),
assets: %{ Path.join(cwd, "/assets") => "assets" },
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Next error I get from the nix build

warning: the VM is running with native name encoding of latin1 which may cause Elixir to malfunction as it expects utf8. Please ensure your locale is set to UTF-8 (which can be verified by running "locale" in your shell) or set the ELIXIR_ERL_OPTIONS="+fnu" environment variable
** (FunctionClauseError) no function clause matching in IO.chardata_to_string/1    
    
    The following arguments were given to IO.chardata_to_string/1:
    
        # 1
        %{"/build/source/lib/stdlib/doc/assets" => "assets"}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stack trace

    (elixir 1.16.3) lib/io.ex:687: IO.chardata_to_string/1
    (elixir 1.16.3) lib/path.ex:662: Path.join/2
    (ex_doc 0.32.1) lib/ex_doc/formatter/html.ex:294: ExDoc.Formatter.HTML.copy_assets/2
    (ex_doc 0.32.1) lib/ex_doc/formatter/html.ex:288: ExDoc.Formatter.HTML.generate_assets/3
    (ex_doc 0.32.1) lib/ex_doc/formatter/html.ex:25: ExDoc.Formatter.HTML.run/3
    (ex_doc 0.32.1) lib/ex_doc/cli.ex:69: anonymous fn/6 in ExDoc.CLI.generate/3
    (elixir 1.16.3) lib/enum.ex:1700: Enum."-map/2-lists^map/1-1-"/2
    (elixir 1.16.3) lib/kernel/cli.ex:136: anonymous fn/3 in Kernel.CLI.exec_fun/2

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this comes from https://github.com/elixir-lang/ex_doc/blob/v0.34.0/CHANGELOG.md#v0340-2024-05-30? i.e. I think this requires a specific version of ex_doc now. Unsure if that is intended?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will always require a specific version of ex_doc, which one is listed in https://github.com/erlang/otp/blob/master/make/ex_doc_vsn

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, okay! Let me update that and keep testing thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some bugs in v0.34.0 of ex_doc that cause some issues, so you may have to wait for v0.35.0 in order for everything to pass.

logo: Path.join(:code.root_dir(), "system/doc/assets/erlang-logo.png"),
before_closing_head_tag: fn _ -> "<style>.dark img { background-color: white; }</style>" end,
before_closing_footer_tag: fn _ ->
Expand Down
25 changes: 0 additions & 25 deletions make/ex_doc_wrapper
View file
Open in desktop

This file was deleted.

52 changes: 52 additions & 0 deletions make/ex_doc_wrapper.in
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
#!/bin/bash

ARGS=("$@")

set -eo pipefail {0}

## If EX_DOC is not set to a file, we search the PATH for it using command -v
if [ ! -f "${EX_DOC}" ]; then
EX_DOC=$(command -v ex_doc || true)
fi

if [ -z "${EX_DOC}" ]; then
echo -n "Could not find ex_doc! "
read -p "Do you want to download latest ex_doc from github? (y/n)? " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]
then
if $ERL_TOP/otp_build download_ex_doc; then
read -p "Press any key to continue..." -n 1 -r
echo "continuing"
EX_DOC=$(command -v ex_doc || true)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is fine because EX_DOC wouldn't have been defined previously

else
exit 1
fi
else
exit 1
fi
fi

## The below bash magic captures the output of stderr into OUTPUT while still printing
## everything we get to stdout and stderr. This is done by:

## 1. duplicating the stdout (1) and stderr (2) streams to fd 3 and 4 respectively.
exec 3>&1 4>&2

## Running the command where we redirect stderr to fd 1 and stdout to fd 3.
## We then use tee on the stderr (which is now fd 1) to print that to fd 4
OUTPUT="$( { escript@EXEEXT@ "${EX_DOC}" "${ARGS[@]}"; } 2>&1 1>&3 | tee /dev/fd/4 )"

## Close fd 3 and 4
exec 3>&- 4>&-

if [ "${EX_DOC_WARNINGS_AS_ERRORS}" != "false" ]; then
if echo "${OUTPUT}" | grep "warning:" 1>/dev/null; then
echo "ex_doc emitted warnings"
## Touch the config file in order to re-trigger make
if [ -f "docs.exs" ]; then
touch "docs.exs"
fi
exit 1;
fi
fi
1 change: 1 addition & 0 deletions system/doc/top/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ testing
documentation
system
man_index.md
README.md
Loading