Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

build: add GitHub Action to update tools modules #40644

Closed
wants to merge 2 commits into from
Closed

build: add GitHub Action to update tools modules #40644

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e8d16ee
build: add GitHub Action to update tools modules
Trott Oct 29, 2021
04451d8
tools: improve update scripts
Trott Oct 30, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions .github/workflows/tools.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: "tools update"
on:
schedule:
# Run once a week at 00:05 AM UTC on Saturday.
- cron: '5 0 * * 6'

workflow_dispatch:

jobs:
tools_update:
if: github.repository == 'nodejs/node'
runs-on: ubuntu-latest
strategy:
fail-fast: false # Prevent other jobs from aborting if one fails
matrix:
include:
- id: eslint
run: |
cd tools
NEW_VERSION=$(npm view eslint dist-tags.latest)
CURRENT_VERSION=$(node -p "require('./node_modules/eslint/package.json').version")
if [ "$NEW_VERSION" != "$CURRENT_VERSION" ]; then
echo "NEW_VERSION=$NEW_VERSION" >> $GITHUB_ENV
tools/update-eslint.sh
fi
- id: "@babel/eslint-parser"
run: |
cd tools
NEW_VERSION=$(npm view @babel/eslint-parser dist-tags.latest)
CURRENT_VERSION=$(node -p "require('./node_modules/@babel/eslint-parser/package.json').version")
if [ "$NEW_VERSION" != "$CURRENT_VERSION" ]; then
echo "NEW_VERSION=$NEW_VERSION" >> $GITHUB_ENV
tools/update-babel-eslint.sh
fi
- id: "lint-md dependencies"
run: |
cd tools/lint-md
NEW_VERSION=$(npm outdated --omit=dev --parseable | cut -d: -f4 | xargs)
if [ "$NEW_VERSION" != "" ]; then
echo "NEW_VERSION=$NEW_VERSION" >> $GITHUB_ENV
rm -rf package-lock.json node_modules && npm install --ignore-scripts)
make lint-md-rollup
fi
steps:
- uses: actions/checkout@v2
- run: ${{ matrix.run }}
- uses: gr2m/create-or-update-pull-request-action@v1 # Create a PR or update the Action's existing PR
env:
GITHUB_TOKEN: ${{ secrets.GH_USER_TOKEN }}
with:
author: Node.js GitHub Bot <github-bot@iojs.org>
body: "This is an automated update of ${{ matrix.id }} to ${{ env.NEW_VERSION }}."
Copy link
Contributor

@Mesteery Mesteery Nov 3, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally I was thinking of something like

This is an automated update of eslint and its dependencies. Updated:
`eslint@x.y.z` to `x.y.z1`
`eslint-plugin-markdown@x.y.z` to `x.y.z1`

or

This is an automated update of eslint and its dependencies. Updated:
`eslint-plugin-markdown@x.y.z` to `x.y.z1`
matrix:
  include:
    - id: eslint
      packages: [eslint, eslint-plugin-markdown] # or '["eslint",...]', I don't know if Actions stringifies this
      before: cd tools
      update: tools/update-eslint.sh
  - run: |
      ${{ matrix.before }}
      NEW_VERSIONS=$(npm outdated --json | jq -r '.[] | select(.current != .wanted and .package as $p | ${{ matrix.packages }} | index($p)) | "`" + .package + "@" + .current + "` to `" + .wanted + "`"')
      if [ "$NEW_VERSIONS" != "" ]; then
        echo "NEW_VERSIONS=$NEW_VERSIONS" >> $GITHUB_ENV
        ${{ matrix.update }}
      fi

  - if: env.NEW_VERSIONS
    ...
    body: |
      This is an automated update of ${{ matrix.id }} and its dependencies. Updated:
      ${{ env.NEW_VERSIONS }}

Copy link
Member Author

@Trott Trott Nov 4, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea, but there a few small obstacles:

  1. npm outdated will not work for eslint or @babel/eslint-parser the way we have them installed. That's why I'm doing things the way I am in those instances.
  2. I don't think the passing of a multiline value for the environment variable works in GitHub Actions. (I could be wrong about that, but that's what I seem to recall. If $NEW_VERSIONS contains newline characters, then "NEW_VERSIONS=$NEW_VERSIONS" won't work.)

For lint-md, the format you propose would be better than what I'm doing, as the way I'm doing it will result in lines longer than 72 characters frequently.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2. I don't think the passing of a multiline value for the environment variable works in GitHub Actions. (I could be wrong about that, but that's what I seem to recall. If $NEW_VERSIONS contains newline characters, then "NEW_VERSIONS=$NEW_VERSIONS" won't work.)

I suppose I can use some other character as a placeholder for the newline and then convert it in the relevant step.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the new versions are in a string with a format like "abc@1.2.3 xyz@4.5.6 foobar@42.0.0" then I can convert the spaces to newlines and the problem is solved.

Copy link
Contributor

@Mesteery Mesteery Nov 4, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

npm outdated will not work for eslint or @babel/eslint-parser the way we have them installed. That's why I'm doing things the way I am in those instances.

Ah

  1. I don't think the passing of a multiline value for the environment variable works in GitHub Actions. (I could be wrong about that, but that's what I seem to recall. If $NEW_VERSIONS contains newline characters, then "NEW_VERSIONS=$NEW_VERSIONS" won't work.)

Indeed, but it is possible by using outputs and escaping new lines. The runner will automatically take care of unescaping it:

  - run: |
      ...
        echo "::set-output name=version::${NEW_VERSIONS//$'\n'/'%0A'}"
      ...
    id: update

And then use it as ${{ steps.update.outputs.version }}

branch: "actions/tools-update-${{ matrix.id }}" # Custom branch *just* for this Action.
commit-message: "tools: update ${{ matrix.id }} to ${{ env.NEW_VERSION }}"
labels: tools
title: "tools: update ${{ matrix.id }} to ${{ env.NEW_VERSION }}"
2 changes: 1 addition & 1 deletion tools/update-babel-eslint.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ ROOT="$PWD/../.."
NPM="$ROOT/deps/npm/bin/npm-cli.js"

"$NODE" "$NPM" init --yes
"$NODE" "$NPM" install --global-style --no-bin-links --production --no-package-lock @babel/core @babel/eslint-parser@latest @babel/plugin-syntax-import-assertions@latest
"$NODE" "$NPM" install --global-style --no-bin-links --ignore-scripts --no-package-lock @babel/core @babel/eslint-parser @babel/plugin-syntax-import-assertions

# Use dmn to remove some unneeded files.
"$NODE" "$NPM" exec -- dmn@2.2.2 -f clean
Expand Down
3 changes: 1 addition & 2 deletions tools/update-eslint.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,7 @@ rm -rf node_modules/eslint node_modules/eslint-plugin-markdown

"$NODE" "$NPM" init --yes

"$NODE" "$NPM" install --global-style --no-bin-links --production --no-package-lock eslint@latest
"$NODE" "$NPM" install --global-style --no-bin-links --production --no-package-lock eslint-plugin-markdown@latest
"$NODE" "$NPM" install --global-style --no-bin-links --ignore-scripts --no-package-lock eslint eslint-plugin-markdown

# Use dmn to remove some unneeded files.
"$NODE" "$NPM" exec -- dmn@2.2.2 -f clean
Expand Down