npm: Cached dependencies can be found under node18, but not node20

[threema-danilo]

flatpak-builder version

1.4.2

Linux distribution and version

Arch Linux x64

Affected flatpak-builder tool

node/flatpak-node-generator.py

flatpak-builder tool cli args

--force-clean --ccache --repo=$REPO --state-dir=$STATE --default-branch=master build $MANIFEST

Source repository URL

No response

Flatpak-builder manifest URL

No response

Description

The Threema 2.0 for Desktop project (not yet open sourced, will be soon-ish) currently uses node 18 / npm 9:

Manifest File

app-id: ch.threema.threema-desktop
runtime: org.freedesktop.Platform
runtime-version: '23.08'
sdk: org.freedesktop.Sdk
base: org.electronjs.Electron2.BaseApp
base-version: '23.08'
sdk-extensions:
  - org.freedesktop.Sdk.Extension.node18
command: run.sh
separate-locales: false
finish-args:
  # Share IPC namespace with the host (improves X11 performance)
  - --share=ipc
  # Show window using Wayland or X11
  - --socket=wayland
  - --socket=x11
  # Play sound using pulseaudio
  - --socket=pulseaudio
  # Allow access to network
  - --share=network
  # OpenGL rendering
  - --device=dri
  # Access directories
  - --filesystem=xdg-download
  # Allow desktop notifications
  - --talk-name=org.freedesktop.Notifications
build-options:
  append-path: /usr/lib/sdk/node18/bin
  cflags: -O2 -g
  cxxflags: -O2 -g
  env:
    NPM_CONFIG_LOGLEVEL: info
modules:
  - name: threema-desktop
    buildsystem: simple
    build-options:
      env:
        XDG_CACHE_HOME: /run/build/threema-desktop/flatpak-node/cache
        npm_config_cache: /run/build/threema-desktop/flatpak-node/npm-cache
        npm_config_offline: 'true'
      secret-env: ['SENTRY_DSN', 'MINIDUMP_ENDPOINT']
    sources:
      # Project source code
      - type: dir
        path: ../../
      # Dependencies, updated with flatpak-node-generator
      - generated-sources.json
      # Wrapper to launch the app
      - type: script
        dest-filename: run.sh
        commands:
          - cd /app/main/
          - |
            while true; do
              # The following options were temporarily disabled to work around a bug in Chromium or Electron:
              #
              # --ozone-platform-hint=auto \
              # --enable-features="WaylandWindowDecorations" \
              #
              # See https://github.com/electron/electron/issues/37531
              zypak-wrapper.sh ./ThreemaDesktop \
                "$@"
              if [ "$?" != "8" ]; then
                break
              fi
            done
    build-commands:
      # Install npm dependencies
      #
      # Note: In `package.json`, we pin the node/npm versions to a specific
      # value. Because the Flatpak runtime does not ensure a specific minor or
      # patch version of node, we skip the strict engine checks in npm by
      # passing in `--engine-strict=false`.
      - npm --engine-strict=false install --offline
      # Remove previous build artifacts
      - rm -rf build/electron/packaged
      # Build Threema Desktop
      - |
        ln -s $XDG_CACHE_HOME/node-gyp $HOME/.electron-gyp # https://github.com/flatpak/flatpak-builder-tools/pull/260#discussion_r822084602
        npm run dist:consumer-live
      # Bundle app and dependencies
      - ls -lah build/electron/packaged/
      - cp -a build/electron/packaged/Threema*-linux-* /app/main
      # Install app wrapper
      - install -Dm755 -t /app/bin/ run.sh
      # Install metadata
      - install -Dm644 packaging/metadata/ch.threema.threema-desktop.metainfo.xml /app/share/metainfo/ch.threema.threema-desktop.metainfo.xml
      - install -Dm644 packaging/metadata/ch.threema.threema-desktop.desktop /app/share/applications/ch.threema.threema-desktop.desktop
      - sed -i 's/ch.threema.threema-desktop/ch.threema.threema-desktop/g' /app/share/metainfo/ch.threema.threema-desktop.metainfo.xml
      - sed -i 's/ch.threema.threema-desktop/ch.threema.threema-desktop/g' /app/share/applications/ch.threema.threema-desktop.desktop
      - sed -i "s#name>Threema<#name>Threema Beta<#" /app/share/metainfo/ch.threema.threema-desktop.metainfo.xml
      - sed -i "s#Name=Threema#Name=Threema Beta#" /app/share/applications/ch.threema.threema-desktop.desktop
      - sed -i "s#StartupWMClass=Threema#StartupWMClass=Threema Beta#" /app/share/applications/ch.threema.threema-desktop.desktop
      # Install Icons
      - mkdir -p /app/share/icons/hicolor/{512x512,scalable}/apps/
      - |
        case consumer-live in
          consumer-sandbox) icon_base=consumer-sandbox ;;
          consumer-live) icon_base=consumer-live ;;
          work-sandbox) icon_base=work-sandbox ;;
          work-live) icon_base=work-live ;;
          work-onprem) icon_base=work-live ;;
          *) echo "Invalid flavor: consumer-live"; exit 1 ;;
        esac
        install -Dm644 packaging/assets/icons/flatpak/${icon_base}.svg /app/share/icons/hicolor/scalable/apps/ch.threema.threema-desktop.svg
        install -Dm644 packaging/assets/icons/flatpak/${icon_base}.png /app/share/icons/hicolor/512x512/apps/ch.threema.threema-desktop.png

This works without any issues. However, if I switch to the node20 image containing npm 10:

@@ -5,7 +5,7 @@ sdk: org.freedesktop.Sdk
 base: org.electronjs.Electron2.BaseApp
 base-version: '23.08'
 sdk-extensions:
-  - org.freedesktop.Sdk.Extension.node18
+  - org.freedesktop.Sdk.Extension.node20
 command: run.sh
 separate-locales: false
 finish-args:
@@ -25,7 +25,7 @@ finish-args:
   # Allow desktop notifications
   - --talk-name=org.freedesktop.Notifications
 build-options:
-  append-path: /usr/lib/sdk/node18/bin
+  append-path: /usr/lib/sdk/node20/bin
   cflags: -O2 -g
   cxxflags: -O2 -g
   env:

...the build fails:

Running: npm --engine-strict=false install --offline
npm info using npm@10.5.0
npm info using node@v20.12.2
npm ERR! code ENOTCACHED
npm ERR! request to https://registry.npmjs.org/fs-extra failed: cache mode is 'only-if-cached' but no cached response is available.

Note that flatpak-node-generator generates the same generated-sources.json when running under node 18 and 20, so there shouldn't be an issue with that.

Does node20 / npm10 use a different cache lookup logic than node18 / npm9?