Automatically commit changes to lockfiles

[armanbilge]

This is a bit of a new issue (at least in typelevel projects) after adding some JS cross-builds that also have npm dependencies.

For example in fs2 3.1.2 we updated the npm dependency to @types/node in typelevel/fs2#2594.

This change caused steward PRs to cross-building downstreams to fail, because they needed to update their (frozen) lockfiles.
http4s/http4s#5182
typelevel/skunk#545

I wonder if updating/committing the lockfile could be implemented in a way similar to how steward does GH workflow regeneration or formatting after scala format is updated?

I'm happy to work on a PR for this if I could get some pointers about the where/how :)

[armanbilge]

See also yarnpkg/yarn#4147 for some more discussion about (frozen) lockfiles and CI.

[fthomas]

Workflow regeneration and formatting are implemented as PostUpdateHooks in Scala Steward. These hooks are executed when predefined dependencies are updated (1, 2).

To implement this, we need know which command needs to executed after bumping the version and under which condition this hook would be executed, for example is it executed when the project depends on a specific library/plugin or only if specific dependencies are updated?

[armanbilge]

Thanks for getting back to me with those pointers! That is very very helpful.

we need know which command needs to executed after bumping the version

We'd want to execute npmInstallDependencies. But only after removing --frozen-lockfile from the yarnExtraArgs setting for each relevant project.

under which condition this hook would be executed

This is quite tricky I think. Relevant conditions:

1. the project being updated should be using the ScalaJS bundler plugin, with useYarn := true;
2. the dependency being updated (or one of its transitives) should also being using the ScalaJS bundler plugin.

I don't think that there's a good way to check (2). Transitive npm dependencies are communicated to downstreams via manifest files included in the jars, so this is not easily accessible. Not sure if it's a good idea but the easy way out would be to just run this when any dependency is updated 🙃

Anyway thanks again, I'm going to take this under advisement, do some more research into ScalaJS bundler and steward, and see if I can put together a PR :)

[fthomas]

We'd want to execute npmInstallDependencies. But only after removing --frozen-lockfile from the yarnExtraArgs setting for each relevant project.

It would be ideal if this could be done in one sbt command.

1. the project being updated should be using the ScalaJS bundler plugin, with useYarn := true;

Checking if the project depends on that plugin is easy. If useYarn is set to true would require more modifications to Scala Steward.

2. the dependency being updated (or one of its transitives) should also being using the ScalaJS bundler plugin.

I don't think that there's a good way to check (2).

It would probably also require more modifcations to Scala Steward than for checking if useYarn is true.

Not sure if it's a good idea but the easy way out would be to just run this when any dependency is updated 🙃

Running the above sbt command for every update on every project would slow down Scala Steward considerably. But it would be fine with me if we introduce a .scala-steward.conf option for running that command on any update in all projects that have set this option.

[armanbilge]

Running the above sbt command for every update on every project would slow down Scala Steward considerably.

Of course, apologies, I meant only for repos/projects with the ScalaJSBundler plugin enabled. But a configuration option seems perfectly fine as well.

checking if useYarn is set to true would require more modifications to Scala Steward.

Actually, maybe an easier thing to do would be to just look for a yarn.lock file in the appropriate directories? That would probably be sufficient.

It would be ideal if this could be done in one sbt command.

Maybe this command could be added upstream in the ScalaJSBundler plugin, I'll look into it 👍

[fthomas]

Actually, maybe an easier thing to do would be to just look for a yarn.lock file in the appropriate directories? That would probably be sufficient.

Checking for ScalaJSBundler and the yarn.lock file sounds good to me. In that case I wouldn't add a new configuration option.

[armanbilge]

Thanks! This sounds do-able 😄

[armanbilge]

Update on this:

• My proposal to enhance support for lockfiles in scalajs-bundler got no response. Indeed, it is effectively unmaintained.
  Add first-class support for frozen lockfiles scalacenter/scalajs-bundler#409
• I've since removed the npm dependencies and lockfiles from Http4s and Skunk, so they are no longer affected by this issue.
• If there's ever a future need, an sbt-typelevel integration would be a good strategy to do this without requiring any changes to Scala Steward.

Since nobody else seems to have a need for this, I will close the issue. Thanks again for your time Frank, sorry that it was all for nothing 🙃