doc: dune pkg tutorial

[Leonidas-from-XIV]

This lays the groundwork for an "Introduction to Dune package management" tutorial that we can point people to so they can form an understanding how to work with package mangement.

It's currently not linked and also not finished. My plan is to eventually show this:

• Create a project
• Add a dependency
• Change the version that is locked
• Remove dependency

To not overcomplicate things in an introduction document my plan is to use fmt as an example dependency, as it is fairly well-known (thus the example code isn't distracting) and also has the same module/findlib/opam name.

[Leonidas-from-XIV]

I've reworked it a bit and improved the structure to make it less of a work in progress and more something that can be read. As such it would be nice to have your input wrt. to the wording, the markup or the content.

In generally I tried to keep it vague enough to apply to main with or without the developer preview features turned on; this way it can be part of the documentation of both regular Dune as well as linked from the developer preview info page without creating confusion (hopefully).

[samoht]

An important concept (useful to existing users, not for newcomers) is that dune pkg is (or will be) able to build any package even if they are not packaged with Dune if they follow a few simple rules (mostly, they need to be relocatable). This means we do not mandate all the dependencies to use dune, nor do we need to patch existing packages (like opam-monorepo used to do) to build with Dune.

I think it would also be useful to explain this in a bit more detail - i.e. the way Dune see a package is separated into a few "macro" rules (that can be cached as all the other Dune rules): fetching and untaring the sources, running the opam build instruction in a "black box" sandbox with the right environment setup, etc. This is all possible as we re-use opam-lib infrastructure as much as possible (to download the package, to setup the required env, etc).

You could also explain why we are using 0install solver (instead of re-using the default opam solver): we always find a solution from an empty set of installed packages (while the opam solver tries to minimize recompilation as it comes from the Debian ecosystem) -- and 0install-solver is optimised for this (with slightly worse error messages unfortunately).

etc, etc. Ie. I do think explaining the mental model of how things actually work is helpful to convince experienced opam/dune users that this is going in the right direction: i.e. there are many more things to add (including stop looking at the packages being built with Dune as a black box, make cross-compilation to work, windows support, etc.) but the architecture is pretty solid (which I believe it is) to allow all of this with enough time and people working on it.

[emillon]

Thanks. Without going into details, I recommend reading https://diataxis.fr/tutorials/ - this is the compass we're using for our docs.

I think it's useful to have a tutorial for dune package management, and its goal should be that people that follow it learn by doing. The rest is important, but can be put in a different place in the documentation. As a litmus test, I think that everything we would show in a demo should be part of the tutorial. This will make the tutorial leaner and easier to follow. The resulting tutorial will be linear: set up, lock, build, add a dependency, build, remove a dependency, etc.

In my mind there are 3 types of content here:

• the "concepts" page is explanatory. I agree it's not useful in a tutorial. But it can be a great "Why is Dune dealing with package management?" doc in the explanations section.
• the "common dependency constraints" examples are useful for some how-to material. This is useful for users that will use that in the context of their project. How about a "How-to: add an external dependency"? this is exactly the workflow we'll simplify by adding an add, so this justifies a dedicated howto
• the linear tutorial itself

Hope that makes sense. I'll add notes in the doc

[emillon]

In the goal of reproducibility, we should make it clear how to get a dune binary that can be used in this tutorial (I assume it doesn't work with a released version), or at least clarify that it doesn't work with the dune that people have on their machines.

[maiste]

Should we also suggest having a clean switch (opam switch create none --empty)?

[Leonidas-from-XIV]

I tried to keep the wording as neutral as possible, in fact I just use main with just toolchains enabled to write the tutorial.

Partly because I think if it is in the Dune documentation it should describe the Dune release, partly because at least with toolchains I hope that when relocatable compilers land that the tutorial will work as-is.

If you think it would be better to start with

export DUNE_CONFIG__TOOLCHAINS=enabled

I think I'm ok with this and it can be removed later easily.

@maiste I'd really want to avoid involving opam in a tutorial which shows how to use package management without opam. For people who are new, they shouldn't need to know about switches, so I don't want to introduce all kinds of mental overheads.

[Leonidas-from-XIV]

I do think explaining the mental model of how things actually work is helpful to convince experienced opam/dune users that this is going in the right direction

I agree as well, but after thinking about it I realized that these should be two different types documents. This PR is a tutorial to describe how to get things going, whereas the mental model document would be describing how things work.

To quote the Diátaxis document:

All learning moves in one direction: from the concrete and particular, towards the general and abstract. The latter will emerge from the former.

(Which also matches my experience learning e.g. git). I expect that a lot of more experienced people will be interested in how it works under the hood, so I think it will make sense to create a second PR after this to explain what is happening under the hood when using these commands.

[christinerose]

Should this maybe be opam repository or opam-repository?

[Leonidas-from-XIV]

I've changed the whole section because I didn't like how it read "when Dune is doing X it is doing X" and it talked about one when in fact there are two (and maybe the explanation is a bit of a detour? What do you think @emillon?). Can you take a look and see if this makes more sense for you?

[ElectreAAS]

I love how simple it all looks!
LGTM 👍

[leostera]

LGTM 🚀 We can refine/iterate as we start getting feedback 🙏🏼

[maiste]

Being nitpicking here, but all good otherwise 👍