[Unison-hackers] dune and opam issues: a summary of my fuzzy understanding
Jacques-Henri Jourdan
jacques-henri.jourdan at cnrs.fr
Tue Nov 15 11:38:06 EST 2022
Hi all,
Let me first give my answers to the questions you asked:
> - Why are control files sometimes in the above repo and sometimes in
> the project repos?
On the one hand, the opam file should be published in opam repository so
that opam, as a package manager, can be used to install unison easily,
i.e., by simply typing "opam install unison". Users of opam need not
explicitly give the URL of the project repository: opam will lookup in
opam-repository the opam file and use it to get all the information it
needs. Also, the opam-repository repo contains one opam file for each
release, so that a user can explicitly ask to install e.g., unison
2.53.0 via "opam install unison.2.53.0", just like we would do with a
package manager from a Linux distribution. Thanks to this repository,
the command "opam upgrade" allows to upgrade the version of unison, but
only when a new release is published (a regular user doesn't want to
upgrade unison at every commit, because the content of the repository
between two releases can be bogus/unstable).
On the other hand, the project repository contains an opam file, so that
its contents can be kept in sync when details specific to the software
changes (e.g., which build system, configuration options, dependencies).
This file is also useful when people want to install a development
version of the software, e.g., for testing an unreleased feature. If
they want to do that, they need to issue special commands to opam to
tell it that this package should be installed from the development
repository. When doing that, opam will stop tracking the version of
unison used (and instead use a generic "dev" version string which tells
nothing), and "opam upgrade" will simply upgrade to the most recent
content of the repository.
When publishing a release, the release manager needs to copy the opam
file from the development repository to opam-repository, and add a "url"
section specifying where to find the sources of that particular release.
>
> - Which location is preferred and why?
Both locations are useful, but for different purposes:
- opam-repository is useful for end-users wanting to use released
versions of unison, without invoking complex command to tell opam where
to find the code
- the dev repo is useful for easier maintenance, and to let advanced
users install and test development versions
> - Does the file have to be at top-level in the project, if it's there
> (there are N packaging systems and it's not really ok for each to
> demand stuff in the top-level directory)?
If this file is used to install development versions, then opam
*expects* it to be there. I imagine there might be options to tell opam
where the opam file is, but this is non-standard and should be avoided.
Of course, if the file is kept in the repo only for maintenance
purposes, then it does not need to be at the root of the project. But
maintaining an opam file in an exotic location is going to be
cumbersome, because one will not be able to test it by installing the
development version.
> - When there are multiple files for multiple build flavors, can one
> build them without colliding? Or is this really a dune issue?
I'm not sure I understand your question: opam is really a "package
manager", so the question is not really *building* the software in
multiple flavors, but rather *installing* it.
It is possible to have several opam files at the root of the same
repository. They will correspond to different packages using the same
source tree, for example because the software has been split into
several packages (a library may be distributed using several packages
representing different features set o the library...). But if these
packages are installed simultaneously, then it's just like with "make
install": they need to install a different set of files. Otherwise, they
are conflicting, and this should be part of a "conflict" clause of the
opam files.
In the case of unison, we have several options:
1- have two packages "unison-cli" and "unison-gui", both installing the
executable "unison", and declare the two packages in conflict. The user
will have to choose between the two packages when typing "opam". The
problem is perhaps that there is no "canonical" package for unison.
2- same as 1-, but with two different names for the executable. The user
can install both packages, but once installed, she will have to type two
different commands depending on whether she wants to use the version
with the GUI or not: this is not the same behavior as with "make
install" and thus I don't really like this solution.
3- use only one package, but with an optional dependency on lablgtk. The
package will be built with two different configuration depending on
whether lablgtk is installed, and an upgrade of lablgtk will
automatically trigger a rebuild of unison. The disadvantage is that a
user willing to install unison with the gui will type "opam install
unison", but will not have the gui installed because lablgtk will not be
installed by default; this can be confusing.
4- same as 3-, but with a pseudo-package "unison-gui" (i.e., without any
code), with a dependency on both "unison" and "lablgtk", so that
installing "unison-gui" will make sure the GUI is enabled. This is
somewhat less confusing than 3-, because a user can install an actual
package that says "I want unison with its GUI".
> - Is the definition of projects necessary to use dune?
Yes, but it can be minimal. It need not duplicate all the information of
the opam file if we decide that we don't want to generate the opam file
with dune.
> - Why are there magic numbers in dune files, e.g.
> "-w -3-6-9-10-26-27-32-34-35-38-39-50-52".
By default, dune enables more warnings than the default OCaml compiler
enables, and make those warnings errors. So we need to disable the
warnings the are triggered by the code of OCaml, because they will be
reported as errors.
These "magic numbers" are simply identifiers of warnings.
> - What are the arguments for and against the two styles
> (dune-generated vs hand-maintained)?
Of course, generating opam files automatically avoids duplicating
information between dune and opam. So, if dune is the primary build
system of a project, and if all the information that one wants to place
in the opam file can be generated by dune, then this is the way to go.
*However, opam can be instructed to use a different build system. There
is no fundamental reason to use dune in the opam file. The opam file can
very well tell opam to use the standard Makefile for both compilation
and installation.* And I would argue that if the main unison developers
are not using dune, then dune should not be used in the opam file,
because if the dune build system is bogus (because not properly
configured...), then the unison developers will not test it, and
therefore they will not notice the bug. I don't see why the validity of
the opam file should rely on the minority of developers using dune.
> - If opam files are in the opam repository, and also generated, how does
> that work?
Each time someone changes the content of the dune-project file, she
should regenerate the opam file(s), and commit both at the same time.
It is a good idea to setup a CI job that checks that the commited opam
file corresponds to the one that would be generated by dune.
> - If files are generated, then mostly-obviously they shouldn't be
> checked in. Do people think they should be? If they are, are we
> sure they will be bit-for-bit identical when regenerated on
> different systems?
They should be, because the opam file needs to be present if one wants
to use opam to install a development version of unison (as explained
above). Yes, they should be bit-for-bit identical (otherwise, I consider
this as a dune bug).
> * way forward
>
> At this point, my inclination is:
>
> - Reset opam maintainer to `unison-hackers at lists.seas.upenn.edu <https://LISTS.SEAS.UPENN.EDU/mailman/listinfo/unison-hackers>`
> because the current maintainer has not answered mail, and we don't
> really have another plan. This is saying that the file is
> maintained by the project, just like every other file in the repo.
Agreed.
> - Apply the lablgtk3 and dune 2.7 changes from #671, basically
> cherrypicking the parts I understand and agree with, and writing a
> commit message.
Agreed.
> - Consider turning on generation, and removing unison.opam from the
> repo, pending feedback, probably via a PR that does only that.
>
> - See if there are things to be done other than the above 3.
I would rather suggest to change the opam file so that it uses the
Makefile. If make is the primary build system, then the dune files
should be maintained by their users. As a result, a user of the main
build system can maintain the opam files without understanding dune,
which is mainly orthogonal.
My 2 cents,
best regards,
--
JH Jourdan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://LISTS.SEAS.UPENN.EDU/pipermail/unison-hackers/attachments/20221115/36e22c7a/attachment-0001.htm>
More information about the Unison-hackers
mailing list