[Unison-hackers] relationship of CI and build

[Greg Troxel]

This is a comment on https://github.com/bcpierce00/unison/pull/427 as
part of a larger and very helpful discussion that is untangling and
improving a lot of things.  I am copying it here as I find the github
discussion style to be contrary to the "development discussion is on the
mailinglist" norm -- not anyone here's fault: this is what happens with
github.



----------------------------------------
I think we need to be very careful about what should be part of tests
and what should be part of CI.

We should keep in mind that we are off the rails of Free Software by
using proprietary CI. While it's convenient now, I don't think it's a
good thing, and we should be careful to have it entangled as little as
possible. Things should be set up so that anyone who wants can set up
some other CI system.

Also, with respect to Free Software norms, people being able to build
from source has to be the main approach. It may be that many people use
binaries, but that is only ok when it is merely a convenience.

It's an important principle that users can run builds, packaging, and
tests. And use those procedures for CI if they want. So I don't think we
should even consider having behavior differ when run under github
CI. The desire for differences I see as a clue that there is some other
problem, and instead we should fix that problem.

So stepping back, there should be

    documentation of build environment prerequisites

    a build procedure, that anyone can run by checking out the repo and
    invoking some short command, via scripts/whatever in the repo

    a test procedure, that perhaps requires build to have been done,
    that anyone can run, again scripted

    an install procedure, that puts the binaries either in the running
    system, or in some DESTDIR (for packaging), again that anyone can
    run

    optionally (as I believe this is really not the proper domain of a
    project, but obviously users expect this), a packaging procedure
    that takes the results of the install and bundles it up

    some kind of CI, that is a wrapper that basically sets up an
    environment and runs the previous steps. The CI wrapper should not
    contain anything that belongs in the previous steps, or rather, if
    it does, that content should be hoisted to the step itself.

That means running tests on packaged bits to me feels
irregular. Normally, tests are run on the built products before they are
installed, and people only install when they pass.

So I think we should

    fix the unison/unison-gtk build collision, not as a CI patch, but
    for real. I don't think it is hard and all that is required is the
    will to do it (and picking a name). That takes most of the trouble
    off the table

    if after having built, on some platform, one can't run, because of
    some dll trouble, add a wrapper script that makes that come out
    right, and then use that wrapper in tests

    probably a bunch of other small things

Finally, I don't mean to criticize anyone's efforts or how we got where
we are. This is all complicated and we have had many local
optimizations. We are in far far better shape than we ever have been
with respect to packaging/etc. I am merely trying to step back and
look at the big picture.  Greg Troxel <gdt at lexort.com>