[Unison-hackers] a common patch set - first attempt [repost]

[Benjamin Pierce]

Hi Andrew,

Your patch repository looks good to me.  (I did not check the patches  
themselves in detail, just the overall picture.)

In the meantime, I've been trying to think a little about how we could  
make this whole thing simpler for you.

The fundamental issue is this.  Unison's RPC mechanism is built on  
OCaml's marshalling and unmarshalling primitives, which are fast but  
type-unsafe: when you unmarshall a value, you just tell OCaml what type  
you expect it to be, and that's the type you get.  If the actual  
bytestring you are unmarshalling is not a marshalled value of that  
type, chaos can ensue.  Since we don't want chaos happening during file  
synchronization, we wrap the raw marshalling routines in a type safety  
layer that adds a tag (unique for each remotely callable procedure) to  
each packet containing marshalled data; this ensures that both ends of  
the RPC agree about what procedure is being called, and from this the  
type system guarantees safely... *provided* that both client and server  
are running binaries compiled from the same sources.  This is why  
Unison checks at the beginning of each run that the server's version  
number is the same as the client's.

Of course, this condition is actually stronger than necessary.  If the  
code has changed but the declarations of data types have not, then  
there is no problem.  But there is no automatic way to tell whether  
this is the situation.

I fee a few possibilities for improving matters...

1) Adopt a convention that, whenever anyone makes a change to a  
marshalled data structure, they bump the major version number, and make  
Unison's startup test just the major version number and ignore the  
minor version number.  This seems viable (especially since we have  
multiple people looking at commit messages and if one person forgets,  
someone else is likely to notice), and it has the great advantage of  
being simple.

2) Add a preference allowing users to specify explicitly that Unison  
should accept specific other versions as "close enough" to talk to  
safely.  This would allow people to upgrade to "slightly newer"  
versions easily, but requires some work from users.  (This work could  
perhaps be performed by packagers on behalf of users, by editing the  
code before compiling it so that some values for this preference are  
built in.)

3) Change the RPC mechanism so that each RPC is tagged with a hash of  
the actual type declaration for the data being communicated.  This is  
quite a bit of work, but it's certainly the best solution technically:  
completely safe, completely automatic.  Eijiro Sumii and I have been  
developing some low-level infrastructure that could perhaps be used for  
this (but don't hold your breath :-).

4) Reorganize the Unison repository and development process into  
different branches -- "trunk", "stable", etc.  I'm reluctant to take  
this path because it increases the overhead of making improvements to  
Unison: we are still in a situation where very few people feel able to  
make significant changes to the code, and I don't want to decrease  
their motivation for doing so!

On the whole, I think doing either (1) or (2) would be an easy and  
useful step over what we have now.  (3) is yet better but will take  
time.  (4) I don't like much, but maybe somebody will convince me.  :-)

Your comments?

     - Benjamin


On Jun 11, 2005, at 4:07 AM, Andrew Schulman wrote:

> Benjamin, Max, and Jérôme, I'd appreciate hearing your comments on  
> this.
> Send them to me privately if you prefer.  Thanks, Andrew.
>
> --------------- Forwarded message (begin)
>
> Subject: a common patch set - first attempt
> From: Andrew Schulman <andrex at comcast.net>
> Date: Mon, 30 May 2005 22:21:11 -0400
> Newsgroup: gmane.network.unison.devel
>
> I've put together a first attempt at a standard set of patches for
> Unison versions 2.9.1, 2.9.20, 2.10.2, and 2.12.0.  The draft page is
> at http://home.comcast.net/~andrex/unison/patches.html.  Please have a
> look at the page, and send me your comments.
>
> The patches listed on the page reflect the recent discussions on
> unison-hackers and unison-users, as well as I've been able to follow
> them, up through today.  The discussion was long and there are a lot of
> patches, so it's likely that I've gotten some details wrong or missed
> some things.  Please check the patches and let me know if you find any
> problems.
>
> Notes on specific patches:  The OSX client patch
> (http://lists.seas.upenn.edu/pipermail/unison-hackers/2005-May/ 
> 000071.html)
> conflicts with Jérôme's later file properties patch
> (http://lists.seas.upenn.edu/pipermail/unison-hackers/2005-May/ 
> 000090.html).
> I'm using only the later one, on the assumption that Jérôme revised and
> rereleased it.  Also I named one patch "osx-unknown" because its  
> purpose
> was never stated.
>
> The goal of the page is to provide a standard set of patches that all  
> of
> us who are developing and packaging Unison can agree to.  The current
> system of posting and pulling patches on unison-users and
> unison-hackers leads to hit-or-miss patching, as the recent discussion
> has shown.  I think it would make more sense to have a single place to
> keep them all.  Developers can call new patches to my attention, and
> I'll add them to the list.  Packagers can agree to keep their packages
> up-to-date with the list, in order to improve reliability and
> cross-platform consistency of their packages.  I hope that packagers
> from more platforms will sign on to keep current with the list, too.
>
> I've jumped the gun on the page by saying that "The patches listed here
> represent the consensus of the Unison developers and the packagers for
> Cygwin, OpenBSD, and Windows."  This isn't true yet, but I would like
> for it to be.  So in particular, Benjamin, Jerome, Max, and Zvezdan,
> I'd like to have each of you review the list of patches, and let me
> know if you can agree to it.  If you can't, then please tell us what
> your issues are and let's try to address them.
>
> I'm sure this isn't the greatest method of keeping a patch list.
> Whenever a new patch becomes available, someone will have to call it my
> attention if I miss it (I follow the mailing lists, but miss things
> sometimes), then I'll have to manually update the page and notify all
> of the packagers.  No doubt there's a better way, and if someone can
> tell or show me what that is, please do.  Also, I don't think it will
> be too much work to maintain the list, but if it is I may have to back
> off of it.  All in all it would be better to have some kind of
> collaborative approach.  But for now, I'm willing to give it a try, and
> I hope this approach will be an improvement over the current method.
>
> If this whole approach is wrong and there's a better/smarter way (CVS
> archives of patched previous releases?), then please tell me about
> that, too.  I won't mind; on the contrary, I don't want to waste my
> effort on an approach that isn't going to be useful to all of us.
>
> Andrew.
>
> --------------- Forwarded message (end)
> _______________________________________________
> Unison-hackers mailing list
> Unison-hackers at lists.seas.upenn.edu
> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers