[Unison-hackers] unison and inotify principle

[Sylvain Le Gall]

On 04-06-2008, Benjamin Pierce <bcpierce at cis.upenn.edu> wrote:
> Hi Sylvain,
>
>>> I am a very humble user who just want to comment on this design (take
>> all of this conditionaly -- if you feel this is just silly, forget  
>> about
>> it).
>>
>> I don't like the idea of the "file which should be a newline-separated
>> list of paths (relative to the root of the synchronization)".
>>
>> My idea is the following:
>> - use "-repeat FOO" but FOO is a program
>> - FOO read file scanned by unison on stdin
>> - if FOO detects a change in a file scanned by UNISON, it writes it on
>>  stdout
>>
>> In this design, unison has just to send absolute filename name to FOO
>> and wait for FOO to write any notification.
>
> I'm all in favor of simplicity, but I'm concerned about a couple of  
> points in this design.
>

I take the example of inotify, which i know a little bit.

> 1) How is Unison going to hear about added files?
>

If you ask to monitor a directory, you will get notified of file added
in this directory (or directory added...).

1) Let say unison scan directory /A/B.
2) unison send "/A/B" to FOO 
3) FOO ask inotify to watch directory "/A/B"
4) user "touch /A/B/c"
5) inotify notify FOO that a file "/A/B/c" has been created
6) FOO send "/A/B/c" to unison (but doesn't ask for its watching)
7) unison take into consideration the new file and apply rules to see if
   it fits every rules (ignore...)
8) if unison is ok to take into account "/A/B/c" it sends "/A/B/c" to
   FOO (stating that FOO should watch also this file).

This way FOO has only little knowledge of what unison rules are. It just
need to check for files that unison is asking him... 

> 2) Is Unison really going to tell FOO about *every* file it scans?   
> There may be very many of these (I haven't counted, but I'll bet there  
> are more than 100k files in my filesystem).

Inotify can handle 100k files.
Transfering 100k x average size of a filename through a pipe
(stdin/stdout for a Unix.open_process) is the fastest communication
channel on most system.

But you have other options, unison can only tell foo to take into
consideration directories (i.e. only send directory filename). inotify
can also watch for changes of file in watched directory.

Last but not least, FOO command line options should be simple and
standard:
- "-enc ENC": define filename encoding for data that will be
  send/receive
- "-only-dir": watch for changes only on directory 

The real advantage of my proposal is that FOO has very little to know
about unison. FOO is only made to watch file...

You also have the speed argument/power efficiency: unison is waked up
only when there is a real change (i.e. FOO send a filename). There is no
regular scan of the file. Scanning a file every now and then is a big
problem (think of laptop user that don't like their hard drive to be
spin up every 10 seconds).

Regards,
Sylvain Le Gall

ps: i am not an expert at all, if you feel this proposal is not good,
just tell me, i will stop arguing.