[Unison-hackers] Experimental "faster check" mode

[Jerome Vouillon]

On Sat, Jul 24, 2010 at 09:44:25AM -0400, Benjamin C. Pierce wrote:
> The test
> 
>   if fastCheck && not ignorearchives then begin
> 
> at the beginning of Fpcache.init was intended to prevent
> pseudo-fingerprints from getting into the cache (by turning off
> caching).

But "ignorearchives" is set to true when the "ignorearchive"
preference is set, not when "fastercheckUNSAFE" preference is set...

> Would it be better to change Fpcache.save to drop
> individual fingeprints when they are pseudos?

The check "Prefs.read fastercheckUNSAFE && newfile" should be at the
beginning of Fpcache.fingerprint rather than in Os.fingerprint.

> > - You should check that the Xferhint code still works properly.
> >  I suspect we don't want to store pseudo-fingerprints there either...
> 
> Pseudo-fingerprints include the fingerprinted path, so we should
> never have a hit in the Xferhint table on a pseudo-fp.

We should include a fingerprint of the path, not the whole path.
Otherwise, we will end up with very large archive files.

  Still, it is a waste of space.  I'll change Xferhint.insertEntry like this:
> 
> let insertEntry fspath path fp =
>   if Prefs.read xferbycopying && not (Os.isPseudoFingerprint fp) then begin
> 
> > - If I read the code correctly, when "fastercheckUNSAFE" is set to
> >  true, any new file will fail to transfer.  This is the safe thing to
> >  do (though the failure happen at a very late stage, once the file
> >  contents has been transferred) but is very restrictive.  I explained
> >  in my previous mail how we can perform a transfer safely in this
> >  case too, but a protocol change is needed.  (That would be useful
> >  when one replica is initially empty: Unison would be able to start
> >  transferring files right away.)
> 
> Sorry -- I had gotten partway though this case and forgot to finish
> it.  I didn't completely understand your recommendation, since
> paranoidCheck is called in two places (and neither of them is in
> checkContentsChangeLocal), but I'll have another look now...

At the moment, paranoidCheck returns a status: either Success or
Failure (with some explanation of the failure).  In Copy.file,
checkContentsChange is then called with a different boolean argument
depending on the status.  The function paranoidCheck could return a
fingerprint instead of the explanation when it cannot decide whether
the file was correctly transferred.  Somethink like:
  let paranoidCheck fspathTo pathTo realPathTo desc fp ress =
    let info = Fileinfo.get false fspathTo pathTo in
    let fp' = Os.fingerprint fspathTo pathTo info in
    if fp' <> fp then begin
      Lwt.return (info, Undecided fp')
    end else
      Lwt.return (info, Success)
This fingerprint would be passed to checkContentsChangeLocal which
compute the fingerprint of the source file and can thus make sure the
two fingerprint matches.

Function checkContentsChangeLocal will have three fingerprints:
the one from update detection which may be a pseudo-fingerprint
(FPU), and the fingerprints of the source (FPS) and destination (FPD)
files.  If FPU is not a pseudo-fingerprint, the behavior of the
function should remain unchanged.  If it is a pseudo-fingerprint and
FPS <> FPD, we should report that the file is not correctly
transferred.

-- Jerome