[Unison-hackers] unison more memory efficient than rsync?

[Benjamin Pierce]

I regularly synchronize 120Gb disks with, among other things, lots  
and lots of mail files.  Update detection is a bit slower than I'd  
like because of all the inodes to be scanned, but not too bad.   
Others have reported success with much larger replicas.

Unison's overall memory usage scales more or less linearly with the  
number of files -- but with a fairly small constant factor.  Its peak  
memory usage scales linearly with the size of the largest individual  
file.  There have been some reports of problems with really gigantic  
(multi-gig) individual files.

I don't know whether the current rsync implementation calculates  
block fingerprints only within each file (as Unison does), or across  
a whole filesystem.  If it's just per file, then the memory usage may  
be comparable, since the core algorithms are very similar.

Regards,

     - Benjamin


On Dec 12, 2005, at 8:17 AM, Geoffrey Alan Washburn wrote:

> I've been using rsync for uni-directional backups for a while, but
> really it has been rather suboptimal because it seems to hold quite a
> bit of data in memory when processing a large disk with lots of files.
> Has anyone done any comparisons with how unison performs on
> uni-directional synchronization memory-wise?  I'm not too concerned
> about speed, but rsync chews up so much memory that it causes my iBook
> to thrash until finished.
>
> Of course, unison doesn't handle extended attributes yet, but I'm
> working on that.
>
> _______________________________________________
> Unison-hackers mailing list
> Unison-hackers at lists.seas.upenn.edu
> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers