From bcpierce at seas.upenn.edu Thu Jan 1 10:37:41 2009 From: bcpierce at seas.upenn.edu (bcpierce@seas.upenn.edu) Date: Thu, 01 Jan 2009 10:37:41 -0500 Subject: [Unison-hackers] Problem with fractional mtime? ("destination updated during synchronization") In-Reply-To: <20081205165014.GA4637@pps.jussieu.fr> References: <200810292158.13947.hans_meine@gmx.net> <200811230147.49766.hans_meine@gmx.net> <200811231342.44731.hans_meine@gmx.net> <39071660-FFFA-4852-A14D-3F57AE75D416@cis.upenn.edu> <20081205165014.GA4637@pps.jussieu.fr> Message-ID: <20090101103741.0v8ffky9s0w88go4@webmail.seas.upenn.edu> Thanks for this, Jerome! I am considering another change that will also require an archive format bump, so I will incorporate this at the same time. - Benjamin Quoting Jerome Vouillon : > On Sun, Nov 23, 2008 at 08:44:54AM -0500, Benjamin Pierce wrote: >> There is some rather tricky code in props.ml (in the sub-module Time) >> that compares file modtimes while trying to take into account some >> possible reasons why unchanged files may appear to have different >> modtimes (e.g., daylight savings time, or one filesystem being FAT, >> which can only store modtimes with a resolution of two seconds). If >> the problem you're seeing is really with fractional modtimes, it may >> be possible to fix it by changing something here... > > The fix should be the following: > - Use function similar_correct, which ignore one-second differences, > instead of function similar in module Time in file props.ml > - Change the definition of function hash from > let hash t h = > Uutil.hash2 > (match t with > Synced f -> Hashtbl.hash (moduloOneHour (approximate f)) > | NotSynced _ -> 0) > h > to > let hash t h = h > (Two similar times should have the same hash; if we ignore > one-second differences, this mean that all times should have the > same hash.) > - Increment the variable archiveFormat in file update.ml > (Indeed, the archive invariant is changed.) > > This is a change that I have wanted to perform for some time, but > never did because of the archive format change. > > -- Jerome > _______________________________________________ > Unison-hackers mailing list > Unison-hackers at lists.seas.upenn.edu > http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers > From zhangweiwu at realss.com Fri Jan 9 22:36:31 2009 From: zhangweiwu at realss.com (Zhang Weiwu) Date: Sat, 10 Jan 2009 11:36:31 +0800 Subject: [Unison-hackers] svn or cvs or access with version control? Message-ID: <496817BF.1000201@realss.com> Hello. I am trying to work on several user interface issue on unison and would like to have a checkout from main repository instead of maintaining my own (though I won't commit; instead produce patches). Is there still a live version control repository that holds unison latest version? Thanks. -- Real Softservice Huateng Tower, Unit 1788 Jia 302 3rd area of Jinsong, Chao Yang Tel: +86 (10) 8773 0650 ext 603 Mobile: 159 1111 7382 http://www.realss.com From alan.schmitt at polytechnique.org Sat Jan 10 03:37:30 2009 From: alan.schmitt at polytechnique.org (Alan Schmitt) Date: Sat, 10 Jan 2009 09:37:30 +0100 Subject: [Unison-hackers] svn or cvs or access with version control? In-Reply-To: <496817BF.1000201@realss.com> References: <496817BF.1000201@realss.com> Message-ID: <2546F677-FED7-44A2-9C59-F97E07BCC79A@polytechnique.org> On 10 janv. 09, at 04:36, Zhang Weiwu wrote: > Hello. I am trying to work on several user interface issue on unison > and > would like to have a checkout from main repository instead of > maintaining my own (though I won't commit; instead produce patches). > Is > there still a live version control repository that holds unison latest > version? If you follow the "Download" and "Live subversion repository" links on Unison's page, you get there: http://www.seas.upenn.edu/~bcpierce/unison//svn-instructions.html Hope this helps, Alan -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 194 bytes Desc: This is a digitally signed message part Url : http://lists.seas.upenn.edu/pipermail/unison-hackers/attachments/20090110/2dee5e24/PGP.sig From duane.mckinney at gmail.com Sat Jan 10 17:05:51 2009 From: duane.mckinney at gmail.com (Duane McKinney) Date: Sat, 10 Jan 2009 14:05:51 -0800 Subject: [Unison-hackers] Sneaker Net or Incremental Backup In-Reply-To: <20081230111900.tkbedc8r8c04so40@webmail.seas.upenn.edu> References: <495A0E6B.2040509@gmail.com> <495A126F.40205@gmail.com> <20081230111900.tkbedc8r8c04so40@webmail.seas.upenn.edu> Message-ID: An embedded and charset-unspecified text was scrubbed... Name: warning1.txt Url: http://lists.seas.upenn.edu/pipermail/unison-hackers/attachments/20090110/030c3086/warning1.txt -------------- next part -------------- > That's a nice hack -- I had no idea the new copyprog functionality > could be used that way. That is what I love about linux/unix. Lots of small programs that can be combined to do great things. Didn't have quite as much free time as I though, but here are he results of my initial testing. It mostly works, and for now works well enough for me. I can now cron my syncs and not have to worry that some large file is going to kill our pipe for a few days :) I'll report back again in a month or so, or sooner if I run into an issue and have to change something. I have attached the shell script and the profile I used for testing. The important parts are: copyprog = sneakernet.sh /home/Duane/test duane at 192.168.113.2:/home/duane/test /home/Duane/usbdrive copyprogrest = sneakernet.sh This replaces what would normally be rsync with the attached shell script. I have only done limited testing so far, but I don't see anything that could be harmful. To be useful you will need 2 profiles a normal and a sneakernet. In your normal profile you should have copyprog = rsync --inplace --compress --max-size=xxx copyprogrest = rsync --partial --inplace --compress --max-size=xxx Where xxx is the size you have determined is too much to transfer via network. For me it is 1073741824 = 1GB The sneakernet profile should be identical except for the copyprog args copyprog = sneakernet.sh LocalRoot RemoteRoot SneakerNetPath copyprogrest = sneakernet.sh LocalRoot is the path on the local machine you want to sync RemoteRoot is not formatted the same as unison, it is formatted as passed by unison to rsync SneakerNetPath is the path to a directory on some removable media so now you will have 2 command to get everything synched unison myprofile unison myprofilesneakernet Issues: On copying from the modified machine, unison complains that the copy did not succeed and moves on, so only one file for each directory will be copied to the usb drive in a directory that is out of sync. This is not a big deal for me, but it may be for others. If it turns out to be a problem I'll probably dig through the source and come up with a real solution rather than this hack/workaround. I have not done any testing with copyprogrest. Since we are using local disks at all times, I don't know if this will be needed. bcpierce at seas.upenn.edu wrote: > That's a nice hack -- I had no idea the new copyprog functionality > could be used that way. Will be interested to hear whether it works... > > - Benjamin > > Quoting Duane McKinney : > >> Or from Version 2.31.4 >> copyprog xxx >> A string giving the name of an external program that can be used to >> copy large files efficiently (plus command-line switches telling it to >> copy files in-place). The default setting invokes rsync with appropriate >> options?most users should not need to change it. >> >> copyprogrest xxx >> A variant of copyprog that names an external program that should be >> used to continue the transfer of a large file that has already been >> partially transferred. Typically, copyprogrest will just be copyprog >> with one extra option (e.g., ?partial, for rsync). The default setting >> invokes rsync with appropriate options?most users should not need to >> change it. >> >> copythreshold n >> A number indicating above what filesize (in kilobytes) Unison >> should use the external copying utility specified by copyprog. >> Specifying 0 will cause all copies to use the external program; a >> negative number will prevent any files from using it. The default is -1. >> See the Making Unison Faster on Large Files section for more information. >> >> I guess I should have checked out the Beta Documentation 1st. I'll try >> messing around with using a shell script as the copyprog. I would >> assume that it would work just fine. My plan is to write a shell script >> which ignores the destination unison tells it, and instead copies it to >> a usb drive. >> >> I'll probably report back on this in a week or so. Currently I am >> running 2.27, so I have some compiling and testing to do. >> >> Duane McKinney wrote: >>> What about a flag that tells it to instead of executing the copies, just >>> prints out a list of the files that would be copied? A proper name is >>> escaping me at the moment, but it is a common option on programs. >>> >>> Benjamin Pierce wrote: >>>> Both of these features would be easy to implement using information >>>> that Unison already has available. If you want to give it a try, I >>>> can tell you where to start. (I'd be reluctant, though, to add this >>>> code to the main sources -- Unison arguably has too many flags and >>>> switches already!) >>>> >>>> Best, >>>> >>>> - Benjamin >>>> >>>> On Dec 27, 2008, at 10:27 AM, Duane McKinney wrote: >>>> >>>>> I searched, but came up empty. Can this be done. >>>>> I sync two offices using unison. >>>>> 1) (Optional)I would like to be able to set a preference that says, >>>>> don't try to >>>>> sync a file if it would require X bytes transferred >>>>> 2) Get a list of changed files from a root. That way I can copy the >>>>> files to a >>>>> removable drive and sync them when I get to the other office. >>>>> >>>>> Here is my reasoning. Most of the time, root changes are very >>>>> small, a few MB. >>>>> But Let's say that I download a new CENTOS release and place it on >>>>> the file >>>>> server. This may be a poor example, because I could retrieve it >>>>> over the >>>>> internet again, but just bear with me. I would like for unison, to >>>>> instead of >>>>> trying to sync this file over the network to skip it. The I would >>>>> like to be >>>>> able to say once a week, run a job, that would take the differences >>>>> in the root >>>>> and sent them to a USB drive. I can then carry this drive to >>>>> location 2, and >>>>> update the root there. Then from my understanding, the network >>>>> sync would >>>>> detect that both roots are identical. >>>>> >>>>> That way, our bandwidth isn't being eaten up for hours/days, trying >>>>> to perform a >>>>> sync that will most likely fail because it will take so long. >>>>> >>>>> Also, this would be more useful, than synchronizing the whole root >>>>> to a usb >>>>> drive, because, the total of the data that I am synchronizing is > >>>>> 1TB. I would >>>>> not mind moving a few GB (<100) via USB. I am trying to avoid >>>>> needing either, a >>>>> lot of time, or a bunch of USB drives (one for each root). >>>>> >>>>> I have not yet looked at the source, but I would assume that most of >>>>> the items >>>>> required for this feature are already in place. Is it feasible? Is >>>>> there >>>>> already a way? >>>>> >>>>> _______________________________________________ >>>>> Unison-hackers mailing list >>>>> Unison-hackers at lists.seas.upenn.edu >>>>> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >> _______________________________________________ >> Unison-hackers mailing list >> Unison-hackers at lists.seas.upenn.edu >> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >> >> > > > _______________________________________________ > Unison-hackers mailing list > Unison-hackers at lists.seas.upenn.edu > http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers -------------- next part -------------- A non-text attachment was scrubbed... Name: test.prf Type: application/pics-rules Size: 535 bytes Desc: not available Url : http://lists.seas.upenn.edu/pipermail/unison-hackers/attachments/20090110/030c3086/test.bin From bcpierce at cis.upenn.edu Sat Jan 10 17:15:03 2009 From: bcpierce at cis.upenn.edu (Benjamin Pierce) Date: Sat, 10 Jan 2009 17:15:03 -0500 Subject: [Unison-hackers] Sneaker Net or Incremental Backup In-Reply-To: References: <495A0E6B.2040509@gmail.com> <495A126F.40205@gmail.com> <20081230111900.tkbedc8r8c04so40@webmail.seas.upenn.edu> Message-ID: <5233E32B-EFB2-4FF2-879E-6FF88BE5CFD3@cis.upenn.edu> Hi Duane, Could you re-post this with the shell script appended in-line instead of attached? And/or perhaps just put this explanation in the FAQ wiki http://alliance.seas.upenn.edu/~bcpierce/wiki/index.php Thanks! - Benjamin On Jan 10, 2009, at 5:05 PM, Duane McKinney wrote: > This message has been automatically modified by CETS's antivirus/ > antispam > filter. If you have questions about this, please send mail to cets. > > An attachment named sneakernet.sh was removed from this document as it > constituted a security hazard. Send mail to cets if you have > questions about this policy. > > > That's a nice hack -- I had no idea the new copyprog functionality > > could be used that way. > That is what I love about linux/unix. Lots of small programs that > can be combined to do great things. > > > Didn't have quite as much free time as I though, but here are he > results of my initial testing. It mostly works, and for now works > well enough for me. I can now cron my syncs and not have to worry > that some large file is going to kill our pipe for a few days :) > I'll report back again in a month or so, or sooner if I run into an > issue and have to change something. > > I have attached the shell script and the profile I used for testing. > The important parts are: > copyprog = sneakernet.sh /home/Duane/test > duane at 192.168.113.2:/home/duane/test /home/Duane/usbdrive > copyprogrest = sneakernet.sh > > This replaces what would normally be rsync with the attached shell > script. > > I have only done limited testing so far, but I don't see anything > that could be harmful. > > To be useful you will need 2 profiles a normal and a sneakernet. In > your normal profile you should have > copyprog = rsync --inplace --compress --max-size=xxx > copyprogrest = rsync --partial --inplace --compress --max-size=xxx > > Where xxx is the size you have determined is too much to transfer > via network. For me it is 1073741824 = 1GB > > The sneakernet profile should be identical except for the copyprog > args > copyprog = sneakernet.sh LocalRoot RemoteRoot SneakerNetPath > copyprogrest = sneakernet.sh > LocalRoot is the path on the local machine you want to sync > RemoteRoot is not formatted the same as unison, it is formatted as > passed by unison to rsync > SneakerNetPath is the path to a directory on some removable media > > so now you will have 2 command to get everything synched > unison myprofile > unison myprofilesneakernet > > Issues: > On copying from the modified machine, unison complains that the copy > did not succeed and moves on, so only one file for each directory > will be copied to the usb drive in a directory that is out of sync. > This is not a big deal for me, but it may be for others. If it > turns out to be a problem I'll probably dig through the source and > come up with a real solution rather than this hack/workaround. > > I have not done any testing with copyprogrest. Since we are using > local disks at all times, I don't know if this will be needed. > > bcpierce at seas.upenn.edu wrote: >> That's a nice hack -- I had no idea the new copyprog functionality >> could be used that way. Will be interested to hear whether it >> works... >> - Benjamin >> Quoting Duane McKinney : >>> Or from Version 2.31.4 >>> copyprog xxx >>> A string giving the name of an external program that can be >>> used to >>> copy large files efficiently (plus command-line switches telling >>> it to >>> copy files in-place). The default setting invokes rsync with >>> appropriate >>> options?most users should not need to change it. >>> >>> copyprogrest xxx >>> A variant of copyprog that names an external program that >>> should be >>> used to continue the transfer of a large file that has already been >>> partially transferred. Typically, copyprogrest will just be copyprog >>> with one extra option (e.g., ?partial, for rsync). The default >>> setting >>> invokes rsync with appropriate options?most users should not need to >>> change it. >>> >>> copythreshold n >>> A number indicating above what filesize (in kilobytes) Unison >>> should use the external copying utility specified by copyprog. >>> Specifying 0 will cause all copies to use the external program; a >>> negative number will prevent any files from using it. The default >>> is -1. >>> See the Making Unison Faster on Large Files section for more >>> information. >>> >>> I guess I should have checked out the Beta Documentation 1st. >>> I'll try >>> messing around with using a shell script as the copyprog. I would >>> assume that it would work just fine. My plan is to write a shell >>> script >>> which ignores the destination unison tells it, and instead copies >>> it to >>> a usb drive. >>> >>> I'll probably report back on this in a week or so. Currently I am >>> running 2.27, so I have some compiling and testing to do. >>> >>> Duane McKinney wrote: >>>> What about a flag that tells it to instead of executing the >>>> copies, just >>>> prints out a list of the files that would be copied? A proper >>>> name is >>>> escaping me at the moment, but it is a common option on programs. >>>> >>>> Benjamin Pierce wrote: >>>>> Both of these features would be easy to implement using >>>>> information >>>>> that Unison already has available. If you want to give it a >>>>> try, I >>>>> can tell you where to start. (I'd be reluctant, though, to add >>>>> this >>>>> code to the main sources -- Unison arguably has too many flags and >>>>> switches already!) >>>>> >>>>> Best, >>>>> >>>>> - Benjamin >>>>> >>>>> On Dec 27, 2008, at 10:27 AM, Duane McKinney wrote: >>>>> >>>>>> I searched, but came up empty. Can this be done. >>>>>> I sync two offices using unison. >>>>>> 1) (Optional)I would like to be able to set a preference that >>>>>> says, >>>>>> don't try to >>>>>> sync a file if it would require X bytes transferred >>>>>> 2) Get a list of changed files from a root. That way I can >>>>>> copy the >>>>>> files to a >>>>>> removable drive and sync them when I get to the other office. >>>>>> >>>>>> Here is my reasoning. Most of the time, root changes are very >>>>>> small, a few MB. >>>>>> But Let's say that I download a new CENTOS release and place it >>>>>> on >>>>>> the file >>>>>> server. This may be a poor example, because I could retrieve it >>>>>> over the >>>>>> internet again, but just bear with me. I would like for >>>>>> unison, to >>>>>> instead of >>>>>> trying to sync this file over the network to skip it. The I >>>>>> would >>>>>> like to be >>>>>> able to say once a week, run a job, that would take the >>>>>> differences >>>>>> in the root >>>>>> and sent them to a USB drive. I can then carry this drive to >>>>>> location 2, and >>>>>> update the root there. Then from my understanding, the network >>>>>> sync would >>>>>> detect that both roots are identical. >>>>>> >>>>>> That way, our bandwidth isn't being eaten up for hours/days, >>>>>> trying >>>>>> to perform a >>>>>> sync that will most likely fail because it will take so long. >>>>>> >>>>>> Also, this would be more useful, than synchronizing the whole >>>>>> root >>>>>> to a usb >>>>>> drive, because, the total of the data that I am synchronizing >>>>>> is > >>>>>> 1TB. I would >>>>>> not mind moving a few GB (<100) via USB. I am trying to avoid >>>>>> needing either, a >>>>>> lot of time, or a bunch of USB drives (one for each root). >>>>>> >>>>>> I have not yet looked at the source, but I would assume that >>>>>> most of >>>>>> the items >>>>>> required for this feature are already in place. Is it >>>>>> feasible? Is >>>>>> there >>>>>> already a way? >>>>>> >>>>>> _______________________________________________ >>>>>> Unison-hackers mailing list >>>>>> Unison-hackers at lists.seas.upenn.edu >>>>>> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >>> _______________________________________________ >>> Unison-hackers mailing list >>> Unison-hackers at lists.seas.upenn.edu >>> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >>> >>> >> _______________________________________________ >> Unison-hackers mailing list >> Unison-hackers at lists.seas.upenn.edu >> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers > _______________________________________________ > Unison-hackers mailing list > Unison-hackers at lists.seas.upenn.edu > http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers From duane.mckinney at gmail.com Tue Jan 13 07:08:09 2009 From: duane.mckinney at gmail.com (Duane McKinney) Date: Tue, 13 Jan 2009 04:08:09 -0800 Subject: [Unison-hackers] Sneaker Net or Incremental Backup In-Reply-To: <5233E32B-EFB2-4FF2-879E-6FF88BE5CFD3@cis.upenn.edu> References: <495A0E6B.2040509@gmail.com> <495A126F.40205@gmail.com> <20081230111900.tkbedc8r8c04so40@webmail.seas.upenn.edu> <5233E32B-EFB2-4FF2-879E-6FF88BE5CFD3@cis.upenn.edu> Message-ID: $ cat sneakernet.sh #!/bin/sh # Usage # $1 = the local root # $2 = the remote path that unison will pass to rsync # $3 = the path used for the sneakernet, # ie the path to the folder on a usb drive moved between locations # $4 = Souce of the copy as sent by unison # $5 = Destination of the copy as sent by unison # Debuggin stuff OUTFILE=/dev/null SOURCEFILE=$4 DESTFILE=`echo $4 | sed "s|$1|$3|"` DESTDIR=`echo $DESTFILE | sed 's:\(/.*/\).*:\1:'` echo $SOURCEFILE echo $DESTFILE echo $DESTDIR if [ -s $SOURCEFILE ] then #echo Copying from local mkdir -p $DESTDIR >> $OUTFILE cp -af $SOURCEFILE $DESTFILE >> $OUTFILE else SOURCEFILE2=`echo $4 | sed "s|$2|$3|"` DESTFILE2=$5 DESTDIR2=`echo $DESTFILE2 | sed 's:\(/.*/\).*:\1:'` #echo $SOURCEFILE2 #echo $DESTFILE2 if [ -s $SOURCEFILE2 ] then mkdir -p $DESTDIR2 >> $OUTFILE cp -af $SOURCEFILE2 $DESTFILE2 >> $OUTFILE fi fi Benjamin Pierce wrote: > Hi Duane, > > Could you re-post this with the shell script appended in-line instead > of attached? And/or perhaps just put this explanation in the FAQ wiki > > http://alliance.seas.upenn.edu/~bcpierce/wiki/index.php > > Thanks! > > - Benjamin > > > On Jan 10, 2009, at 5:05 PM, Duane McKinney wrote: > >> This message has been automatically modified by CETS's antivirus/ >> antispam >> filter. If you have questions about this, please send mail to cets. >> >> An attachment named sneakernet.sh was removed from this document as it >> constituted a security hazard. Send mail to cets if you have >> questions about this policy. >> >>> That's a nice hack -- I had no idea the new copyprog functionality >>> could be used that way. >> That is what I love about linux/unix. Lots of small programs that >> can be combined to do great things. >> >> >> Didn't have quite as much free time as I though, but here are he >> results of my initial testing. It mostly works, and for now works >> well enough for me. I can now cron my syncs and not have to worry >> that some large file is going to kill our pipe for a few days :) >> I'll report back again in a month or so, or sooner if I run into an >> issue and have to change something. >> >> I have attached the shell script and the profile I used for testing. >> The important parts are: >> copyprog = sneakernet.sh /home/Duane/test >> duane at 192.168.113.2:/home/duane/test /home/Duane/usbdrive >> copyprogrest = sneakernet.sh >> >> This replaces what would normally be rsync with the attached shell >> script. >> >> I have only done limited testing so far, but I don't see anything >> that could be harmful. >> >> To be useful you will need 2 profiles a normal and a sneakernet. In >> your normal profile you should have >> copyprog = rsync --inplace --compress --max-size=xxx >> copyprogrest = rsync --partial --inplace --compress --max-size=xxx >> >> Where xxx is the size you have determined is too much to transfer >> via network. For me it is 1073741824 = 1GB >> >> The sneakernet profile should be identical except for the copyprog >> args >> copyprog = sneakernet.sh LocalRoot RemoteRoot SneakerNetPath >> copyprogrest = sneakernet.sh >> LocalRoot is the path on the local machine you want to sync >> RemoteRoot is not formatted the same as unison, it is formatted as >> passed by unison to rsync >> SneakerNetPath is the path to a directory on some removable media >> >> so now you will have 2 command to get everything synched >> unison myprofile >> unison myprofilesneakernet >> >> Issues: >> On copying from the modified machine, unison complains that the copy >> did not succeed and moves on, so only one file for each directory >> will be copied to the usb drive in a directory that is out of sync. >> This is not a big deal for me, but it may be for others. If it >> turns out to be a problem I'll probably dig through the source and >> come up with a real solution rather than this hack/workaround. >> >> I have not done any testing with copyprogrest. Since we are using >> local disks at all times, I don't know if this will be needed. >> >> bcpierce at seas.upenn.edu wrote: >>> That's a nice hack -- I had no idea the new copyprog functionality >>> could be used that way. Will be interested to hear whether it >>> works... >>> - Benjamin >>> Quoting Duane McKinney : >>>> Or from Version 2.31.4 >>>> copyprog xxx >>>> A string giving the name of an external program that can be >>>> used to >>>> copy large files efficiently (plus command-line switches telling >>>> it to >>>> copy files in-place). The default setting invokes rsync with >>>> appropriate >>>> options?most users should not need to change it. >>>> >>>> copyprogrest xxx >>>> A variant of copyprog that names an external program that >>>> should be >>>> used to continue the transfer of a large file that has already been >>>> partially transferred. Typically, copyprogrest will just be copyprog >>>> with one extra option (e.g., ?partial, for rsync). The default >>>> setting >>>> invokes rsync with appropriate options?most users should not need to >>>> change it. >>>> >>>> copythreshold n >>>> A number indicating above what filesize (in kilobytes) Unison >>>> should use the external copying utility specified by copyprog. >>>> Specifying 0 will cause all copies to use the external program; a >>>> negative number will prevent any files from using it. The default >>>> is -1. >>>> See the Making Unison Faster on Large Files section for more >>>> information. >>>> >>>> I guess I should have checked out the Beta Documentation 1st. >>>> I'll try >>>> messing around with using a shell script as the copyprog. I would >>>> assume that it would work just fine. My plan is to write a shell >>>> script >>>> which ignores the destination unison tells it, and instead copies >>>> it to >>>> a usb drive. >>>> >>>> I'll probably report back on this in a week or so. Currently I am >>>> running 2.27, so I have some compiling and testing to do. >>>> >>>> Duane McKinney wrote: >>>>> What about a flag that tells it to instead of executing the >>>>> copies, just >>>>> prints out a list of the files that would be copied? A proper >>>>> name is >>>>> escaping me at the moment, but it is a common option on programs. >>>>> >>>>> Benjamin Pierce wrote: >>>>>> Both of these features would be easy to implement using >>>>>> information >>>>>> that Unison already has available. If you want to give it a >>>>>> try, I >>>>>> can tell you where to start. (I'd be reluctant, though, to add >>>>>> this >>>>>> code to the main sources -- Unison arguably has too many flags and >>>>>> switches already!) >>>>>> >>>>>> Best, >>>>>> >>>>>> - Benjamin >>>>>> >>>>>> On Dec 27, 2008, at 10:27 AM, Duane McKinney wrote: >>>>>> >>>>>>> I searched, but came up empty. Can this be done. >>>>>>> I sync two offices using unison. >>>>>>> 1) (Optional)I would like to be able to set a preference that >>>>>>> says, >>>>>>> don't try to >>>>>>> sync a file if it would require X bytes transferred >>>>>>> 2) Get a list of changed files from a root. That way I can >>>>>>> copy the >>>>>>> files to a >>>>>>> removable drive and sync them when I get to the other office. >>>>>>> >>>>>>> Here is my reasoning. Most of the time, root changes are very >>>>>>> small, a few MB. >>>>>>> But Let's say that I download a new CENTOS release and place it >>>>>>> on >>>>>>> the file >>>>>>> server. This may be a poor example, because I could retrieve it >>>>>>> over the >>>>>>> internet again, but just bear with me. I would like for >>>>>>> unison, to >>>>>>> instead of >>>>>>> trying to sync this file over the network to skip it. The I >>>>>>> would >>>>>>> like to be >>>>>>> able to say once a week, run a job, that would take the >>>>>>> differences >>>>>>> in the root >>>>>>> and sent them to a USB drive. I can then carry this drive to >>>>>>> location 2, and >>>>>>> update the root there. Then from my understanding, the network >>>>>>> sync would >>>>>>> detect that both roots are identical. >>>>>>> >>>>>>> That way, our bandwidth isn't being eaten up for hours/days, >>>>>>> trying >>>>>>> to perform a >>>>>>> sync that will most likely fail because it will take so long. >>>>>>> >>>>>>> Also, this would be more useful, than synchronizing the whole >>>>>>> root >>>>>>> to a usb >>>>>>> drive, because, the total of the data that I am synchronizing >>>>>>> is > >>>>>>> 1TB. I would >>>>>>> not mind moving a few GB (<100) via USB. I am trying to avoid >>>>>>> needing either, a >>>>>>> lot of time, or a bunch of USB drives (one for each root). >>>>>>> >>>>>>> I have not yet looked at the source, but I would assume that >>>>>>> most of >>>>>>> the items >>>>>>> required for this feature are already in place. Is it >>>>>>> feasible? Is >>>>>>> there >>>>>>> already a way? >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Unison-hackers mailing list >>>>>>> Unison-hackers at lists.seas.upenn.edu >>>>>>> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >>>> _______________________________________________ >>>> Unison-hackers mailing list >>>> Unison-hackers at lists.seas.upenn.edu >>>> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >>>> >>>> >>> _______________________________________________ >>> Unison-hackers mailing list >>> Unison-hackers at lists.seas.upenn.edu >>> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >> _______________________________________________ >> Unison-hackers mailing list >> Unison-hackers at lists.seas.upenn.edu >> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers From bcpierce at cis.upenn.edu Tue Jan 13 08:44:40 2009 From: bcpierce at cis.upenn.edu (Benjamin Pierce) Date: Tue, 13 Jan 2009 08:44:40 -0500 Subject: [Unison-hackers] Sneaker Net or Incremental Backup In-Reply-To: References: <495A0E6B.2040509@gmail.com> <495A126F.40205@gmail.com> <20081230111900.tkbedc8r8c04so40@webmail.seas.upenn.edu> <5233E32B-EFB2-4FF2-879E-6FF88BE5CFD3@cis.upenn.edu> Message-ID: <8DC30F30-62E9-4BCE-AA2B-52BA43EFFC56@cis.upenn.edu> Thanks, Duane! (Also for the FAQ entry!) On Jan 13, 2009, at 7:08 AM, Duane McKinney wrote: > $ cat sneakernet.sh > #!/bin/sh > > # Usage > # $1 = the local root > # $2 = the remote path that unison will pass to rsync > # $3 = the path used for the sneakernet, > # ie the path to the folder on a usb drive moved between > locations > # $4 = Souce of the copy as sent by unison > # $5 = Destination of the copy as sent by unison > > > # Debuggin stuff > OUTFILE=/dev/null > > SOURCEFILE=$4 > DESTFILE=`echo $4 | sed "s|$1|$3|"` > DESTDIR=`echo $DESTFILE | sed 's:\(/.*/\).*:\1:'` > > echo $SOURCEFILE > echo $DESTFILE > echo $DESTDIR > > if [ -s $SOURCEFILE ] > then > #echo Copying from local > mkdir -p $DESTDIR >> $OUTFILE > cp -af $SOURCEFILE $DESTFILE >> $OUTFILE > else > SOURCEFILE2=`echo $4 | sed "s|$2|$3|"` > DESTFILE2=$5 > DESTDIR2=`echo $DESTFILE2 | sed 's:\(/.*/\).*:\1:'` > > #echo $SOURCEFILE2 > #echo $DESTFILE2 > > if [ -s $SOURCEFILE2 ] > then > mkdir -p $DESTDIR2 >> $OUTFILE > cp -af $SOURCEFILE2 $DESTFILE2 >> $OUTFILE > fi > fi > > > Benjamin Pierce wrote: >> Hi Duane, >> >> Could you re-post this with the shell script appended in-line instead >> of attached? And/or perhaps just put this explanation in the FAQ >> wiki >> >> http://alliance.seas.upenn.edu/~bcpierce/wiki/index.php >> >> Thanks! >> >> - Benjamin >> >> >> On Jan 10, 2009, at 5:05 PM, Duane McKinney wrote: >> >>> This message has been automatically modified by CETS's antivirus/ >>> antispam >>> filter. If you have questions about this, please send mail to cets. >>> >>> An attachment named sneakernet.sh was removed from this document >>> as it >>> constituted a security hazard. Send mail to cets if you have >>> questions about this policy. >>> >>>> That's a nice hack -- I had no idea the new copyprog functionality >>>> could be used that way. >>> That is what I love about linux/unix. Lots of small programs that >>> can be combined to do great things. >>> >>> >>> Didn't have quite as much free time as I though, but here are he >>> results of my initial testing. It mostly works, and for now works >>> well enough for me. I can now cron my syncs and not have to worry >>> that some large file is going to kill our pipe for a few days :) >>> I'll report back again in a month or so, or sooner if I run into an >>> issue and have to change something. >>> >>> I have attached the shell script and the profile I used for testing. >>> The important parts are: >>> copyprog = sneakernet.sh /home/Duane/test >>> duane at 192.168.113.2:/home/duane/test /home/Duane/usbdrive >>> copyprogrest = sneakernet.sh >>> >>> This replaces what would normally be rsync with the attached shell >>> script. >>> >>> I have only done limited testing so far, but I don't see anything >>> that could be harmful. >>> >>> To be useful you will need 2 profiles a normal and a sneakernet. In >>> your normal profile you should have >>> copyprog = rsync --inplace --compress --max-size=xxx >>> copyprogrest = rsync --partial --inplace --compress --max- >>> size=xxx >>> >>> Where xxx is the size you have determined is too much to transfer >>> via network. For me it is 1073741824 = 1GB >>> >>> The sneakernet profile should be identical except for the copyprog >>> args >>> copyprog = sneakernet.sh LocalRoot RemoteRoot SneakerNetPath >>> copyprogrest = sneakernet.sh >>> LocalRoot is the path on the local machine you want to sync >>> RemoteRoot is not formatted the same as unison, it is formatted as >>> passed by unison to rsync >>> SneakerNetPath is the path to a directory on some removable media >>> >>> so now you will have 2 command to get everything synched >>> unison myprofile >>> unison myprofilesneakernet >>> >>> Issues: >>> On copying from the modified machine, unison complains that the copy >>> did not succeed and moves on, so only one file for each directory >>> will be copied to the usb drive in a directory that is out of sync. >>> This is not a big deal for me, but it may be for others. If it >>> turns out to be a problem I'll probably dig through the source and >>> come up with a real solution rather than this hack/workaround. >>> >>> I have not done any testing with copyprogrest. Since we are using >>> local disks at all times, I don't know if this will be needed. >>> >>> bcpierce at seas.upenn.edu wrote: >>>> That's a nice hack -- I had no idea the new copyprog functionality >>>> could be used that way. Will be interested to hear whether it >>>> works... >>>> - Benjamin >>>> Quoting Duane McKinney : >>>>> Or from Version 2.31.4 >>>>> copyprog xxx >>>>> A string giving the name of an external program that can be >>>>> used to >>>>> copy large files efficiently (plus command-line switches telling >>>>> it to >>>>> copy files in-place). The default setting invokes rsync with >>>>> appropriate >>>>> options?most users should not need to change it. >>>>> >>>>> copyprogrest xxx >>>>> A variant of copyprog that names an external program that >>>>> should be >>>>> used to continue the transfer of a large file that has already >>>>> been >>>>> partially transferred. Typically, copyprogrest will just be >>>>> copyprog >>>>> with one extra option (e.g., ?partial, for rsync). The default >>>>> setting >>>>> invokes rsync with appropriate options?most users should not >>>>> need to >>>>> change it. >>>>> >>>>> copythreshold n >>>>> A number indicating above what filesize (in kilobytes) Unison >>>>> should use the external copying utility specified by copyprog. >>>>> Specifying 0 will cause all copies to use the external program; a >>>>> negative number will prevent any files from using it. The default >>>>> is -1. >>>>> See the Making Unison Faster on Large Files section for more >>>>> information. >>>>> >>>>> I guess I should have checked out the Beta Documentation 1st. >>>>> I'll try >>>>> messing around with using a shell script as the copyprog. I would >>>>> assume that it would work just fine. My plan is to write a shell >>>>> script >>>>> which ignores the destination unison tells it, and instead copies >>>>> it to >>>>> a usb drive. >>>>> >>>>> I'll probably report back on this in a week or so. Currently I am >>>>> running 2.27, so I have some compiling and testing to do. >>>>> >>>>> Duane McKinney wrote: >>>>>> What about a flag that tells it to instead of executing the >>>>>> copies, just >>>>>> prints out a list of the files that would be copied? A proper >>>>>> name is >>>>>> escaping me at the moment, but it is a common option on programs. >>>>>> >>>>>> Benjamin Pierce wrote: >>>>>>> Both of these features would be easy to implement using >>>>>>> information >>>>>>> that Unison already has available. If you want to give it a >>>>>>> try, I >>>>>>> can tell you where to start. (I'd be reluctant, though, to add >>>>>>> this >>>>>>> code to the main sources -- Unison arguably has too many flags >>>>>>> and >>>>>>> switches already!) >>>>>>> >>>>>>> Best, >>>>>>> >>>>>>> - Benjamin >>>>>>> >>>>>>> On Dec 27, 2008, at 10:27 AM, Duane McKinney wrote: >>>>>>> >>>>>>>> I searched, but came up empty. Can this be done. >>>>>>>> I sync two offices using unison. >>>>>>>> 1) (Optional)I would like to be able to set a preference that >>>>>>>> says, >>>>>>>> don't try to >>>>>>>> sync a file if it would require X bytes transferred >>>>>>>> 2) Get a list of changed files from a root. That way I can >>>>>>>> copy the >>>>>>>> files to a >>>>>>>> removable drive and sync them when I get to the other office. >>>>>>>> >>>>>>>> Here is my reasoning. Most of the time, root changes are very >>>>>>>> small, a few MB. >>>>>>>> But Let's say that I download a new CENTOS release and place it >>>>>>>> on >>>>>>>> the file >>>>>>>> server. This may be a poor example, because I could retrieve >>>>>>>> it >>>>>>>> over the >>>>>>>> internet again, but just bear with me. I would like for >>>>>>>> unison, to >>>>>>>> instead of >>>>>>>> trying to sync this file over the network to skip it. The I >>>>>>>> would >>>>>>>> like to be >>>>>>>> able to say once a week, run a job, that would take the >>>>>>>> differences >>>>>>>> in the root >>>>>>>> and sent them to a USB drive. I can then carry this drive to >>>>>>>> location 2, and >>>>>>>> update the root there. Then from my understanding, the >>>>>>>> network >>>>>>>> sync would >>>>>>>> detect that both roots are identical. >>>>>>>> >>>>>>>> That way, our bandwidth isn't being eaten up for hours/days, >>>>>>>> trying >>>>>>>> to perform a >>>>>>>> sync that will most likely fail because it will take so long. >>>>>>>> >>>>>>>> Also, this would be more useful, than synchronizing the whole >>>>>>>> root >>>>>>>> to a usb >>>>>>>> drive, because, the total of the data that I am synchronizing >>>>>>>> is > >>>>>>>> 1TB. I would >>>>>>>> not mind moving a few GB (<100) via USB. I am trying to avoid >>>>>>>> needing either, a >>>>>>>> lot of time, or a bunch of USB drives (one for each root). >>>>>>>> >>>>>>>> I have not yet looked at the source, but I would assume that >>>>>>>> most of >>>>>>>> the items >>>>>>>> required for this feature are already in place. Is it >>>>>>>> feasible? Is >>>>>>>> there >>>>>>>> already a way? >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Unison-hackers mailing list >>>>>>>> Unison-hackers at lists.seas.upenn.edu >>>>>>>> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >>>>> _______________________________________________ >>>>> Unison-hackers mailing list >>>>> Unison-hackers at lists.seas.upenn.edu >>>>> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >>>>> >>>>> >>>> _______________________________________________ >>>> Unison-hackers mailing list >>>> Unison-hackers at lists.seas.upenn.edu >>>> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >>> _______________________________________________ >>> Unison-hackers mailing list >>> Unison-hackers at lists.seas.upenn.edu >>> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers > > _______________________________________________ > Unison-hackers mailing list > Unison-hackers at lists.seas.upenn.edu > http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers > From duane.mckinney at gmail.com Wed Jan 14 07:14:43 2009 From: duane.mckinney at gmail.com (Duane McKinney) Date: Wed, 14 Jan 2009 04:14:43 -0800 Subject: [Unison-hackers] Sneaker Net or Incremental Backup In-Reply-To: <8DC30F30-62E9-4BCE-AA2B-52BA43EFFC56@cis.upenn.edu> References: <495A0E6B.2040509@gmail.com> <495A126F.40205@gmail.com> <20081230111900.tkbedc8r8c04so40@webmail.seas.upenn.edu> <5233E32B-EFB2-4FF2-879E-6FF88BE5CFD3@cis.upenn.edu> <8DC30F30-62E9-4BCE-AA2B-52BA43EFFC56@cis.upenn.edu> Message-ID: np, Thank you for such a great tool. I updated the tips & tricks faq as well. Benjamin Pierce wrote: > Thanks, Duane! (Also for the FAQ entry!) > > On Jan 13, 2009, at 7:08 AM, Duane McKinney wrote: > >> $ cat sneakernet.sh >> #!/bin/sh >> >> # Usage >> # $1 = the local root >> # $2 = the remote path that unison will pass to rsync >> # $3 = the path used for the sneakernet, >> # ie the path to the folder on a usb drive moved between >> locations >> # $4 = Souce of the copy as sent by unison >> # $5 = Destination of the copy as sent by unison >> >> >> # Debuggin stuff >> OUTFILE=/dev/null >> >> SOURCEFILE=$4 >> DESTFILE=`echo $4 | sed "s|$1|$3|"` >> DESTDIR=`echo $DESTFILE | sed 's:\(/.*/\).*:\1:'` >> >> echo $SOURCEFILE >> echo $DESTFILE >> echo $DESTDIR >> >> if [ -s $SOURCEFILE ] >> then >> #echo Copying from local >> mkdir -p $DESTDIR >> $OUTFILE >> cp -af $SOURCEFILE $DESTFILE >> $OUTFILE >> else >> SOURCEFILE2=`echo $4 | sed "s|$2|$3|"` >> DESTFILE2=$5 >> DESTDIR2=`echo $DESTFILE2 | sed 's:\(/.*/\).*:\1:'` >> >> #echo $SOURCEFILE2 >> #echo $DESTFILE2 >> >> if [ -s $SOURCEFILE2 ] >> then >> mkdir -p $DESTDIR2 >> $OUTFILE >> cp -af $SOURCEFILE2 $DESTFILE2 >> $OUTFILE >> fi >> fi >> >> >> Benjamin Pierce wrote: >>> Hi Duane, >>> >>> Could you re-post this with the shell script appended in-line instead >>> of attached? And/or perhaps just put this explanation in the FAQ >>> wiki >>> >>> http://alliance.seas.upenn.edu/~bcpierce/wiki/index.php >>> >>> Thanks! >>> >>> - Benjamin >>> >>> >>> On Jan 10, 2009, at 5:05 PM, Duane McKinney wrote: >>> >>>> This message has been automatically modified by CETS's antivirus/ >>>> antispam >>>> filter. If you have questions about this, please send mail to cets. >>>> >>>> An attachment named sneakernet.sh was removed from this document >>>> as it >>>> constituted a security hazard. Send mail to cets if you have >>>> questions about this policy. >>>> >>>>> That's a nice hack -- I had no idea the new copyprog functionality >>>>> could be used that way. >>>> That is what I love about linux/unix. Lots of small programs that >>>> can be combined to do great things. >>>> >>>> >>>> Didn't have quite as much free time as I though, but here are he >>>> results of my initial testing. It mostly works, and for now works >>>> well enough for me. I can now cron my syncs and not have to worry >>>> that some large file is going to kill our pipe for a few days :) >>>> I'll report back again in a month or so, or sooner if I run into an >>>> issue and have to change something. >>>> >>>> I have attached the shell script and the profile I used for testing. >>>> The important parts are: >>>> copyprog = sneakernet.sh /home/Duane/test >>>> duane at 192.168.113.2:/home/duane/test /home/Duane/usbdrive >>>> copyprogrest = sneakernet.sh >>>> >>>> This replaces what would normally be rsync with the attached shell >>>> script. >>>> >>>> I have only done limited testing so far, but I don't see anything >>>> that could be harmful. >>>> >>>> To be useful you will need 2 profiles a normal and a sneakernet. In >>>> your normal profile you should have >>>> copyprog = rsync --inplace --compress --max-size=xxx >>>> copyprogrest = rsync --partial --inplace --compress --max- >>>> size=xxx >>>> >>>> Where xxx is the size you have determined is too much to transfer >>>> via network. For me it is 1073741824 = 1GB >>>> >>>> The sneakernet profile should be identical except for the copyprog >>>> args >>>> copyprog = sneakernet.sh LocalRoot RemoteRoot SneakerNetPath >>>> copyprogrest = sneakernet.sh >>>> LocalRoot is the path on the local machine you want to sync >>>> RemoteRoot is not formatted the same as unison, it is formatted as >>>> passed by unison to rsync >>>> SneakerNetPath is the path to a directory on some removable media >>>> >>>> so now you will have 2 command to get everything synched >>>> unison myprofile >>>> unison myprofilesneakernet >>>> >>>> Issues: >>>> On copying from the modified machine, unison complains that the copy >>>> did not succeed and moves on, so only one file for each directory >>>> will be copied to the usb drive in a directory that is out of sync. >>>> This is not a big deal for me, but it may be for others. If it >>>> turns out to be a problem I'll probably dig through the source and >>>> come up with a real solution rather than this hack/workaround. >>>> >>>> I have not done any testing with copyprogrest. Since we are using >>>> local disks at all times, I don't know if this will be needed. >>>> >>>> bcpierce at seas.upenn.edu wrote: >>>>> That's a nice hack -- I had no idea the new copyprog functionality >>>>> could be used that way. Will be interested to hear whether it >>>>> works... >>>>> - Benjamin >>>>> Quoting Duane McKinney : >>>>>> Or from Version 2.31.4 >>>>>> copyprog xxx >>>>>> A string giving the name of an external program that can be >>>>>> used to >>>>>> copy large files efficiently (plus command-line switches telling >>>>>> it to >>>>>> copy files in-place). The default setting invokes rsync with >>>>>> appropriate >>>>>> options?most users should not need to change it. >>>>>> >>>>>> copyprogrest xxx >>>>>> A variant of copyprog that names an external program that >>>>>> should be >>>>>> used to continue the transfer of a large file that has already >>>>>> been >>>>>> partially transferred. Typically, copyprogrest will just be >>>>>> copyprog >>>>>> with one extra option (e.g., ?partial, for rsync). The default >>>>>> setting >>>>>> invokes rsync with appropriate options?most users should not >>>>>> need to >>>>>> change it. >>>>>> >>>>>> copythreshold n >>>>>> A number indicating above what filesize (in kilobytes) Unison >>>>>> should use the external copying utility specified by copyprog. >>>>>> Specifying 0 will cause all copies to use the external program; a >>>>>> negative number will prevent any files from using it. The default >>>>>> is -1. >>>>>> See the Making Unison Faster on Large Files section for more >>>>>> information. >>>>>> >>>>>> I guess I should have checked out the Beta Documentation 1st. >>>>>> I'll try >>>>>> messing around with using a shell script as the copyprog. I would >>>>>> assume that it would work just fine. My plan is to write a shell >>>>>> script >>>>>> which ignores the destination unison tells it, and instead copies >>>>>> it to >>>>>> a usb drive. >>>>>> >>>>>> I'll probably report back on this in a week or so. Currently I am >>>>>> running 2.27, so I have some compiling and testing to do. >>>>>> >>>>>> Duane McKinney wrote: >>>>>>> What about a flag that tells it to instead of executing the >>>>>>> copies, just >>>>>>> prints out a list of the files that would be copied? A proper >>>>>>> name is >>>>>>> escaping me at the moment, but it is a common option on programs. >>>>>>> >>>>>>> Benjamin Pierce wrote: >>>>>>>> Both of these features would be easy to implement using >>>>>>>> information >>>>>>>> that Unison already has available. If you want to give it a >>>>>>>> try, I >>>>>>>> can tell you where to start. (I'd be reluctant, though, to add >>>>>>>> this >>>>>>>> code to the main sources -- Unison arguably has too many flags >>>>>>>> and >>>>>>>> switches already!) >>>>>>>> >>>>>>>> Best, >>>>>>>> >>>>>>>> - Benjamin >>>>>>>> >>>>>>>> On Dec 27, 2008, at 10:27 AM, Duane McKinney wrote: >>>>>>>> >>>>>>>>> I searched, but came up empty. Can this be done. >>>>>>>>> I sync two offices using unison. >>>>>>>>> 1) (Optional)I would like to be able to set a preference that >>>>>>>>> says, >>>>>>>>> don't try to >>>>>>>>> sync a file if it would require X bytes transferred >>>>>>>>> 2) Get a list of changed files from a root. That way I can >>>>>>>>> copy the >>>>>>>>> files to a >>>>>>>>> removable drive and sync them when I get to the other office. >>>>>>>>> >>>>>>>>> Here is my reasoning. Most of the time, root changes are very >>>>>>>>> small, a few MB. >>>>>>>>> But Let's say that I download a new CENTOS release and place it >>>>>>>>> on >>>>>>>>> the file >>>>>>>>> server. This may be a poor example, because I could retrieve >>>>>>>>> it >>>>>>>>> over the >>>>>>>>> internet again, but just bear with me. I would like for >>>>>>>>> unison, to >>>>>>>>> instead of >>>>>>>>> trying to sync this file over the network to skip it. The I >>>>>>>>> would >>>>>>>>> like to be >>>>>>>>> able to say once a week, run a job, that would take the >>>>>>>>> differences >>>>>>>>> in the root >>>>>>>>> and sent them to a USB drive. I can then carry this drive to >>>>>>>>> location 2, and >>>>>>>>> update the root there. Then from my understanding, the >>>>>>>>> network >>>>>>>>> sync would >>>>>>>>> detect that both roots are identical. >>>>>>>>> >>>>>>>>> That way, our bandwidth isn't being eaten up for hours/days, >>>>>>>>> trying >>>>>>>>> to perform a >>>>>>>>> sync that will most likely fail because it will take so long. >>>>>>>>> >>>>>>>>> Also, this would be more useful, than synchronizing the whole >>>>>>>>> root >>>>>>>>> to a usb >>>>>>>>> drive, because, the total of the data that I am synchronizing >>>>>>>>> is > >>>>>>>>> 1TB. I would >>>>>>>>> not mind moving a few GB (<100) via USB. I am trying to avoid >>>>>>>>> needing either, a >>>>>>>>> lot of time, or a bunch of USB drives (one for each root). >>>>>>>>> >>>>>>>>> I have not yet looked at the source, but I would assume that >>>>>>>>> most of >>>>>>>>> the items >>>>>>>>> required for this feature are already in place. Is it >>>>>>>>> feasible? Is >>>>>>>>> there >>>>>>>>> already a way? >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Unison-hackers mailing list >>>>>>>>> Unison-hackers at lists.seas.upenn.edu >>>>>>>>> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >>>>>> _______________________________________________ >>>>>> Unison-hackers mailing list >>>>>> Unison-hackers at lists.seas.upenn.edu >>>>>> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> Unison-hackers mailing list >>>>> Unison-hackers at lists.seas.upenn.edu >>>>> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >>>> _______________________________________________ >>>> Unison-hackers mailing list >>>> Unison-hackers at lists.seas.upenn.edu >>>> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >> _______________________________________________ >> Unison-hackers mailing list >> Unison-hackers at lists.seas.upenn.edu >> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >> From bcpierce at seas.upenn.edu Wed Jan 14 22:30:49 2009 From: bcpierce at seas.upenn.edu (Benjamin C. Pierce) Date: Wed, 14 Jan 2009 22:30:49 -0500 Subject: [Unison-hackers] [unison-svn] r315 - trunk/src Message-ID: <200901150330.n0F3UnSi006178@yaws.seas.upenn.edu> Author: bcpierce Date: 2009-01-14 22:30:46 -0500 (Wed, 14 Jan 2009) New Revision: 315 Modified: trunk/src/Makefile.OCaml trunk/src/NEWS trunk/src/RECENTNEWS trunk/src/TODO.txt trunk/src/copy.ml trunk/src/mkProjectInfo.ml trunk/src/strings.ml trunk/src/update.ml Log: * Applied a patch from Karl M to make the GTK2 version build with OCaml 3.11 on Windows. * Don't use shortcuts or external copy programs to transfer zero-length files (to avoid confusing status messages about shortcut-copying completely unrelated files that happen to also have zero length!). Modified: trunk/src/Makefile.OCaml =================================================================== --- trunk/src/Makefile.OCaml 2008-12-01 14:00:40 UTC (rev 314) +++ trunk/src/Makefile.OCaml 2009-01-15 03:30:46 UTC (rev 315) @@ -91,8 +91,15 @@ EXEC_EXT=.exe OBJ_EXT=.obj CWD=. - CLIBS+=-cclib win32rc/unison.res - STATICLIBS+=-cclib win32rc/unison.res +# Fix suggested by Karl M, Jan 2009: +# "The new flexlink wrapper that OCaml 3.11 uses was gagging on the res +# file. So the res file has to be passed through flexlink untouched to +# the linker. I only touched the MSVC side, but mingw may have the same +# issue." +# CLIBS+=-cclib win32rc/unison.res +# STATICLIBS+=-cclib win32rc/unison.res + CLIBS+=-cclib "-link win32rc/unison.res" + STATICLIBS+=-cclib "-link win32rc/unison.res" buildexecutable:: @echo Building for Windows else Modified: trunk/src/NEWS =================================================================== --- trunk/src/NEWS 2008-12-01 14:00:40 UTC (rev 314) +++ trunk/src/NEWS 2009-01-15 03:30:46 UTC (rev 315) @@ -1,5 +1,5 @@ -Changes in Version 2.31.4 +Changes in Version 2.32.1 Changes since 2.17: * Major rewrite and cleanup of the whole Mac OS X graphical user Modified: trunk/src/RECENTNEWS =================================================================== --- trunk/src/RECENTNEWS 2008-12-01 14:00:40 UTC (rev 314) +++ trunk/src/RECENTNEWS 2009-01-15 03:30:46 UTC (rev 315) @@ -1,3 +1,20 @@ +CHANGES FROM VERSION 2.32.1 + +* Applied a patch from Karl M to make the GTK2 version build with + OCaml 3.11 on Windows. + +* Don't use shortcuts or external copy programs to transfer + zero-length files (to avoid confusing status messages about + shortcut-copying completely unrelated files that happen to also have + zero length!). + + + + + + + +------------------------------- CHANGES FROM VERSION 2.32.0 * Bumped version number to reflect newly added preference Modified: trunk/src/TODO.txt =================================================================== --- trunk/src/TODO.txt 2008-12-01 14:00:40 UTC (rev 314) +++ trunk/src/TODO.txt 2009-01-15 03:30:46 UTC (rev 315) @@ -17,6 +17,7 @@ - The function that runs the external program should not grab stdin / stdout / stderr if Unison is running with the text UI. - The confirmation step should offer to display the new merged file. + - (There are some older merge issues documented below) * Makefile for fstest @@ -87,6 +88,15 @@ **** When deleting a directory, we should *not* skip over Unison temp files in the process of listing children +*** Un-writeable directories can't be copied. + The 'rename' operation at the end of Files.copy will fail (at least on + OSX) if the path being renamed points to a directory and that directory + (not the one containing it!) is not writeable by the user. To fix this, + we'd need to notice when we are renaming a directory and temporarily + make it writeable just before the rename and then make it what it should + be just after. But I don't feel like writing this bit of code right + now, to handle such a corner case. [BCP, November 2008] + *** make the ETA bar show which file is actually transferring bytes at the moment Modified: trunk/src/copy.ml =================================================================== --- trunk/src/copy.ml 2008-12-01 14:00:40 UTC (rev 314) +++ trunk/src/copy.ml 2009-01-15 03:30:46 UTC (rev 315) @@ -671,9 +671,11 @@ end else if Prefs.read copyprog <> "" && Prefs.read copythreshold >= 0 - && Props.length desc >= Uutil.Filesize.ofInt64 - (Int64.mul (Int64.of_int 1000) - (Int64.of_int (Prefs.read copythreshold))) + && Props.length desc >= Uutil.Filesize.ofInt64 (Int64.of_int 1) + && Props.length desc >= + Uutil.Filesize.ofInt64 + (Int64.mul (Int64.of_int 1000) + (Int64.of_int (Prefs.read copythreshold))) && update = `Copy then begin (* First use the external program to copy the data fork *) Modified: trunk/src/mkProjectInfo.ml =================================================================== --- trunk/src/mkProjectInfo.ml 2008-12-01 14:00:40 UTC (rev 314) +++ trunk/src/mkProjectInfo.ml 2009-01-15 03:30:46 UTC (rev 315) @@ -99,3 +99,4 @@ + Modified: trunk/src/strings.ml =================================================================== --- trunk/src/strings.ml 2008-12-01 14:00:40 UTC (rev 314) +++ trunk/src/strings.ml 2009-01-15 03:30:46 UTC (rev 315) @@ -4,7 +4,7 @@ let docs = ("about", ("About Unison", "Unison File Synchronizer\n\ - Version 2.31.4\n\ + Version 2.32.1\n\ \n\ ")) :: @@ -1148,9 +1148,12 @@ \032-contactquietly suppress the 'contacting server' message during startup\n\ \032-copyprog xxx external program for copying large files\n\ \032-copyprogrest xxx variant of copyprog for resuming partial transfers\n\ + \032-copyquoterem xxx add quotes to remote file name for copyprog (true/false/def\n\ + ault)\n\ \032-copythreshold n use copyprog on files bigger than this (if >=0, in Kb)\n\ \032-debug xxx debug module xxx ('all' -> everything, 'verbose' -> more)\n\ \032-diff xxx command for showing differences between files\n\ + \032-dontchmod When set, never use the chmod system call\n\ \032-dumbtty do not change terminal settings in text UI (default true)\n\ \032-fastcheck xxx do fast update detection (true/false/default)\n\ \032-forcepartial xxx add a pattern to the forcepartial list\n\ @@ -1263,6 +1266,11 @@ \032 prefix; if it does not appear anywhere in the prefix or the\n\ \032 suffix, it will be automatically placed at the beginning of the\n\ \032 suffix.\n\ + \032 One thing to be careful of: If the backuploc preference is set\n\ + \032 to local, Unison will automatically ignore all files whose\n\ + \032 prefix and suffix match backupprefix and backupsuffix. So be\n\ + \032 careful to choose values for these preferences that are\n\ + \032 sufficiently different from the names of your real files.\n\ \032 backups \n\ \032 Setting this flag to true is equivalent to setting\n\ \032 backuplocation to local and backup to Name *.\n\ @@ -1305,6 +1313,14 @@ \032 will just be copyprog with one extra option (e.g., -partial,\n\ \032 for rsync). The default setting invokes rsync with appropriate\n\ \032 options--most users should not need to change it.\n\ + \032 copyquoterem xxx\n\ + \032 When set to true, this flag causes Unison to add an extra layer\n\ + \032 of quotes to the remote path passed to the external copy\n\ + \032 program. This is needed by rsync, for example, which internally\n\ + \032 uses an ssh connection requiring an extra level of quoting for\n\ + \032 paths containing spaces. When this flag is set to default,\n\ + \032 extra quotes are added if the value of copyprog contains the\n\ + \032 string rsync.\n\ \032 copythreshold n\n\ \032 A number indicating above what filesize (in kilobytes) Unison\n\ \032 should use the external copying utility specified by copyprog.\n\ @@ -1341,6 +1357,12 @@ \032 exactly the same information as the printed and HTML manuals,\n\ \032 modulo formatting. Use -doc topics to obtain a list of the\n\ \032 names of the various sections that can be printed.\n\ + \032 dontchmod \n\ + \032 By default, Unison uses the 'chmod' system call to set the\n\ + \032 permission bits of files after it has copied them. But in some\n\ + \032 circumstances (and under some operating systems), the chmod\n\ + \032 call always fails. Setting this preference completely prevents\n\ + \032 Unison from ever calling chmod.\n\ \032 dumbtty \n\ \032 When set to true, this flag makes the text mode user interface\n\ \032 avoid trying to change any of the terminal settings. (Normally,\n\ @@ -2560,8 +2582,8 @@ \n\ ")) :: - ("news", ("Changes in Version 2.31.4", - "Changes in Version 2.31.4\n\ + ("news", ("Changes in Version 2.32.1", + "Changes in Version 2.32.1\n\ \n\ \032 Changes since 2.17:\n\ \032 * Major rewrite and cleanup of the whole Mac OS X graphical user\n\ Modified: trunk/src/update.ml =================================================================== --- trunk/src/update.ml 2008-12-01 14:00:40 UTC (rev 314) +++ trunk/src/update.ml 2009-01-15 03:30:46 UTC (rev 315) @@ -21,6 +21,8 @@ format is modified (see file props.ml for the new function) *) (*FIX: also change Fileinfo.stamp to drop the info.ctime component, next time the format is modified *) +(*FIX: also make Jerome's suggested change about file times (see his mesg in + unison-pending email folder). *) let archiveFormat = 22 module NameMap = MyMap.Make (Name) From bcpierce at seas.upenn.edu Wed Jan 14 22:34:28 2009 From: bcpierce at seas.upenn.edu (Benjamin C. Pierce) Date: Wed, 14 Jan 2009 22:34:28 -0500 Subject: [Unison-hackers] [unison-svn] r316 - in branches/2.27: . doc src src/lwt Message-ID: <200901150334.n0F3YSNN006317@yaws.seas.upenn.edu> Author: bcpierce Date: 2009-01-14 22:34:22 -0500 (Wed, 14 Jan 2009) New Revision: 316 Removed: branches/2.27/COPYING Modified: branches/2.27/Makefile branches/2.27/doc/unison-manual.tex branches/2.27/src/Makefile branches/2.27/src/Makefile.OCaml branches/2.27/src/RECENTNEWS branches/2.27/src/lwt/Makefile branches/2.27/src/mkProjectInfo.ml branches/2.27/src/pty.c branches/2.27/src/remote.ml Log: * Applied a patch from Karl M to make the GTK2 version build with OCaml 3.11 on Windows. * Added some extra debugging code to remote.ml to give more informative error messages when people encounter the longstanding "assert failed during file transfer" bug. * Applied patch from Antoine Reilles for NetBSD compilation * Makefile tidying Deleted: branches/2.27/COPYING =================================================================== --- branches/2.27/COPYING 2009-01-15 03:30:46 UTC (rev 315) +++ branches/2.27/COPYING 2009-01-15 03:34:22 UTC (rev 316) @@ -1,341 +0,0 @@ - GNU GENERAL PUBLIC LICENSE - Version 2, June 1991 - - Copyright (C) 1989, 1991 Free Software Foundation, Inc. - 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - Everyone is permitted to copy and distribute verbatim copies - of this license document, but changing it is not allowed. - - Preamble - - The licenses for most software are designed to take away your -freedom to share and change it. By contrast, the GNU General Public -License is intended to guarantee your freedom to share and change free -software--to make sure the software is free for all its users. This -General Public License applies to most of the Free Software -Foundation's software and to any other program whose authors commit to -using it. (Some other Free Software Foundation software is covered by -the GNU Library General Public License instead.) You can apply it to -your programs, too. - - When we speak of free software, we are referring to freedom, not -price. Our General Public Licenses are designed to make sure that you -have the freedom to distribute copies of free software (and charge for -this service if you wish), that you receive source code or can get it -if you want it, that you can change the software or use pieces of it -in new free programs; and that you know you can do these things. - - To protect your rights, we need to make restrictions that forbid -anyone to deny you these rights or to ask you to surrender the rights. -These restrictions translate to certain responsibilities for you if you -distribute copies of the software, or if you modify it. - - For example, if you distribute copies of such a program, whether -gratis or for a fee, you must give the recipients all the rights that -you have. You must make sure that they, too, receive or can get the -source code. And you must show them these terms so they know their -rights. - - We protect your rights with two steps: (1) copyright the software, and -(2) offer you this license which gives you legal permission to copy, -distribute and/or modify the software. - - Also, for each author's protection and ours, we want to make certain -that everyone understands that there is no warranty for this free -software. If the software is modified by someone else and passed on, we -want its recipients to know that what they have is not the original, so -that any problems introduced by others will not reflect on the original -authors' reputations. - - Finally, any free program is threatened constantly by software -patents. We wish to avoid the danger that redistributors of a free -program will individually obtain patent licenses, in effect making the -program proprietary. To prevent this, we have made it clear that any -patent must be licensed for everyone's free use or not licensed at all. - - The precise terms and conditions for copying, distribution and -modification follow. - - GNU GENERAL PUBLIC LICENSE - TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION - - 0. This License applies to any program or other work which contains -a notice placed by the copyright holder saying it may be distributed -under the terms of this General Public License. The "Program", below, -refers to any such program or work, and a "work based on the Program" -means either the Program or any derivative work under copyright law: -that is to say, a work containing the Program or a portion of it, -either verbatim or with modifications and/or translated into another -language. (Hereinafter, translation is included without limitation in -the term "modification".) Each licensee is addressed as "you". - -Activities other than copying, distribution and modification are not -covered by this License; they are outside its scope. The act of -running the Program is not restricted, and the output from the Program -is covered only if its contents constitute a work based on the -Program (independent of having been made by running the Program). -Whether that is true depends on what the Program does. - - 1. You may copy and distribute verbatim copies of the Program's -source code as you receive it, in any medium, provided that you -conspicuously and appropriately publish on each copy an appropriate -copyright notice and disclaimer of warranty; keep intact all the -notices that refer to this License and to the absence of any warranty; -and give any other recipients of the Program a copy of this License -along with the Program. - -You may charge a fee for the physical act of transferring a copy, and -you may at your option offer warranty protection in exchange for a fee. - - 2. You may modify your copy or copies of the Program or any portion -of it, thus forming a work based on the Program, and copy and -distribute such modifications or work under the terms of Section 1 -above, provided that you also meet all of these conditions: - - a) You must cause the modified files to carry prominent notices - stating that you changed the files and the date of any change. - - b) You must cause any work that you distribute or publish, that in - whole or in part contains or is derived from the Program or any - part thereof, to be licensed as a whole at no charge to all third - parties under the terms of this License. - - c) If the modified program normally reads commands interactively - when run, you must cause it, when started running for such - interactive use in the most ordinary way, to print or display an - announcement including an appropriate copyright notice and a - notice that there is no warranty (or else, saying that you provide - a warranty) and that users may redistribute the program under - these conditions, and telling the user how to view a copy of this - License. (Exception: if the Program itself is interactive but - does not normally print such an announcement, your work based on - the Program is not required to print an announcement.) - -These requirements apply to the modified work as a whole. If -identifiable sections of that work are not derived from the Program, -and can be reasonably considered independent and separate works in -themselves, then this License, and its terms, do not apply to those -sections when you distribute them as separate works. But when you -distribute the same sections as part of a whole which is a work based -on the Program, the distribution of the whole must be on the terms of -this License, whose permissions for other licensees extend to the -entire whole, and thus to each and every part regardless of who wrote it. - -Thus, it is not the intent of this section to claim rights or contest -your rights to work written entirely by you; rather, the intent is to -exercise the right to control the distribution of derivative or -collective works based on the Program. - -In addition, mere aggregation of another work not based on the Program -with the Program (or with a work based on the Program) on a volume of -a storage or distribution medium does not bring the other work under -the scope of this License. - - 3. You may copy and distribute the Program (or a work based on it, -under Section 2) in object code or executable form under the terms of -Sections 1 and 2 above provided that you also do one of the following: - - a) Accompany it with the complete corresponding machine-readable - source code, which must be distributed under the terms of Sections - 1 and 2 above on a medium customarily used for software interchange; or, - - b) Accompany it with a written offer, valid for at least three - years, to give any third party, for a charge no more than your - cost of physically performing source distribution, a complete - machine-readable copy of the corresponding source code, to be - distributed under the terms of Sections 1 and 2 above on a medium - customarily used for software interchange; or, - - c) Accompany it with the information you received as to the offer - to distribute corresponding source code. (This alternative is - allowed only for noncommercial distribution and only if you - received the program in object code or executable form with such - an offer, in accord with Subsection b above.) - -The source code for a work means the preferred form of the work for -making modifications to it. For an executable work, complete source -code means all the source code for all modules it contains, plus any -associated interface definition files, plus the scripts used to -control compilation and installation of the executable. However, as a -special exception, the source code distributed need not include -anything that is normally distributed (in either source or binary -form) with the major components (compiler, kernel, and so on) of the -operating system on which the executable runs, unless that component -itself accompanies the executable. - -If distribution of executable or object code is made by offering -access to copy from a designated place, then offering equivalent -access to copy the source code from the same place counts as -distribution of the source code, even though third parties are not -compelled to copy the source along with the object code. - - 4. You may not copy, modify, sublicense, or distribute the Program -except as expressly provided under this License. Any attempt -otherwise to copy, modify, sublicense or distribute the Program is -void, and will automatically terminate your rights under this License. -However, parties who have received copies, or rights, from you under -this License will not have their licenses terminated so long as such -parties remain in full compliance. - - 5. You are not required to accept this License, since you have not -signed it. However, nothing else grants you permission to modify or -distribute the Program or its derivative works. These actions are -prohibited by law if you do not accept this License. Therefore, by -modifying or distributing the Program (or any work based on the -Program), you indicate your acceptance of this License to do so, and -all its terms and conditions for copying, distributing or modifying -the Program or works based on it. - - 6. Each time you redistribute the Program (or any work based on the -Program), the recipient automatically receives a license from the -original licensor to copy, distribute or modify the Program subject to -these terms and conditions. You may not impose any further -restrictions on the recipients' exercise of the rights granted herein. -You are not responsible for enforcing compliance by third parties to -this License. - - 7. If, as a consequence of a court judgment or allegation of patent -infringement or for any other reason (not limited to patent issues), -conditions are imposed on you (whether by court order, agreement or -otherwise) that contradict the conditions of this License, they do not -excuse you from the conditions of this License. If you cannot -distribute so as to satisfy simultaneously your obligations under this -License and any other pertinent obligations, then as a consequence you -may not distribute the Program at all. For example, if a patent -license would not permit royalty-free redistribution of the Program by -all those who receive copies directly or indirectly through you, then -the only way you could satisfy both it and this License would be to -refrain entirely from distribution of the Program. - -If any portion of this section is held invalid or unenforceable under -any particular circumstance, the balance of the section is intended to -apply and the section as a whole is intended to apply in other -circumstances. - -It is not the purpose of this section to induce you to infringe any -patents or other property right claims or to contest validity of any -such claims; this section has the sole purpose of protecting the -integrity of the free software distribution system, which is -implemented by public license practices. Many people have made -generous contributions to the wide range of software distributed -through that system in reliance on consistent application of that -system; it is up to the author/donor to decide if he or she is willing -to distribute software through any other system and a licensee cannot -impose that choice. - -This section is intended to make thoroughly clear what is believed to -be a consequence of the rest of this License. - - 8. If the distribution and/or use of the Program is restricted in -certain countries either by patents or by copyrighted interfaces, the -original copyright holder who places the Program under this License -may add an explicit geographical distribution limitation excluding -those countries, so that distribution is permitted only in or among -countries not thus excluded. In such case, this License incorporates -the limitation as if written in the body of this License. - - 9. The Free Software Foundation may publish revised and/or new versions -of the General Public License from time to time. Such new versions will -be similar in spirit to the present version, but may differ in detail to -address new problems or concerns. - -Each version is given a distinguishing version number. If the Program -specifies a version number of this License which applies to it and "any -later version", you have the option of following the terms and conditions -either of that version or of any later version published by the Free -Software Foundation. If the Program does not specify a version number of -this License, you may choose any version ever published by the Free Software -Foundation. - - 10. If you wish to incorporate parts of the Program into other free -programs whose distribution conditions are different, write to the author -to ask for permission. For software which is copyrighted by the Free -Software Foundation, write to the Free Software Foundation; we sometimes -make exceptions for this. Our decision will be guided by the two goals -of preserving the free status of all derivatives of our free software and -of promoting the sharing and reuse of software generally. - - NO WARRANTY - - 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY -FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN -OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES -PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED -OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF -MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS -TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE -PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, -REPAIR OR CORRECTION. - - 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING -WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR -REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, -INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING -OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED -TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY -YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER -PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE -POSSIBILITY OF SUCH DAMAGES. - - END OF TERMS AND CONDITIONS - - How to Apply These Terms to Your New Programs - - If you develop a new program, and you want it to be of the greatest -possible use to the public, the best way to achieve this is to make it -free software which everyone can redistribute and change under these terms. - - To do so, attach the following notices to the program. It is safest -to attach them to the start of each source file to most effectively -convey the exclusion of warranty; and each file should have at least -the "copyright" line and a pointer to where the full notice is found. - - - Copyright (C) 19yy - - This program is free software; you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation; either version 2 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - - -Also add information on how to contact you by electronic and paper mail. - -If the program is interactive, make it output a short notice like this -when it starts in an interactive mode: - - Gnomovision version 69, Copyright (C) 19yy name of author - Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. - This is free software, and you are welcome to redistribute it - under certain conditions; type `show c' for details. - -The hypothetical commands `show w' and `show c' should show the appropriate -parts of the General Public License. Of course, the commands you use may -be called something other than `show w' and `show c'; they could even be -mouse-clicks or menu items--whatever suits your program. - -You should also get your employer (if you work as a programmer) or your -school, if any, to sign a "copyright disclaimer" for the program, if -necessary. Here is a sample; alter the names: - - Yoyodyne, Inc., hereby disclaims all copyright interest in the program - `Gnomovision' (which makes passes at compilers) written by James Hacker. - - , 1 April 1989 - Ty Coon, President of Vice - -This General Public License does not permit incorporating your program into -proprietary programs. If your program is a subroutine library, you may -consider it more useful to permit linking proprietary applications with the -library. If this is what you want to do, use the GNU Library General -Public License instead of this License. - Modified: branches/2.27/Makefile =================================================================== --- branches/2.27/Makefile 2009-01-15 03:30:46 UTC (rev 315) +++ branches/2.27/Makefile 2009-01-15 03:34:22 UTC (rev 316) @@ -171,9 +171,9 @@ exportnative: -$(RM) -r $(EXPORTTMP) cp -r src $(EXPORTTMP) - make realexportnative + $(MAKE) realexportnative ifeq ($(OSARCH),linux) - make realexportnative EXPORTSTATIC=true KIND=-static + $(MAKE) realexportnative EXPORTSTATIC=true KIND=-static endif $(RM) -r $(EXPORTTMP) Modified: branches/2.27/doc/unison-manual.tex =================================================================== --- branches/2.27/doc/unison-manual.tex 2009-01-15 03:30:46 UTC (rev 315) +++ branches/2.27/doc/unison-manual.tex 2009-01-15 03:34:22 UTC (rev 316) @@ -2065,11 +2065,11 @@ To prevent accidents, Unison provides a preference called \verb|mountpoint|. Including a line like \begin{verbatim} - mountpoint = /mnt/foo + mountpoint = foo \end{verbatim} in your preference file will cause Unison to check, after it finishes detecting updates, that something actually exists at the path -\verb|/mnt/foo| on both replicas; if it does not, the Unison run will +\verb|foo| on both replicas; if it does not, the Unison run will abort. \SUBSECTION{Click-starting Unison}{click} Modified: branches/2.27/src/Makefile =================================================================== --- branches/2.27/src/Makefile 2009-01-15 03:30:46 UTC (rev 315) +++ branches/2.27/src/Makefile 2009-01-15 03:34:22 UTC (rev 316) @@ -74,10 +74,10 @@ install: doinstall installtext: - make -C .. installtext + $(MAKE) -C .. installtext text: - make -C .. text + $(MAKE) -C .. text doinstall: $(NAME)$(EXEC_EXT) -mv $(INSTALLDIR)/$(NAME)$(EXEC_EXT) /tmp/$(NAME)-$(shell echo $$$$) Modified: branches/2.27/src/Makefile.OCaml =================================================================== --- branches/2.27/src/Makefile.OCaml 2009-01-15 03:30:46 UTC (rev 315) +++ branches/2.27/src/Makefile.OCaml 2009-01-15 03:34:22 UTC (rev 316) @@ -29,9 +29,13 @@ else ifeq ($(shell uname),OpenBSD) OSARCH=OpenBSD +else +ifeq ($(shell uname),NetBSD) + OSARCH=NetBSD endif endif endif +endif ETAGS=etags endif endif @@ -87,8 +91,15 @@ EXEC_EXT=.exe OBJ_EXT=.obj CWD=. - CLIBS+=-cclib win32rc/unison.res - STATICLIBS+=-cclib win32rc/unison.res +# Fix suggested by Karl M, Jan 2009: +# "The new flexlink wrapper that OCaml 3.11 uses was gagging on the res +# file. So the res file has to be passed through flexlink untouched to +# the linker. I only touched the MSVC side, but mingw may have the same +# issue." +# CLIBS+=-cclib win32rc/unison.res +# STATICLIBS+=-cclib win32rc/unison.res + CLIBS+=-cclib "-link win32rc/unison.res" + STATICLIBS+=-cclib "-link win32rc/unison.res" buildexecutable:: @echo Building for Windows else @@ -261,6 +272,10 @@ endif endif +ifeq ($(OSARCH), NetBSD) + OCAMLDOT=false +endif + # Rebuild dependencies (must be invoked manually) .PHONY: depend depend:: Modified: branches/2.27/src/RECENTNEWS =================================================================== --- branches/2.27/src/RECENTNEWS 2009-01-15 03:30:46 UTC (rev 315) +++ branches/2.27/src/RECENTNEWS 2009-01-15 03:34:22 UTC (rev 316) @@ -1,5 +1,34 @@ +CHANGES FROM VERSION 2.27.101 + +* Applied a patch from Karl M to make the GTK2 version build with + OCaml 3.11 on Windows. + +* Added some extra debugging code to remote.ml to give more + informative error messages when people encounter the longstanding + "assert failed during file transfer" bug. + +* Applied patch from Antoine Reilles for NetBSD compilation + +* Makefile tidying + + +------------------------------- CHANGES FROM VERSION 2.27.76 +* Applied a patch from Karl M to make the GTK2 version build with + OCaml 3.11 on Windows. + +* Added some extra debugging code to remote.ml to give more + informative error messages when people encounter the longstanding + "assert failed during file transfer" bug. + +* Applied patch from Antoine Reilles for NetBSD compilation + +* Makefile tidying + + +CHANGES FROM VERSION 2.27.76 + Resizing the update window vertically no longer moves the status label. Fix contributed by Pedro Melo. ------------------------------- CHANGES FROM VERSION 2.27.75 Modified: branches/2.27/src/lwt/Makefile =================================================================== --- branches/2.27/src/lwt/Makefile 2009-01-15 03:30:46 UTC (rev 315) +++ branches/2.27/src/lwt/Makefile 2009-01-15 03:34:22 UTC (rev 316) @@ -51,4 +51,4 @@ rm -f *.cmi *.cmo *.cmx *.cma *.cmxa *.a *.o *~ *.bak clean:: - cd example && make clean + cd example && $(MAKE) clean Modified: branches/2.27/src/mkProjectInfo.ml =================================================================== --- branches/2.27/src/mkProjectInfo.ml 2009-01-15 03:30:46 UTC (rev 315) +++ branches/2.27/src/mkProjectInfo.ml 2009-01-15 03:34:22 UTC (rev 316) @@ -77,3 +77,4 @@ + Modified: branches/2.27/src/pty.c =================================================================== --- branches/2.27/src/pty.c 2009-01-15 03:30:46 UTC (rev 315) +++ branches/2.27/src/pty.c 2009-01-15 03:34:22 UTC (rev 316) @@ -15,7 +15,7 @@ #define HAS_OPENPTY 1 #endif -#ifdef __APPLE__ +#if defined(__APPLE__) || defined(__NetBSD__) #include #define HAS_OPENPTY 1 #endif Modified: branches/2.27/src/remote.ml =================================================================== --- branches/2.27/src/remote.ml 2009-01-15 03:30:46 UTC (rev 315) +++ branches/2.27/src/remote.ml 2009-01-15 03:34:22 UTC (rev 316) @@ -343,11 +343,22 @@ else String.sub s p len +(* An integer just a little smaller than the maximum representable in 30 bits *) +let hugeint = 1000000000 + let safeMarshal marshalPayload tag data rem = let (rem', length) = marshalPayload data rem in + if length > hugeint then begin + let start = first_chars (min length 10) rem' in + let start = if length > 10 then start ^ "..." else start in + let start = String.escaped start in + Util.msg "Fatal error in safeMarshal: sending too many (%d) bytes with tag %s and contents [%s]\n" length tag start; + raise (Util.Fatal ((Printf.sprintf + "Message payload too large (%d, %s, [%s]). \n" length tag start) + ^ "This is a bug in Unison; if it happens to you in a repeatable way, \n" + ^ "please post a report on the unison-users mailing list.")) + end; let l = String.length tag in - assert (length > 0); (* tracking down an assert failure in receivePacket... *) - assert (l > 0); debugE (fun() -> let start = first_chars (min length 10) rem' in let start = if length > 10 then start ^ "..." else start in @@ -585,7 +596,7 @@ type msgId = int module MsgIdMap = Map.Make (struct type t = msgId let compare = compare end) let ids = ref 1 -let newMsgId () = incr ids; if !ids = 1000000000 then ids := 2; !ids +let newMsgId () = incr ids; if !ids = hugeint then ids := 2; !ids (* Threads waiting for a response from the other side *) let receivers = ref MsgIdMap.empty From cameronthorne+unison-users at gmail.com Fri Jan 16 23:04:19 2009 From: cameronthorne+unison-users at gmail.com (Cameron Thorne) Date: Fri, 16 Jan 2009 22:04:19 -0600 Subject: [Unison-hackers] Hello! Message-ID: <8677555a0901162004g20ac02f9maba149d0920300ab@mail.gmail.com> As I mentioned on unison-users, I'd really like to see a patch that lets Unison read files on the fly as it scans a replica, and adds ignore Paths on the fly based on what it finds. Suggested by Martin Pohlack over on unison-users, I think implementing functionality that works just like GNU tar's --exclude-tag-all option seems to make sense. I have never worked with OCaml before, nor am I familiar with the Unison source code (yet), but I'd like to learn more about it to the point where I could submit a patch for this feature that is up to your coding standards. I'll be hacking at that for a while I'm sure. I'll start with the guidelines for "Where to start" you posted on the website, and in the READMEs, but I'm sure I'll be back for questions along the way. Just wanted to introduce myself for now. -- Cameron From bcpierce at cis.upenn.edu Fri Jan 16 23:14:16 2009 From: bcpierce at cis.upenn.edu (Benjamin Pierce) Date: Fri, 16 Jan 2009 23:14:16 -0500 Subject: [Unison-hackers] Hello! In-Reply-To: <8677555a0901162004g20ac02f9maba149d0920300ab@mail.gmail.com> References: <8677555a0901162004g20ac02f9maba149d0920300ab@mail.gmail.com> Message-ID: I'd be glad to see something done along these lines. Main problem I can see is designing something reasonably general that is not too (a) expensive or (b) difficult for people to understand how to use... - Benjamin On Jan 16, 2009, at 11:04 PM, Cameron Thorne wrote: > As I mentioned on unison-users, I'd really like to see a patch that > lets Unison read files on the fly as it scans a replica, and adds > ignore Paths on the fly based on what it finds. Suggested by Martin > Pohlack over on unison-users, I think implementing functionality that > works just like GNU tar's --exclude-tag-all option seems to make > sense. > > I have never worked with OCaml before, nor am I familiar with the > Unison source code (yet), but I'd like to learn more about it to the > point where I could submit a patch for this feature that is up to your > coding standards. I'll be hacking at that for a while I'm sure. > > I'll start with the guidelines for "Where to start" you posted on the > website, and in the READMEs, but I'm sure I'll be back for questions > along the way. > > Just wanted to introduce myself for now. > > -- Cameron > _______________________________________________ > Unison-hackers mailing list > Unison-hackers at lists.seas.upenn.edu > http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers From stefan.goldmann at googlemail.com Sat Jan 17 08:41:17 2009 From: stefan.goldmann at googlemail.com (Stefan Goldmann) Date: Sat, 17 Jan 2009 14:41:17 +0100 Subject: [Unison-hackers] Asymmetric DSL and large files Message-ID: <4971DFFD.5070201@googlemail.com> Hello everyone, I've asked this question at the users list already, but we couldn't find an answer, so here it goes again. I have a large file, about 200MB, which I want to sync between home and work computer. My internet at home is ADSL, 50kbyte/s upload and 350kbyte/s download, my internet at work is 100MBit/s in both directions. When I trigger syncing these files from my home computer, and the file has been changed on the work computer, then the following happens: Unison states "Propagating changes" and thats it. Nothing more happens for a couple of minutes, and after that I aborted the process. I didn't even get the percentage display for the progress. A look into the network monitor revealed, that for the "Propagating changes" stage, Unison used the complete upload bandwidth of 50kbyte/s, whereas the larger download direction wasn't utilized at all. Is this how it is supposed to be? Am I getting something wrong? In my eyes it should be no problem to determine the parts that have been changed the other way round. I hope my problem's explanation is somehow clear enough. Have a nice weekend, Stefan From bcpierce at cis.upenn.edu Sat Jan 17 08:52:59 2009 From: bcpierce at cis.upenn.edu (Benjamin Pierce) Date: Sat, 17 Jan 2009 08:52:59 -0500 Subject: [Unison-hackers] Asymmetric DSL and large files In-Reply-To: <4971DFFD.5070201@googlemail.com> References: <4971DFFD.5070201@googlemail.com> Message-ID: Hi Stefan, I'm afraid I didn't really understand your question the first time, and I don't really understand it now. Unison has to propagate the changes in your file *from* the machine that has the new version *to* the machine that doesn't, no? (If all you want is to make the two versions the same, then you can tell Unison manually to overwrite the new version with the old one, and then you'll be utilizing the network in the fast direction. But usually that's not what's wanted.) Best, - Benjamin On Jan 17, 2009, at 8:41 AM, Stefan Goldmann wrote: > Hello everyone, > > I've asked this question at the users list already, but we couldn't > find > an answer, so here it goes again. > > I have a large file, about 200MB, which I want to sync between home > and > work computer. My internet at home is ADSL, 50kbyte/s upload and > 350kbyte/s download, my internet at work is 100MBit/s in both > directions. > > When I trigger syncing these files from my home computer, and the > file > has been changed on the work computer, then the following happens: > Unison states "Propagating changes" and thats it. Nothing more happens > for a couple of minutes, and after that I aborted the process. I > didn't > even get the percentage display for the progress. > > A look into the network monitor revealed, that for the "Propagating > changes" stage, Unison used the complete upload bandwidth of 50kbyte/ > s, > whereas the larger download direction wasn't utilized at all. > > Is this how it is supposed to be? Am I getting something wrong? In my > eyes it should be no problem to determine the parts that have been > changed the other way round. > > I hope my problem's explanation is somehow clear enough. Have a nice > weekend, > Stefan > _______________________________________________ > Unison-hackers mailing list > Unison-hackers at lists.seas.upenn.edu > http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers From bcpierce at cis.upenn.edu Sat Jan 17 11:42:56 2009 From: bcpierce at cis.upenn.edu (Benjamin Pierce) Date: Sat, 17 Jan 2009 11:42:56 -0500 Subject: [Unison-hackers] Help building OSX GUI Message-ID: <3A36B4E3-2A27-435E-9E11-B852AC372A58@cis.upenn.edu> All, Sometime in the past couple of months, I stopped being able to build the OSX GUI version of Unison on one of my machines. It's running OSX 10.5.6 and XCode 3.1.2. I've tried building both Unison 2.32 and 2.27, with the same results. What happens is that the build gets to this line /bin/sh -c "\"/Users/bcpierce/current/unison/branches/2.27/src/ uimacnew/build/uimacnew.build/Default/Create ExternalSettings.build/ Script-2A124EA10DE1C67100524237.sh\"" and then never finishes. Investigating a little further, it seems that the call to /usr/libexec/path_helper -s from this script never returns. From a little googling, I see that path_helper is a relatively new thing in OSX, but my bash hacking skills are not quite up to seeing what is wrong with it on my system. Anybody seeing similar behavior? Thanks, - Benjamin From alan.schmitt at polytechnique.org Sun Jan 18 04:19:46 2009 From: alan.schmitt at polytechnique.org (Alan Schmitt) Date: Sun, 18 Jan 2009 10:19:46 +0100 Subject: [Unison-hackers] Help building OSX GUI In-Reply-To: <3A36B4E3-2A27-435E-9E11-B852AC372A58@cis.upenn.edu> References: <3A36B4E3-2A27-435E-9E11-B852AC372A58@cis.upenn.edu> Message-ID: <4DBFC2EC-8355-478D-A199-7D7DB3220A37@polytechnique.org> On 17 janv. 09, at 17:42, Benjamin Pierce wrote: > Investigating a little further, it seems > that the call to > > /usr/libexec/path_helper -s > > from this script never returns. From a little googling, I see that > path_helper is a relatively new thing in OSX, but my bash hacking > skills are not quite up to seeing what is wrong with it on my system. > > Anybody seeing similar behavior? It works well here. Does path_helper also get stuck when you manually launch it? Alan -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 194 bytes Desc: This is a digitally signed message part Url : http://lists.seas.upenn.edu/pipermail/unison-hackers/attachments/20090118/50de3043/PGP.sig From stefan.goldmann at googlemail.com Sun Jan 18 09:22:14 2009 From: stefan.goldmann at googlemail.com (Stefan Goldmann) Date: Sun, 18 Jan 2009 15:22:14 +0100 Subject: [Unison-hackers] Asymmetric DSL and large files In-Reply-To: References: <4971DFFD.5070201@googlemail.com> Message-ID: <49733B16.8020504@googlemail.com> Hi Benjamin. Unison has to propagate the changes in your file *from* the machine that has the new version *to* the machine that doesn't, no? Yes, exactly. And the *from* machine is the distant work computer, and the *to* machine is the local home computer. Unison detects, that the work computer's version has changed, so the changes should be propagated from work to home, using the 350kb/s bandwidth. Instead, somehow Unison starts "eating up" all of the 50kb/s bandwidth from home to work. Why?? That's the wrong direction. I hope this clarifies things. :-) How does Unison determine what parts of a file have changed? I guess that this is the problem's root. Best wishes, Stefan Benjamin Pierce wrote: > Hi Stefan, > > I'm afraid I didn't really understand your question the first time, > and I don't really understand it now. Unison has to propagate the > changes in your file *from* the machine that has the new version *to* > the machine that doesn't, no? (If all you want is to make the two > versions the same, then you can tell Unison manually to overwrite the > new version with the old one, and then you'll be utilizing the network > in the fast direction. But usually that's not what's wanted.) > > Best, > > - Benjamin > > > > On Jan 17, 2009, at 8:41 AM, Stefan Goldmann wrote: > > >> Hello everyone, >> >> I've asked this question at the users list already, but we couldn't >> find >> an answer, so here it goes again. >> >> I have a large file, about 200MB, which I want to sync between home >> and >> work computer. My internet at home is ADSL, 50kbyte/s upload and >> 350kbyte/s download, my internet at work is 100MBit/s in both >> directions. >> >> When I trigger syncing these files from my home computer, and the >> file >> has been changed on the work computer, then the following happens: >> Unison states "Propagating changes" and thats it. Nothing more happens >> for a couple of minutes, and after that I aborted the process. I >> didn't >> even get the percentage display for the progress. >> >> A look into the network monitor revealed, that for the "Propagating >> changes" stage, Unison used the complete upload bandwidth of 50kbyte/ >> s, >> whereas the larger download direction wasn't utilized at all. >> >> Is this how it is supposed to be? Am I getting something wrong? In my >> eyes it should be no problem to determine the parts that have been >> changed the other way round. >> >> I hope my problem's explanation is somehow clear enough. Have a nice >> weekend, >> Stefan >> _______________________________________________ >> Unison-hackers mailing list >> Unison-hackers at lists.seas.upenn.edu >> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >> > > _______________________________________________ > Unison-hackers mailing list > Unison-hackers at lists.seas.upenn.edu > http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers > > From bcpierce at cis.upenn.edu Sun Jan 18 09:37:55 2009 From: bcpierce at cis.upenn.edu (Benjamin Pierce) Date: Sun, 18 Jan 2009 09:37:55 -0500 Subject: [Unison-hackers] Help building OSX GUI In-Reply-To: <4DBFC2EC-8355-478D-A199-7D7DB3220A37@polytechnique.org> References: <3A36B4E3-2A27-435E-9E11-B852AC372A58@cis.upenn.edu> <4DBFC2EC-8355-478D-A199-7D7DB3220A37@polytechnique.org> Message-ID: <3BB79FBC-BB3C-429A-B9ED-433A9C92BDDE@cis.upenn.edu> >> Investigating a little further, it seems >> that the call to >> >> /usr/libexec/path_helper -s >> >> from this script never returns. From a little googling, I see that >> path_helper is a relatively new thing in OSX, but my bash hacking >> skills are not quite up to seeing what is wrong with it on my system. >> >> Anybody seeing similar behavior? > > It works well here. Does path_helper also get stuck when you > manually launch it? Yes, it does. - B From bcpierce at cis.upenn.edu Sun Jan 18 09:43:16 2009 From: bcpierce at cis.upenn.edu (Benjamin Pierce) Date: Sun, 18 Jan 2009 09:43:16 -0500 Subject: [Unison-hackers] Asymmetric DSL and large files In-Reply-To: <49733B16.8020504@googlemail.com> References: <4971DFFD.5070201@googlemail.com> <49733B16.8020504@googlemail.com> Message-ID: <9B290F32-7E4A-492B-AC44-F585C9F1F67D@cis.upenn.edu> It uses the same algorithm as rsync: the receiving machine makes a summary of the blocks in the file on its end and gives it to the sending maching (this must be what you're seeing, I guess), which then sends it the changed blocks. There's a nice paper by Andrew Tridgell if you're interested in details... - B On Jan 18, 2009, at 9:22 AM, Stefan Goldmann wrote: > Hi Benjamin. > > > > Unison has to propagate the > changes in your file *from* the machine that has the new version *to* > the machine that doesn't, no? > > > > Yes, exactly. And the *from* machine is the distant work computer, and > the *to* > machine is the local home computer. Unison detects, that the work > computer's > version has changed, so the changes should be propagated from > work to home, using the 350kb/s bandwidth. Instead, somehow Unison > starts > "eating up" all of the 50kb/s bandwidth from home to work. Why?? > That's the > wrong direction. > > I hope this clarifies things. :-) > > How does Unison determine what parts of a file have changed? > I guess that this is the problem's root. > > Best wishes, > Stefan > > > Benjamin Pierce wrote: >> Hi Stefan, >> >> I'm afraid I didn't really understand your question the first time, >> and I don't really understand it now. Unison has to propagate the >> changes in your file *from* the machine that has the new version *to* >> the machine that doesn't, no? (If all you want is to make the two >> versions the same, then you can tell Unison manually to overwrite the >> new version with the old one, and then you'll be utilizing the >> network >> in the fast direction. But usually that's not what's wanted.) >> >> Best, >> >> - Benjamin >> >> >> >> On Jan 17, 2009, at 8:41 AM, Stefan Goldmann wrote: >> >> >>> Hello everyone, >>> >>> I've asked this question at the users list already, but we couldn't >>> find >>> an answer, so here it goes again. >>> >>> I have a large file, about 200MB, which I want to sync between home >>> and >>> work computer. My internet at home is ADSL, 50kbyte/s upload and >>> 350kbyte/s download, my internet at work is 100MBit/s in both >>> directions. >>> >>> When I trigger syncing these files from my home computer, and the >>> file >>> has been changed on the work computer, then the following happens: >>> Unison states "Propagating changes" and thats it. Nothing more >>> happens >>> for a couple of minutes, and after that I aborted the process. I >>> didn't >>> even get the percentage display for the progress. >>> >>> A look into the network monitor revealed, that for the "Propagating >>> changes" stage, Unison used the complete upload bandwidth of >>> 50kbyte/ >>> s, >>> whereas the larger download direction wasn't utilized at all. >>> >>> Is this how it is supposed to be? Am I getting something wrong? In >>> my >>> eyes it should be no problem to determine the parts that have been >>> changed the other way round. >>> >>> I hope my problem's explanation is somehow clear enough. Have a nice >>> weekend, >>> Stefan >>> _______________________________________________ >>> Unison-hackers mailing list >>> Unison-hackers at lists.seas.upenn.edu >>> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >>> >> >> _______________________________________________ >> Unison-hackers mailing list >> Unison-hackers at lists.seas.upenn.edu >> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >> >> > > _______________________________________________ > Unison-hackers mailing list > Unison-hackers at lists.seas.upenn.edu > http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers From alan.schmitt at polytechnique.org Sun Jan 18 09:58:59 2009 From: alan.schmitt at polytechnique.org (Alan Schmitt) Date: Sun, 18 Jan 2009 15:58:59 +0100 Subject: [Unison-hackers] Help building OSX GUI In-Reply-To: <3BB79FBC-BB3C-429A-B9ED-433A9C92BDDE@cis.upenn.edu> References: <3A36B4E3-2A27-435E-9E11-B852AC372A58@cis.upenn.edu> <4DBFC2EC-8355-478D-A199-7D7DB3220A37@polytechnique.org> <3BB79FBC-BB3C-429A-B9ED-433A9C92BDDE@cis.upenn.edu> Message-ID: <87A2B849-D292-461F-9880-DD7670552370@polytechnique.org> On 18 janv. 09, at 15:37, Benjamin Pierce wrote: >> It works well here. Does path_helper also get stuck when you >> manually launch it? > > Yes, it does. I googled a bit, and I could not find a definitive answer. You could try: - launching a new shell (for instance zsh) and see whether from this new shell path_helper -s works; - check that the /etc/profile was properly upgraded when you moved to Leopard. This may also hint at a solution: http://iparrizar.mnstate.edu/~juan/urania/2008/03/25/leopards-path_helper-seems-a-bit-buggy/ Hope this helps, Alan -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 194 bytes Desc: This is a digitally signed message part Url : http://lists.seas.upenn.edu/pipermail/unison-hackers/attachments/20090118/3626f27c/PGP.sig From stefan.goldmann at googlemail.com Sun Jan 18 10:03:25 2009 From: stefan.goldmann at googlemail.com (Stefan Goldmann) Date: Sun, 18 Jan 2009 16:03:25 +0100 Subject: [Unison-hackers] Asymmetric DSL and large files In-Reply-To: <9B290F32-7E4A-492B-AC44-F585C9F1F67D@cis.upenn.edu> References: <4971DFFD.5070201@googlemail.com> <49733B16.8020504@googlemail.com> <9B290F32-7E4A-492B-AC44-F585C9F1F67D@cis.upenn.edu> Message-ID: <497344BD.8020205@googlemail.com> Benjamin Pierce wrote: > It uses the same algorithm as rsync: the receiving machine makes a > summary of the blocks in the file on its end and gives it to the > sending maching (this must be what you're seeing, I guess), which then > sends it the changed blocks. There's a nice paper by Andrew Tridgell > if you're interested in details... > > - B And here the problem is revealed. In my case, this is the bottle neck! Is there any specific reason why the block summary list has to go from receiving machine to sending machine? If it would be done vice versa, the receiving machine would merely have to transmit a rather compact list of blocks it requests to the sender. I could imagine, there are quite a couple of ADSL users out there, who might be happy about an option to change the standard behaviour. ;-) Cheers, Stefan From bcpierce at cis.upenn.edu Sun Jan 18 10:16:12 2009 From: bcpierce at cis.upenn.edu (Benjamin Pierce) Date: Sun, 18 Jan 2009 10:16:12 -0500 Subject: [Unison-hackers] Asymmetric DSL and large files In-Reply-To: <497344BD.8020205@googlemail.com> References: <4971DFFD.5070201@googlemail.com> <49733B16.8020504@googlemail.com> <9B290F32-7E4A-492B-AC44-F585C9F1F67D@cis.upenn.edu> <497344BD.8020205@googlemail.com> Message-ID: <86B53730-972C-4D13-AE6E-88D718D469B5@cis.upenn.edu> Yes, I suppose in principle the other direction would work fine. But implementing it in Unison would be a nontrivial amount of coding, I'm afraid. Best, - Benjamin On Jan 18, 2009, at 10:03 AM, Stefan Goldmann wrote: > Benjamin Pierce wrote: >> It uses the same algorithm as rsync: the receiving machine makes a >> summary of the blocks in the file on its end and gives it to the >> sending maching (this must be what you're seeing, I guess), which >> then >> sends it the changed blocks. There's a nice paper by Andrew Tridgell >> if you're interested in details... >> >> - B > And here the problem is revealed. In my case, this is the bottle neck! > > Is there any specific reason why the block summary list has to go from > receiving machine to sending machine? If it would be done vice versa, > the receiving machine would merely have to transmit a rather compact > list > of blocks it requests to the sender. > > I could imagine, there are quite a couple of ADSL users out there, > who might > be happy about an option to change the standard behaviour. ;-) > > Cheers, Stefan > > _______________________________________________ > Unison-hackers mailing list > Unison-hackers at lists.seas.upenn.edu > http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers From stefan.goldmann at googlemail.com Sun Jan 18 15:20:21 2009 From: stefan.goldmann at googlemail.com (Stefan Goldmann) Date: Sun, 18 Jan 2009 21:20:21 +0100 Subject: [Unison-hackers] Asymmetric DSL and large files In-Reply-To: <86B53730-972C-4D13-AE6E-88D718D469B5@cis.upenn.edu> References: <4971DFFD.5070201@googlemail.com> <49733B16.8020504@googlemail.com> <9B290F32-7E4A-492B-AC44-F585C9F1F67D@cis.upenn.edu> <497344BD.8020205@googlemail.com> <86B53730-972C-4D13-AE6E-88D718D469B5@cis.upenn.edu> Message-ID: <49738F05.8030800@googlemail.com> Benjamin Pierce wrote: > Yes, I suppose in principle the other direction would work fine. But > implementing it in Unison would be a nontrivial amount of coding, I'm > afraid. > > Best, > > - Benjamin I understand. It doesn't sound to me as if this would have any priority > 0. ;-) Well, maybe I can spare some time some day and learn OCaml and can implement this ... hmmm, doesn't sound realistic even to myself. ;-) Best wishes, Stefan From MLueck at sjnma.org Sun Jan 18 15:47:13 2009 From: MLueck at sjnma.org (MLueck) Date: Sun, 18 Jan 2009 14:47:13 -0600 Subject: [Unison-hackers] Asymmetric DSL and large files In-Reply-To: <49738F05.8030800@googlemail.com> References: <4971DFFD.5070201@googlemail.com> <49733B16.8020504@googlemail.com> <9B290F32-7E4A-492B-AC44-F585C9F1F67D@cis.upenn.edu> <497344BD.8020205@googlemail.com><86B53730-972C-4D13-AE6E-88D718D469B5@cis.upenn.edu> <49738F05.8030800@googlemail.com> Message-ID: <0C6F7B008709D142A1152C5E3A26DCA0EF48D3@sjnmamail04.SJNMA.ORG> Hey do you know any good proxy that will get me onto MySpace and stuff like that -----Original Message----- From: unison-hackers-bounces at lists.seas.upenn.edu [mailto:unison-hackers-bounces at lists.seas.upenn.edu] On Behalf Of Stefan Goldmann Sent: Sunday, January 18, 2009 2:20 PM To: Unison hackers Subject: Re: [Unison-hackers] Asymmetric DSL and large files Benjamin Pierce wrote: > Yes, I suppose in principle the other direction would work fine. But > implementing it in Unison would be a nontrivial amount of coding, I'm > afraid. > > Best, > > - Benjamin I understand. It doesn't sound to me as if this would have any priority > 0. ;-) Well, maybe I can spare some time some day and learn OCaml and can implement this ... hmmm, doesn't sound realistic even to myself. ;-) Best wishes, Stefan _______________________________________________ Unison-hackers mailing list Unison-hackers at lists.seas.upenn.edu http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers