From hadmut at danisch.de Sat Sep 12 06:53:27 2015 From: hadmut at danisch.de (Hadmut Danisch) Date: Sat, 12 Sep 2015 12:53:27 +0200 Subject: [Unison-hackers] move instead of copy Message-ID: <55F40427.8040407@danisch.de> Hi, I'm aware that unison is not fully supported anymore, but I'd like to ask to add a little feature. I tried to patch it myself, but although familiar with >20 programming languages, I'm not familiar at all with erlang and did not even find in the source codes where those things were happening I am looking for. I'm using unison for years and was always happy with it, but since I am using new filesystems with snapshots, such as btrfs and zfs, unison causes some trouble. If there is a file, synchronized on two machines, and it is moved to a different location (or simply renamed) on one machine, then unison does copy that file on the new machine and then delete the old one. Although unison recognizes, that it does not need to transmit the file over the network, because it already has such a file on the target machine, it creates a new file and deletes the old one. This is a problem if there is an older snapshot in the filesystem, because if you move a file, it does not occupy new disk space, while it does when copied. Since the old file is still contained in the snapshot, the disk space is not released. While the occupied disk space remains constand on the first system, it doubles at the target system. So unison is somewhat incompatible with filesystems supporting snapshots. It should not be too difficult to implement a move instead of copy and delete, since unison already knows what it needs to know: It detects that the new file is a copy of a file already existent at the target machine and thus does not need to be transmitted. And it detects that the old file is not needed anymore and can be deleted. Would be quite important to convert this copy+delete into a rename in order to support new file systems, since especially for storage these file systems are used (e.g. because of checksums, raid, etc.). Would be much easier to implement for someone familiar with erlang and unison, than for me. regards Hadmut From mario at emmenlauer.de Sat Sep 12 07:24:24 2015 From: mario at emmenlauer.de (Mario Emmenlauer) Date: Sat, 12 Sep 2015 13:24:24 +0200 Subject: [Unison-hackers] move instead of copy In-Reply-To: <55F40427.8040407@danisch.de> References: <55F40427.8040407@danisch.de> Message-ID: <55F40B68.9090201@emmenlauer.de> Dear Hadmut, +1 from me for this request! On 12.09.2015 12:53, Hadmut Danisch wrote: > Hi, > > I'm aware that unison is not fully supported anymore, but I'd like to > ask to add a little feature. I tried to patch it myself, but although > familiar with >20 programming languages, I'm not familiar at all with > erlang and did not even find in the source codes where those things were > happening I am looking for. > > I'm using unison for years and was always happy with it, but since I am > using new filesystems with snapshots, such as btrfs and zfs, unison > causes some trouble. > > > If there is a file, synchronized on two machines, and it is moved to a > different location (or simply renamed) on one machine, then unison does > copy that file on the new machine and then delete the old one. Although > unison recognizes, that it does not need to transmit the file over the > network, because it already has such a file on the target machine, it > creates a new file and deletes the old one. > > This is a problem if there is an older snapshot in the filesystem, > because if you move a file, it does not occupy new disk space, while it > does when copied. Since the old file is still contained in the snapshot, > the disk space is not released. While the occupied disk space remains > constand on the first system, it doubles at the target system. > > So unison is somewhat incompatible with filesystems supporting snapshots. > > It should not be too difficult to implement a move instead of copy and > delete, since unison already knows what it needs to know: It detects > that the new file is a copy of a file already existent at the target > machine and thus does not need to be transmitted. And it detects that > the old file is not needed anymore and can be deleted. > > Would be quite important to convert this copy+delete into a rename in > order to support new file systems, since especially for storage these > file systems are used (e.g. because of checksums, raid, etc.). > > Would be much easier to implement for someone familiar with erlang and > unison, than for me. > > regards > Hadmut Viele Gruesse, Mario -- BioDataAnalysis GmbH, Mario Emmenlauer Mobil: +49-(0)151-68108489 Balanstrasse 43 mailto: memmenlauer * biodataanalysis.de D-81669 M?nchen http://www.biodataanalysis.de/ From bcpierce at cis.upenn.edu Mon Sep 14 11:20:08 2015 From: bcpierce at cis.upenn.edu (Benjamin C. Pierce) Date: Mon, 14 Sep 2015 11:20:08 -0400 Subject: [Unison-hackers] move instead of copy In-Reply-To: <55F40427.8040407@danisch.de> References: <55F40427.8040407@danisch.de> Message-ID: <2DACC353-E076-4DC7-B97F-276176E3FA4D@cis.upenn.edu> I agree this would be a useful feature. It may not be completely trivial to implement ? in general, the file transfer code is somewhat complex ? but if you want to take a look, I think the right place to add this functionality would be as a special case in the doAction function in the module transport.ml: before calling Copy.copy to do the file copy, you?d check whether the copy is local and the source of the copy (i.e., the file at the old location) is scheduled to be deleted (because the file was actually moved), and if so implement the copy as a move. (The last check ? whether the file is scheduled to be deleted ? is needed because Unison?s model of filesystems doesn?t actually recognize moves as such: a move will be recognized as a create and a delete, and the user might well allow the create to proceed but cancel the delete in the user interface, in which case Unison should really perform a copy. Changing the fundamental model to recognize moves as such would be a bigger job, involving UI changes for example.) - Benjamin > On Sep 12, 2015, at 6:53 AM, Hadmut Danisch wrote: > > Hi, > > I'm aware that unison is not fully supported anymore, but I'd like to > ask to add a little feature. I tried to patch it myself, but although > familiar with >20 programming languages, I'm not familiar at all with > erlang and did not even find in the source codes where those things were > happening I am looking for. > > I'm using unison for years and was always happy with it, but since I am > using new filesystems with snapshots, such as btrfs and zfs, unison > causes some trouble. > > > If there is a file, synchronized on two machines, and it is moved to a > different location (or simply renamed) on one machine, then unison does > copy that file on the new machine and then delete the old one. Although > unison recognizes, that it does not need to transmit the file over the > network, because it already has such a file on the target machine, it > creates a new file and deletes the old one. > > This is a problem if there is an older snapshot in the filesystem, > because if you move a file, it does not occupy new disk space, while it > does when copied. Since the old file is still contained in the snapshot, > the disk space is not released. While the occupied disk space remains > constand on the first system, it doubles at the target system. > > So unison is somewhat incompatible with filesystems supporting snapshots. > > It should not be too difficult to implement a move instead of copy and > delete, since unison already knows what it needs to know: It detects > that the new file is a copy of a file already existent at the target > machine and thus does not need to be transmitted. And it detects that > the old file is not needed anymore and can be deleted. > > Would be quite important to convert this copy+delete into a rename in > order to support new file systems, since especially for storage these > file systems are used (e.g. because of checksums, raid, etc.). > > Would be much easier to implement for someone familiar with erlang and > unison, than for me. > > regards > Hadmut > > _______________________________________________ > Unison-hackers mailing list > Unison-hackers at lists.seas.upenn.edu > http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers From frederik at ofb.net Mon Sep 14 12:15:04 2015 From: frederik at ofb.net (Frederik Eaton) Date: Mon, 14 Sep 2015 09:15:04 -0700 Subject: [Unison-hackers] move instead of copy In-Reply-To: <2DACC353-E076-4DC7-B97F-276176E3FA4D@cis.upenn.edu> References: <55F40427.8040407@danisch.de> <2DACC353-E076-4DC7-B97F-276176E3FA4D@cis.upenn.edu> Message-ID: <20150914161504.GB11379@ofb.net> One could also fix the problem by copying with something like 'cp --reflink', right? Perhaps that would also be useful. P.S. I thought Unison used O'Caml not Erlang... On Mon, Sep 14, 2015 at 11:20:08AM -0400, Benjamin C. Pierce wrote: > I agree this would be a useful feature. > > It may not be completely trivial to implement ? in general, the file transfer code is somewhat complex ? but if you want to take a look, I think the right place to add this functionality would be as a special case in the doAction function in the module transport.ml: before calling Copy.copy to do the file copy, you?d check whether the copy is local and the source of the copy (i.e., the file at the old location) is scheduled to be deleted (because the file was actually moved), and if so implement the copy as a move. (The last check ? whether the file is scheduled to be deleted ? is needed because Unison?s model of filesystems doesn?t actually recognize moves as such: a move will be recognized as a create and a delete, and the user might well allow the create to proceed but cancel the delete in the user interface, in which case Unison should really perform a copy. Changing the fundamental model to recognize moves as such would be a bigger job, involving UI changes for example.) > > - Benjamin > > > > On Sep 12, 2015, at 6:53 AM, Hadmut Danisch wrote: > > > > Hi, > > > > I'm aware that unison is not fully supported anymore, but I'd like to > > ask to add a little feature. I tried to patch it myself, but although > > familiar with >20 programming languages, I'm not familiar at all with > > erlang and did not even find in the source codes where those things were > > happening I am looking for. > > > > I'm using unison for years and was always happy with it, but since I am > > using new filesystems with snapshots, such as btrfs and zfs, unison > > causes some trouble. > > > > > > If there is a file, synchronized on two machines, and it is moved to a > > different location (or simply renamed) on one machine, then unison does > > copy that file on the new machine and then delete the old one. Although > > unison recognizes, that it does not need to transmit the file over the > > network, because it already has such a file on the target machine, it > > creates a new file and deletes the old one. > > > > This is a problem if there is an older snapshot in the filesystem, > > because if you move a file, it does not occupy new disk space, while it > > does when copied. Since the old file is still contained in the snapshot, > > the disk space is not released. While the occupied disk space remains > > constand on the first system, it doubles at the target system. > > > > So unison is somewhat incompatible with filesystems supporting snapshots. > > > > It should not be too difficult to implement a move instead of copy and > > delete, since unison already knows what it needs to know: It detects > > that the new file is a copy of a file already existent at the target > > machine and thus does not need to be transmitted. And it detects that > > the old file is not needed anymore and can be deleted. > > > > Would be quite important to convert this copy+delete into a rename in > > order to support new file systems, since especially for storage these > > file systems are used (e.g. because of checksums, raid, etc.). > > > > Would be much easier to implement for someone familiar with erlang and > > unison, than for me. > > > > regards > > Hadmut > > > > _______________________________________________ > > Unison-hackers mailing list > > Unison-hackers at lists.seas.upenn.edu > > http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers > > _______________________________________________ > Unison-hackers mailing list > Unison-hackers at lists.seas.upenn.edu > http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers From bcpierce at cis.upenn.edu Mon Sep 14 13:59:43 2015 From: bcpierce at cis.upenn.edu (Benjamin C. Pierce) Date: Mon, 14 Sep 2015 13:59:43 -0400 Subject: [Unison-hackers] move instead of copy In-Reply-To: <20150914161504.GB11379@ofb.net> References: <55F40427.8040407@danisch.de> <2DACC353-E076-4DC7-B97F-276176E3FA4D@cis.upenn.edu> <20150914161504.GB11379@ofb.net> Message-ID: > One could also fix the problem by copying with something like 'cp > --reflink', right? Perhaps that would also be useful. Yes, that would be a simple alternative on a filesystem that supports reflinks. > P.S. I thought Unison used O'Caml not Erlang? Yes. - B > > On Mon, Sep 14, 2015 at 11:20:08AM -0400, Benjamin C. Pierce wrote: >> I agree this would be a useful feature. >> >> It may not be completely trivial to implement ? in general, the file transfer code is somewhat complex ? but if you want to take a look, I think the right place to add this functionality would be as a special case in the doAction function in the module transport.ml: before calling Copy.copy to do the file copy, you?d check whether the copy is local and the source of the copy (i.e., the file at the old location) is scheduled to be deleted (because the file was actually moved), and if so implement the copy as a move. (The last check ? whether the file is scheduled to be deleted ? is needed because Unison?s model of filesystems doesn?t actually recognize moves as such: a move will be recognized as a create and a delete, and the user might well allow the create to proceed but cancel the delete in the user interface, in which case Unison should really perform a copy. Changing the fundamental model to recognize moves as such would be a bigger job, involving UI changes for example.) >> >> - Benjamin >> >> >>> On Sep 12, 2015, at 6:53 AM, Hadmut Danisch wrote: >>> >>> Hi, >>> >>> I'm aware that unison is not fully supported anymore, but I'd like to >>> ask to add a little feature. I tried to patch it myself, but although >>> familiar with >20 programming languages, I'm not familiar at all with >>> erlang and did not even find in the source codes where those things were >>> happening I am looking for. >>> >>> I'm using unison for years and was always happy with it, but since I am >>> using new filesystems with snapshots, such as btrfs and zfs, unison >>> causes some trouble. >>> >>> >>> If there is a file, synchronized on two machines, and it is moved to a >>> different location (or simply renamed) on one machine, then unison does >>> copy that file on the new machine and then delete the old one. Although >>> unison recognizes, that it does not need to transmit the file over the >>> network, because it already has such a file on the target machine, it >>> creates a new file and deletes the old one. >>> >>> This is a problem if there is an older snapshot in the filesystem, >>> because if you move a file, it does not occupy new disk space, while it >>> does when copied. Since the old file is still contained in the snapshot, >>> the disk space is not released. While the occupied disk space remains >>> constand on the first system, it doubles at the target system. >>> >>> So unison is somewhat incompatible with filesystems supporting snapshots. >>> >>> It should not be too difficult to implement a move instead of copy and >>> delete, since unison already knows what it needs to know: It detects >>> that the new file is a copy of a file already existent at the target >>> machine and thus does not need to be transmitted. And it detects that >>> the old file is not needed anymore and can be deleted. >>> >>> Would be quite important to convert this copy+delete into a rename in >>> order to support new file systems, since especially for storage these >>> file systems are used (e.g. because of checksums, raid, etc.). >>> >>> Would be much easier to implement for someone familiar with erlang and >>> unison, than for me. >>> >>> regards >>> Hadmut >>> >>> _______________________________________________ >>> Unison-hackers mailing list >>> Unison-hackers at lists.seas.upenn.edu >>> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >> >> _______________________________________________ >> Unison-hackers mailing list >> Unison-hackers at lists.seas.upenn.edu >> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers > _______________________________________________ > Unison-hackers mailing list > Unison-hackers at lists.seas.upenn.edu > http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers From tjim at mac.com Tue Sep 15 12:51:55 2015 From: tjim at mac.com (Trevor Jim) Date: Tue, 15 Sep 2015 12:51:55 -0400 Subject: [Unison-hackers] move instead of copy In-Reply-To: References: <55F40427.8040407@danisch.de> <2DACC353-E076-4DC7-B97F-276176E3FA4D@cis.upenn.edu> <20150914161504.GB11379@ofb.net> Message-ID: As I recall, the reason for the copy is that sometimes move is not atomic. For example, a move from one hard disk to another is actually a copy and a delete. The copy could fail in the middle. It is hard to tell how a move will be implemented from just the path of source and destination. Using copy+delete instead of a move is therefore a safer approach, since Unison knows that copies can fail and takes this into account so that you do not lose data. Something to keep in mind... -Trevor From bcpierce at cis.upenn.edu Tue Sep 15 14:01:34 2015 From: bcpierce at cis.upenn.edu (Benjamin C. Pierce) Date: Tue, 15 Sep 2015 14:01:34 -0400 Subject: [Unison-hackers] move instead of copy In-Reply-To: References: <55F40427.8040407@danisch.de> <2DACC353-E076-4DC7-B97F-276176E3FA4D@cis.upenn.edu> <20150914161504.GB11379@ofb.net> Message-ID: <313322F9-223C-4B7C-B104-0EC7B968FC84@cis.upenn.edu> Yes, we wouldn?t want to make move-by-rename be the default behavior. Though I am becoming convinced it could be a useful option for expert users (if someone were up for implementing it, which I am not volunteering to do :-)... - B > On Sep 15, 2015, at 12:51 PM, Trevor Jim wrote: > > As I recall, the reason for the copy is that sometimes move is not > atomic. > > For example, a move from one hard disk to another is actually a copy and > a delete. The copy could fail in the middle. > > It is hard to tell how a move will be implemented from just the path of > source and destination. > > Using copy+delete instead of a move is therefore a safer approach, since > Unison knows that copies can fail and takes this into account so that > you do not lose data. > > Something to keep in mind... > > -Trevor From julian at cipht.net Tue Sep 15 17:51:51 2015 From: julian at cipht.net (Julian Squires) Date: Tue, 15 Sep 2015 17:51:51 -0400 Subject: [Unison-hackers] move instead of copy In-Reply-To: References: <55F40427.8040407@danisch.de> <2DACC353-E076-4DC7-B97F-276176E3FA4D@cis.upenn.edu> <20150914161504.GB11379@ofb.net> Message-ID: I am considering implementing this. At least in the case of Posix systems, couldn't one rename(2) and if it fails with EXDEV, then fall back to copy-and-delete? On Tue, Sep 15, 2015 at 12:51 PM, Trevor Jim wrote: > As I recall, the reason for the copy is that sometimes move is not > atomic. > > For example, a move from one hard disk to another is actually a copy and > a delete. The copy could fail in the middle. > > It is hard to tell how a move will be implemented from just the path of > source and destination. > > Using copy+delete instead of a move is therefore a safer approach, since > Unison knows that copies can fail and takes this into account so that > you do not lose data. > > Something to keep in mind... > > -Trevor > _______________________________________________ > Unison-hackers mailing list > Unison-hackers at lists.seas.upenn.edu > http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers -- Julian Squires From mario at emmenlauer.de Wed Sep 16 04:02:49 2015 From: mario at emmenlauer.de (Mario Emmenlauer) Date: Wed, 16 Sep 2015 10:02:49 +0200 Subject: [Unison-hackers] move instead of copy In-Reply-To: References: <55F40427.8040407@danisch.de> <2DACC353-E076-4DC7-B97F-276176E3FA4D@cis.upenn.edu> <20150914161504.GB11379@ofb.net> Message-ID: <55F92229.2060701@emmenlauer.de> On 15.09.2015 23:51, Julian Squires wrote: > I am considering implementing this. > > At least in the case of Posix systems, couldn't one rename(2) and if > it fails with EXDEV, then fall back to copy-and-delete? It would be great if an ocaml-literate person could look into this, thanks in advance! I think this approach seems reasonable if you have a solid check for errors/failure around the rename(2). One thing to consider is that rename will leave hardlinks intact, which is a very desirable behaviour for making backups with unison! Please if possible try to achieve an implementation that keeps the hardlinks when using rename(2) - it should work out of the box, but might still be worth to keep in mind. Cheers, Mario > On Tue, Sep 15, 2015 at 12:51 PM, Trevor Jim wrote: >> As I recall, the reason for the copy is that sometimes move is not >> atomic. >> >> For example, a move from one hard disk to another is actually a copy and >> a delete. The copy could fail in the middle. >> >> It is hard to tell how a move will be implemented from just the path of >> source and destination. >> >> Using copy+delete instead of a move is therefore a safer approach, since >> Unison knows that copies can fail and takes this into account so that >> you do not lose data. >> >> Something to keep in mind... >> >> -Trevor -- BioDataAnalysis GmbH, Mario Emmenlauer Mobil: +49-(0)151-68108489 Balanstrasse 43 mailto: memmenlauer * biodataanalysis.de D-81669 M?nchen http://www.biodataanalysis.de/ From hadmut at danisch.de Wed Sep 16 17:44:21 2015 From: hadmut at danisch.de (Hadmut Danisch) Date: Wed, 16 Sep 2015 23:44:21 +0200 Subject: [Unison-hackers] move instead of copy In-Reply-To: References: <55F40427.8040407@danisch.de> <2DACC353-E076-4DC7-B97F-276176E3FA4D@cis.upenn.edu> <20150914161504.GB11379@ofb.net> Message-ID: <55F9E2B5.4090907@danisch.de> On 15.09.2015 23:51, Julian Squires wrote: > At least in the case of Posix systems, couldn't one rename(2) and if > it fails with EXDEV, then fall back to copy-and-delete? Well, you could stat (2) the source file and the target directory and compare st_dev, and try the move only if equal, i.e. both on the same file system. (needs to be verified for nested file systems in zfs or btrfs). Another option would be to make a hard link and then remove, but I would not do this since it is not atomar and could cause race conditions. regards Hadmut From frederik at ofb.net Wed Sep 16 20:30:20 2015 From: frederik at ofb.net (Frederik Eaton) Date: Wed, 16 Sep 2015 17:30:20 -0700 Subject: [Unison-hackers] move instead of copy In-Reply-To: <55F9E2B5.4090907@danisch.de> References: <55F40427.8040407@danisch.de> <2DACC353-E076-4DC7-B97F-276176E3FA4D@cis.upenn.edu> <20150914161504.GB11379@ofb.net> <55F9E2B5.4090907@danisch.de> Message-ID: <20150917003020.GK1401@ofb.net> On Wed, Sep 16, 2015 at 11:44:21PM +0200, Hadmut Danisch wrote: > > On 15.09.2015 23:51, Julian Squires wrote: > > At least in the case of Posix systems, couldn't one rename(2) and if > > it fails with EXDEV, then fall back to copy-and-delete? > > Well, you could stat (2) the source file and the target directory and > compare st_dev, and try the move only if equal, i.e. both on the same > file system. (needs to be verified for nested file systems in zfs or > btrfs). But the source and the target could have changed filesystems between stat(2) and rename(2)... not that this would ever happen, but isn't it better to just try the rename and fall back to something else if you get EXDEV? Frederick From alan.schmitt at polytechnique.org Sat Sep 19 11:09:06 2015 From: alan.schmitt at polytechnique.org (Alan Schmitt) Date: Sat, 19 Sep 2015 17:09:06 +0200 Subject: [Unison-hackers] another way to produce path.ml assertion failure assertion failure, tested versions 2.40.65 and 2.48.3 In-Reply-To: (Julian Squires's message of "Tue, 21 Jul 2015 16:22:57 -0400") References: <20150710183053.GA19883@ofb.net> <20150721023517.GA5977@ofb.net> <20150721043354.GA9669@ofb.net> <20150721185836.GB9669@ofb.net> Message-ID: Hello, On 2015-07-21 22:22, Julian Squires writes: > On Tue, Jul 21, 2015 at 2:58 PM, Frederik Eaton wrote: >> The bit about syncing between different versions of unison/ocaml is >> interesting, but it's not relevant to the bug I reported. >> >> Can you reproduce the bug by running my script? It syncs locally so >> there is only one version of unison/ocaml involved at a time. > > I've attached a script that may be easier for people to use to > reproduce the bug, as it uses the text UI and does not depend on > xvkbd. > > FWIW, for me the bug occurs as you describe, with OCaml 4.01 and unison 2.48.0. I?ve looked a little more into this, and I?m seeing very strange things. I would be very grateful if other unison hackers could give me a hand going through this code. First, the setup. This is what I?m running in a temporary directory: --8<---------------cut here---------------start------------->8--- #!/bin/bash UNI=~/src/unison/trunk/src/unison rm -rf .unison touch foo bar echo hi > foo # echo two > bar UNISON=.unison $UNI -backup 'Name *' -batch -force foo foo bar echo bye > bar UNISON=.unison $UNI -backup 'Name *' -batch foo bar --8<---------------cut here---------------end--------------->8--- I see the bug with trunk, 2.48, and 2.40 (I could not compile earlier versions with my ocaml, but I can install another one if need be). The setup is a bit strange: it directly synchronizes two files, which means that the relative path to be synchronized in empty. This breaks some assumptions for the stasher when it needs to move the current backup file (named simply "backup") to a version with a number appended: Path.addPrefixToFinalName asserts the path is not empty, which is wrong in this case. Should the assertion go away? While reproducing this, I found another very strange bug. If I run the following, where the length of the file is the same: --8<---------------cut here---------------start------------->8--- #!/bin/bash UNI=~/src/unison/trunk/src/unison rm -rf .unison touch foo bar echo hi > foo # echo two > bar UNISON=.unison $UNI -backup 'Name *' -batch -force foo foo bar echo by > bar UNISON=.unison $UNI -backup 'Name *' -batch foo bar --8<---------------cut here---------------end--------------->8--- then unison tells me that there is nothing to synchronize for the second run! This is the full output: --8<---------------cut here---------------start------------->8--- [schmitta at top:~/tmp/unison_bug] % ./bug.sh Contacting server... Looking for changes Warning: No archive files were found for these roots, whose canonical names are: /Users/schmitta/tmp/unison_bug/foo /Users/schmitta/tmp/unison_bug/bar This can happen either because this is the first time you have synchronized these roots, or because you have upgraded Unison to a new version with a different archive format. Update detection may take a while on this run if the replicas are large. Unison will assume that the 'last synchronized state' of both replicas was completely empty. This means that any files that are different will be reported as conflicts, and any files that exist only on one replica will be judged as new and propagated to the other replica. If the two replicas are identical, then no changes will be reported. If you see this message repeatedly, it may be because one of your machines is getting its address from DHCP, which is causing its host name to change between synchronizations. See the documentation for the UNISONLOCALHOSTNAME environment variable for advice on how to correct this. Donations to the Unison project are gratefully accepted: http://www.cis.upenn.edu/~bcpierce/unison Reconciling changes file ====> file / foo : file modified on 2015-09-19 at 15:43:54 size 3 rw-r--r-- bar : file modified on 2015-09-19 at 15:43:54 size 4 rw-r--r-- Propagating updates UNISON 2.49.2 started propagating changes at 15:43:55.97 on 19 Sep 2015 [BGN] Updating file from /Users/schmitta/tmp/unison_bug/foo to /Users/schmitta/tmp/unison_bug/bar [END] Updating file UNISON 2.49.2 finished propagating changes at 15:43:55.97 on 19 Sep 2015 Saving synchronizer state Synchronization complete at 15:43:55 (1 item transferred, 0 skipped, 0 failed) Contacting server... Looking for changes Reconciling changes Nothing to do: replicas have not changed since last sync. [schmitta at top:~/tmp/unison_bug] % cat foo hi [schmitta at top:~/tmp/unison_bug] % cat bar by --8<---------------cut here---------------end--------------->8--- This makes me wonder: is unison supposed to work in this scenario (directly synchronizing two files)? Best, Alan -- OpenPGP Key ID : 040D0A3B4ED2E5C7 Last week athmospheric CO? average (Updated September 13, 2015, Mauna Loa Obs.): 397.77 ppm -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 472 bytes Desc: not available URL: From c.buhtz at posteo.jp Sat Sep 26 16:27:14 2015 From: c.buhtz at posteo.jp (c.buhtz at posteo.jp) Date: Sat, 26 Sep 2015 22:27:14 +0200 Subject: [Unison-hackers] [FeatureReqest] auto-delete unneeded backup-files Message-ID: <3nNhY205ptz5vMw@dovecot03.posteo.de> This FR is related to this Thread on the user-List I am asking for such a feature because I want to have the opinion of the maintainers of that project. I am aware of the current status (not in active developend) and of the less resources of the project. I don't ask about if you (the maintainers) have the resources to implement that. I am just asking if you think it is a good idea and you would accept a patch/pull-request. Maybe you hate this idea and would never accept that feature in an official unison release. It is important do know for any developer who try to implement that. The Feature: Unison should check if the backup-files have a corrosponding original file. If the original file doesn't exists anymore something should be done (e.g. delete it). This could be depended on the age of the bak-file, on the size of the complete backup-directorie, etc. This would prevend the user for wasting storage space. When the user delete an original file the backup-file is still there. Most of the use cases it is in a hidden directory. So the user never think about that. Using unison for some years this would produce a lot of files. In my opinion and habit of developing software this FR could be a bug-report, too. The use of bak-files is not fully implemented. Unison create them automaticly but doesn't take care of there deletion. In german we say: "When you say A you have to say B, too." Minimaly there should be a big-red-bold-hint in the manual about the possible excessive use of storage space the user should take care about himself. So what is your opinion about this feature and would you accept it no matter about your own resources? -- -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v1 mQENBFQIluABCACfPwAhRAwFD3NXgv5CtVUGSiqdfJGVViVBqaKd+14E0pASA0MU G0Ewj7O7cGy/ZIoiZ0+lIEZmzJKHfuGwYhXjR/PhnUDrQIHLBvh9WuD6JQuULXfH kXtVm/i9wm76QAcvr2pwYgNzhcJntUHl2GcgnInYbZDeVmg+p9yIPJjuq73/lRS3 0/McgNoFOBhKK/S6STQuFyjr9OyJyYd1shoM3hmy+kg0HYm6OgQBJNg92WV9jwGe GzlipvEp2jpLwVsTxYir2oOPhfd9D1fC9F/l/3gXbfjd5GIIVrZFq2haZmoVeJ33 LJxo3RA5Tf9LoUeels1b4s9kFz6h7+AHERUpABEBAAG0IUNocmlzdGlhbiBCdWh0 eiA8YnVodHpAcG9zdGVvLmRlPokBPgQTAQIAKAUCVAiW4AIbAwUJAeEzgAYLCQgH AwIGFQgCCQoLBBYCAwECHgECF4AACgkQZLsXsAdRqOxNUAf/V/hDA5zGDpySuCEj DhjiVRK74J9Wd8gfH0WAf1Co5HZ24wZH8rgOIVIgXw8rWkOw/VA6xfdfT+64xjTY Fhkpbrk199nDzp72F7Jc4NC+x8xac2e3rK5ifSWhZx7L5A32pGYE+d16m3EEqImK D4gcZl38x9zdUnD4hHyXkIPz1uCfuMuGgWEnaUk4Wbj41CBZr3O0ABue6regV15U jaes8r+B8iCcY+0yP2kse+3iaCaMqNv5FgQZ9+b2Cql8pFkZJVtBVUw4GW3DWZJi du0O/YrC9TgS+xY9ht/MD2qSHwjcK1sdImjqBO7xP8TIOwKeYyDvGKnSO3EJ/sSA UPGEPrkBDQRUCJbgAQgA0k/Qg67CCUJE2/zuxBEoK4wLJpDRJzh8CQPZpjWx8VP0 KL892jwfxymXn8KNhuy1SgCBFSeV9jg4VZNWDlUGJc2lo82ajr9PzIsrQwu4lf0B zrUWV5hWepKu/kb8uSjx58YYfx0SFz4+9akX3Wwu9TUHntzL5Gk3Q26nnsr1xEJ+ VEumvCH9AE0Tk0K7dQpJ2/JcLuO+uhrpd/lHFDYVN5NsG3P015uFOkDI6N/xNFCj v95XNR93QlfKpK3qWlFGescfG+o/7Ub6s67/i/JoNbw0XgPEHmQfXpD7IHO4cu+p +ETb11cz+1mmi96cy98ID+uTiToJ8G//yD9rmtyxoQARAQABiQElBBgBAgAPBQJU CJbgAhsMBQkB4TOAAAoJEGS7F7AHUajs6sQH/iKs6sPc0vkRJLfbwrijZeecwCWF blo/jzIQ8jPykAj9SLjV20Xwqg3XcJyko8ZU6/zuRJq9xjlv9pZr/oVudQAt6v+h 2Cf4rKEjmau483wjMV2xjTXQhZi9+ttDbia4fgdmGtKsOicn5ae2fFXcXNPu3RiW sZKifWdokA6xqMW6iIG9YjjI5ShxngHWp2xfPscBFMDRtFOMags/Yx+YvwoyEZ4A dURYMFHFqpwILEc8hIzhRg1gq40AHbOaEdczS1Rr3T7/gS6eBs4u6HuY5g2Bierm lLjpspFPjMXwJAa/XLOBjMF2vsHPrZNcouNKkumQ36yq/Pm6DFXAseQDxOk= =PGP9 -----END PGP PUBLIC KEY BLOCK----- From bcpierce at cis.upenn.edu Sat Sep 26 23:32:49 2015 From: bcpierce at cis.upenn.edu (Benjamin C. Pierce) Date: Sat, 26 Sep 2015 23:32:49 -0400 Subject: [Unison-hackers] [FeatureReqest] auto-delete unneeded backup-files In-Reply-To: <3nNhY205ptz5vMw@dovecot03.posteo.de> References: <3nNhY205ptz5vMw@dovecot03.posteo.de> Message-ID: <08AAA8B4-86DD-409B-9FD0-0F2B8B742D1D@cis.upenn.edu> I?d be interested to hear opinions from others on the list. Personally, I think I probably would not want Unison ever to remove backups of deleted files (though, somewhat inconsistently, I do use the maxbackups feature). But if there are many others that would find this feature useful, I would not object strongly to its being added, as long as it is easy to control and off by default. - Benjamin > On Sep 26, 2015, at 4:27 PM, c.buhtz at posteo.jp wrote: > > This FR is related to this Thread on the user-List > > > I am asking for such a feature because I want to have the opinion of > the maintainers of that project. > I am aware of the current status (not in active developend) and of the > less resources of the project. > I don't ask about if you (the maintainers) have the resources to > implement that. I am just asking if you think it is a good idea and you > would accept a patch/pull-request. Maybe you hate this idea and would > never accept that feature in an official unison release. It is > important do know for any developer who try to implement that. > > The Feature: > Unison should check if the backup-files have a corrosponding original > file. If the original file doesn't exists anymore something should be > done (e.g. delete it). This could be depended on the age of the > bak-file, on the size of the complete backup-directorie, etc. > This would prevend the user for wasting storage space. When the user > delete an original file the backup-file is still there. Most of the use > cases it is in a hidden directory. So the user never think about that. > Using unison for some years this would produce a lot of files. > > In my opinion and habit of developing software this FR could be a > bug-report, too. > The use of bak-files is not fully implemented. Unison create them > automaticly but doesn't take care of there deletion. In german we say: > "When you say A you have to say B, too." > > Minimaly there should be a big-red-bold-hint in the manual about the > possible excessive use of storage space the user should take care about > himself. > > So what is your opinion about this feature and would you accept it no > matter about your own resources? > -- > -----BEGIN PGP PUBLIC KEY BLOCK----- > Version: GnuPG v1 > > mQENBFQIluABCACfPwAhRAwFD3NXgv5CtVUGSiqdfJGVViVBqaKd+14E0pASA0MU > G0Ewj7O7cGy/ZIoiZ0+lIEZmzJKHfuGwYhXjR/PhnUDrQIHLBvh9WuD6JQuULXfH > kXtVm/i9wm76QAcvr2pwYgNzhcJntUHl2GcgnInYbZDeVmg+p9yIPJjuq73/lRS3 > 0/McgNoFOBhKK/S6STQuFyjr9OyJyYd1shoM3hmy+kg0HYm6OgQBJNg92WV9jwGe > GzlipvEp2jpLwVsTxYir2oOPhfd9D1fC9F/l/3gXbfjd5GIIVrZFq2haZmoVeJ33 > LJxo3RA5Tf9LoUeels1b4s9kFz6h7+AHERUpABEBAAG0IUNocmlzdGlhbiBCdWh0 > eiA8YnVodHpAcG9zdGVvLmRlPokBPgQTAQIAKAUCVAiW4AIbAwUJAeEzgAYLCQgH > AwIGFQgCCQoLBBYCAwECHgECF4AACgkQZLsXsAdRqOxNUAf/V/hDA5zGDpySuCEj > DhjiVRK74J9Wd8gfH0WAf1Co5HZ24wZH8rgOIVIgXw8rWkOw/VA6xfdfT+64xjTY > Fhkpbrk199nDzp72F7Jc4NC+x8xac2e3rK5ifSWhZx7L5A32pGYE+d16m3EEqImK > D4gcZl38x9zdUnD4hHyXkIPz1uCfuMuGgWEnaUk4Wbj41CBZr3O0ABue6regV15U > jaes8r+B8iCcY+0yP2kse+3iaCaMqNv5FgQZ9+b2Cql8pFkZJVtBVUw4GW3DWZJi > du0O/YrC9TgS+xY9ht/MD2qSHwjcK1sdImjqBO7xP8TIOwKeYyDvGKnSO3EJ/sSA > UPGEPrkBDQRUCJbgAQgA0k/Qg67CCUJE2/zuxBEoK4wLJpDRJzh8CQPZpjWx8VP0 > KL892jwfxymXn8KNhuy1SgCBFSeV9jg4VZNWDlUGJc2lo82ajr9PzIsrQwu4lf0B > zrUWV5hWepKu/kb8uSjx58YYfx0SFz4+9akX3Wwu9TUHntzL5Gk3Q26nnsr1xEJ+ > VEumvCH9AE0Tk0K7dQpJ2/JcLuO+uhrpd/lHFDYVN5NsG3P015uFOkDI6N/xNFCj > v95XNR93QlfKpK3qWlFGescfG+o/7Ub6s67/i/JoNbw0XgPEHmQfXpD7IHO4cu+p > +ETb11cz+1mmi96cy98ID+uTiToJ8G//yD9rmtyxoQARAQABiQElBBgBAgAPBQJU > CJbgAhsMBQkB4TOAAAoJEGS7F7AHUajs6sQH/iKs6sPc0vkRJLfbwrijZeecwCWF > blo/jzIQ8jPykAj9SLjV20Xwqg3XcJyko8ZU6/zuRJq9xjlv9pZr/oVudQAt6v+h > 2Cf4rKEjmau483wjMV2xjTXQhZi9+ttDbia4fgdmGtKsOicn5ae2fFXcXNPu3RiW > sZKifWdokA6xqMW6iIG9YjjI5ShxngHWp2xfPscBFMDRtFOMags/Yx+YvwoyEZ4A > dURYMFHFqpwILEc8hIzhRg1gq40AHbOaEdczS1Rr3T7/gS6eBs4u6HuY5g2Bierm > lLjpspFPjMXwJAa/XLOBjMF2vsHPrZNcouNKkumQ36yq/Pm6DFXAseQDxOk= > =PGP9 > -----END PGP PUBLIC KEY BLOCK----- > _______________________________________________ > Unison-hackers mailing list > Unison-hackers at lists.seas.upenn.edu > http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers From c.buhtz at posteo.jp Sun Sep 27 04:53:42 2015 From: c.buhtz at posteo.jp (c.buhtz at posteo.jp) Date: Sun, 27 Sep 2015 10:53:42 +0200 Subject: [Unison-hackers] [FeatureReqest] auto-delete unneeded backup-files In-Reply-To: <08AAA8B4-86DD-409B-9FD0-0F2B8B742D1D@cis.upenn.edu> References: <3nNhY205ptz5vMw@dovecot03.posteo.de> <08AAA8B4-86DD-409B-9FD0-0F2B8B742D1D@cis.upenn.edu> Message-ID: <3nP1456mNqz5vMK@dovecot03.posteo.de> On 2015-09-26 23:32 "Benjamin C. Pierce" wrote: > I probably would not want Unison ever to remove > backups of deleted files So you take care of bak-files from deleted files manually for your own? > (though, somewhat inconsistently, I do use > the maxbackups feature). But (when I am right) 'maxbackups' doesn't affect bak-files of deleted files. That is the point here. > as long as it is easy to control and off by default. FACK! -- -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v1 mQENBFQIluABCACfPwAhRAwFD3NXgv5CtVUGSiqdfJGVViVBqaKd+14E0pASA0MU G0Ewj7O7cGy/ZIoiZ0+lIEZmzJKHfuGwYhXjR/PhnUDrQIHLBvh9WuD6JQuULXfH kXtVm/i9wm76QAcvr2pwYgNzhcJntUHl2GcgnInYbZDeVmg+p9yIPJjuq73/lRS3 0/McgNoFOBhKK/S6STQuFyjr9OyJyYd1shoM3hmy+kg0HYm6OgQBJNg92WV9jwGe GzlipvEp2jpLwVsTxYir2oOPhfd9D1fC9F/l/3gXbfjd5GIIVrZFq2haZmoVeJ33 LJxo3RA5Tf9LoUeels1b4s9kFz6h7+AHERUpABEBAAG0IUNocmlzdGlhbiBCdWh0 eiA8YnVodHpAcG9zdGVvLmRlPokBPgQTAQIAKAUCVAiW4AIbAwUJAeEzgAYLCQgH AwIGFQgCCQoLBBYCAwECHgECF4AACgkQZLsXsAdRqOxNUAf/V/hDA5zGDpySuCEj DhjiVRK74J9Wd8gfH0WAf1Co5HZ24wZH8rgOIVIgXw8rWkOw/VA6xfdfT+64xjTY Fhkpbrk199nDzp72F7Jc4NC+x8xac2e3rK5ifSWhZx7L5A32pGYE+d16m3EEqImK D4gcZl38x9zdUnD4hHyXkIPz1uCfuMuGgWEnaUk4Wbj41CBZr3O0ABue6regV15U jaes8r+B8iCcY+0yP2kse+3iaCaMqNv5FgQZ9+b2Cql8pFkZJVtBVUw4GW3DWZJi du0O/YrC9TgS+xY9ht/MD2qSHwjcK1sdImjqBO7xP8TIOwKeYyDvGKnSO3EJ/sSA UPGEPrkBDQRUCJbgAQgA0k/Qg67CCUJE2/zuxBEoK4wLJpDRJzh8CQPZpjWx8VP0 KL892jwfxymXn8KNhuy1SgCBFSeV9jg4VZNWDlUGJc2lo82ajr9PzIsrQwu4lf0B zrUWV5hWepKu/kb8uSjx58YYfx0SFz4+9akX3Wwu9TUHntzL5Gk3Q26nnsr1xEJ+ VEumvCH9AE0Tk0K7dQpJ2/JcLuO+uhrpd/lHFDYVN5NsG3P015uFOkDI6N/xNFCj v95XNR93QlfKpK3qWlFGescfG+o/7Ub6s67/i/JoNbw0XgPEHmQfXpD7IHO4cu+p +ETb11cz+1mmi96cy98ID+uTiToJ8G//yD9rmtyxoQARAQABiQElBBgBAgAPBQJU CJbgAhsMBQkB4TOAAAoJEGS7F7AHUajs6sQH/iKs6sPc0vkRJLfbwrijZeecwCWF blo/jzIQ8jPykAj9SLjV20Xwqg3XcJyko8ZU6/zuRJq9xjlv9pZr/oVudQAt6v+h 2Cf4rKEjmau483wjMV2xjTXQhZi9+ttDbia4fgdmGtKsOicn5ae2fFXcXNPu3RiW sZKifWdokA6xqMW6iIG9YjjI5ShxngHWp2xfPscBFMDRtFOMags/Yx+YvwoyEZ4A dURYMFHFqpwILEc8hIzhRg1gq40AHbOaEdczS1Rr3T7/gS6eBs4u6HuY5g2Bierm lLjpspFPjMXwJAa/XLOBjMF2vsHPrZNcouNKkumQ36yq/Pm6DFXAseQDxOk= =PGP9 -----END PGP PUBLIC KEY BLOCK----- From c.buhtz at posteo.jp Sun Sep 27 08:33:16 2015 From: c.buhtz at posteo.jp (c.buhtz at posteo.jp) Date: Sun, 27 Sep 2015 14:33:16 +0200 Subject: [Unison-hackers] [FeatureReqest] auto-delete unneeded backup-files In-Reply-To: <08AAA8B4-86DD-409B-9FD0-0F2B8B742D1D@cis.upenn.edu> References: <3nNhY205ptz5vMw@dovecot03.posteo.de> <08AAA8B4-86DD-409B-9FD0-0F2B8B742D1D@cis.upenn.edu> Message-ID: <3nP5xL6mdyz5vN5@dovecot03.posteo.de> On 2015-09-26 23:32 "Benjamin C. Pierce" wrote: > I?d be interested to hear opinions from others on the list. When I would be more concrete about my wishes: - write about this behaviour in the manual (red-bold-whatever) - make the here wished auto-delete feauter off by default - take care of the age of the bak-files, too + don't delete only if the original files are gone + look for age in days (e.g. default 356 days) + make the age-option mandatory Nice to have but maybe to complex/much: - take care of the summarized size of the backup-folder + but could this be handled if backloc=local? -- -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v1 mQENBFQIluABCACfPwAhRAwFD3NXgv5CtVUGSiqdfJGVViVBqaKd+14E0pASA0MU G0Ewj7O7cGy/ZIoiZ0+lIEZmzJKHfuGwYhXjR/PhnUDrQIHLBvh9WuD6JQuULXfH kXtVm/i9wm76QAcvr2pwYgNzhcJntUHl2GcgnInYbZDeVmg+p9yIPJjuq73/lRS3 0/McgNoFOBhKK/S6STQuFyjr9OyJyYd1shoM3hmy+kg0HYm6OgQBJNg92WV9jwGe GzlipvEp2jpLwVsTxYir2oOPhfd9D1fC9F/l/3gXbfjd5GIIVrZFq2haZmoVeJ33 LJxo3RA5Tf9LoUeels1b4s9kFz6h7+AHERUpABEBAAG0IUNocmlzdGlhbiBCdWh0 eiA8YnVodHpAcG9zdGVvLmRlPokBPgQTAQIAKAUCVAiW4AIbAwUJAeEzgAYLCQgH AwIGFQgCCQoLBBYCAwECHgECF4AACgkQZLsXsAdRqOxNUAf/V/hDA5zGDpySuCEj DhjiVRK74J9Wd8gfH0WAf1Co5HZ24wZH8rgOIVIgXw8rWkOw/VA6xfdfT+64xjTY Fhkpbrk199nDzp72F7Jc4NC+x8xac2e3rK5ifSWhZx7L5A32pGYE+d16m3EEqImK D4gcZl38x9zdUnD4hHyXkIPz1uCfuMuGgWEnaUk4Wbj41CBZr3O0ABue6regV15U jaes8r+B8iCcY+0yP2kse+3iaCaMqNv5FgQZ9+b2Cql8pFkZJVtBVUw4GW3DWZJi du0O/YrC9TgS+xY9ht/MD2qSHwjcK1sdImjqBO7xP8TIOwKeYyDvGKnSO3EJ/sSA UPGEPrkBDQRUCJbgAQgA0k/Qg67CCUJE2/zuxBEoK4wLJpDRJzh8CQPZpjWx8VP0 KL892jwfxymXn8KNhuy1SgCBFSeV9jg4VZNWDlUGJc2lo82ajr9PzIsrQwu4lf0B zrUWV5hWepKu/kb8uSjx58YYfx0SFz4+9akX3Wwu9TUHntzL5Gk3Q26nnsr1xEJ+ VEumvCH9AE0Tk0K7dQpJ2/JcLuO+uhrpd/lHFDYVN5NsG3P015uFOkDI6N/xNFCj v95XNR93QlfKpK3qWlFGescfG+o/7Ub6s67/i/JoNbw0XgPEHmQfXpD7IHO4cu+p +ETb11cz+1mmi96cy98ID+uTiToJ8G//yD9rmtyxoQARAQABiQElBBgBAgAPBQJU CJbgAhsMBQkB4TOAAAoJEGS7F7AHUajs6sQH/iKs6sPc0vkRJLfbwrijZeecwCWF blo/jzIQ8jPykAj9SLjV20Xwqg3XcJyko8ZU6/zuRJq9xjlv9pZr/oVudQAt6v+h 2Cf4rKEjmau483wjMV2xjTXQhZi9+ttDbia4fgdmGtKsOicn5ae2fFXcXNPu3RiW sZKifWdokA6xqMW6iIG9YjjI5ShxngHWp2xfPscBFMDRtFOMags/Yx+YvwoyEZ4A dURYMFHFqpwILEc8hIzhRg1gq40AHbOaEdczS1Rr3T7/gS6eBs4u6HuY5g2Bierm lLjpspFPjMXwJAa/XLOBjMF2vsHPrZNcouNKkumQ36yq/Pm6DFXAseQDxOk= =PGP9 -----END PGP PUBLIC KEY BLOCK----- From frederik at ofb.net Mon Sep 28 01:00:49 2015 From: frederik at ofb.net (Frederik Eaton) Date: Sun, 27 Sep 2015 22:00:49 -0700 Subject: [Unison-hackers] [FeatureReqest] auto-delete unneeded backup-files In-Reply-To: <08AAA8B4-86DD-409B-9FD0-0F2B8B742D1D@cis.upenn.edu> References: <3nNhY205ptz5vMw@dovecot03.posteo.de> <08AAA8B4-86DD-409B-9FD0-0F2B8B742D1D@cis.upenn.edu> Message-ID: <20150928050049.GB31302@ofb.net> Since Benjamin asked for other opinions, I'll agree that removing backups of deleted files sounds like a bad idea, but I think there's an issue regarding how to manage backups in general. My preference would be to avoid having this complexity in Unison, but to make it easy for users to delete backups made more than X days ago using a special 'find' command with an example in the manual. Experimenting with my own 'backup' directory, it looks like the 'mtime' of backups reflects when the backup was made, is this correct? In which case one could do find .unison/backup/ -mtime +20 | xargs rm to remove backup files created more than 20 days ago. However, running just 'find' I now see that there seem to be some files in this directory whose 'mtime' predates the time when I started saving backups, so maybe 'mtime' isn't the right metric, or perhaps the Unison code needs to be modified so that 'mtime' works for this purpose. Of course, I mostly use Unison backups to recover from typing the wrong thing at the Unison prompt, so I could just delete the entire backup directory as soon as I'm confident that I ran Unison correctly. Frederick On Sat, Sep 26, 2015 at 11:32:49PM -0400, Benjamin C. Pierce wrote: > I?d be interested to hear opinions from others on the list. Personally, I think I probably would not want Unison ever to remove backups of deleted files (though, somewhat inconsistently, I do use the maxbackups feature). But if there are many others that would find this feature useful, I would not object strongly to its being added, as long as it is easy to control and off by default. > > - Benjamin > > > On Sep 26, 2015, at 4:27 PM, c.buhtz at posteo.jp wrote: > > > > This FR is related to this Thread on the user-List > > > > > > I am asking for such a feature because I want to have the opinion of > > the maintainers of that project. > > I am aware of the current status (not in active developend) and of the > > less resources of the project. > > I don't ask about if you (the maintainers) have the resources to > > implement that. I am just asking if you think it is a good idea and you > > would accept a patch/pull-request. Maybe you hate this idea and would > > never accept that feature in an official unison release. It is > > important do know for any developer who try to implement that. > > > > The Feature: > > Unison should check if the backup-files have a corrosponding original > > file. If the original file doesn't exists anymore something should be > > done (e.g. delete it). This could be depended on the age of the > > bak-file, on the size of the complete backup-directorie, etc. > > This would prevend the user for wasting storage space. When the user > > delete an original file the backup-file is still there. Most of the use > > cases it is in a hidden directory. So the user never think about that. > > Using unison for some years this would produce a lot of files. > > > > In my opinion and habit of developing software this FR could be a > > bug-report, too. > > The use of bak-files is not fully implemented. Unison create them > > automaticly but doesn't take care of there deletion. In german we say: > > "When you say A you have to say B, too." > > > > Minimaly there should be a big-red-bold-hint in the manual about the > > possible excessive use of storage space the user should take care about > > himself. > > > > So what is your opinion about this feature and would you accept it no > > matter about your own resources? > > -- > > -----BEGIN PGP PUBLIC KEY BLOCK----- > > Version: GnuPG v1 > > > > mQENBFQIluABCACfPwAhRAwFD3NXgv5CtVUGSiqdfJGVViVBqaKd+14E0pASA0MU > > G0Ewj7O7cGy/ZIoiZ0+lIEZmzJKHfuGwYhXjR/PhnUDrQIHLBvh9WuD6JQuULXfH > > kXtVm/i9wm76QAcvr2pwYgNzhcJntUHl2GcgnInYbZDeVmg+p9yIPJjuq73/lRS3 > > 0/McgNoFOBhKK/S6STQuFyjr9OyJyYd1shoM3hmy+kg0HYm6OgQBJNg92WV9jwGe > > GzlipvEp2jpLwVsTxYir2oOPhfd9D1fC9F/l/3gXbfjd5GIIVrZFq2haZmoVeJ33 > > LJxo3RA5Tf9LoUeels1b4s9kFz6h7+AHERUpABEBAAG0IUNocmlzdGlhbiBCdWh0 > > eiA8YnVodHpAcG9zdGVvLmRlPokBPgQTAQIAKAUCVAiW4AIbAwUJAeEzgAYLCQgH > > AwIGFQgCCQoLBBYCAwECHgECF4AACgkQZLsXsAdRqOxNUAf/V/hDA5zGDpySuCEj > > DhjiVRK74J9Wd8gfH0WAf1Co5HZ24wZH8rgOIVIgXw8rWkOw/VA6xfdfT+64xjTY > > Fhkpbrk199nDzp72F7Jc4NC+x8xac2e3rK5ifSWhZx7L5A32pGYE+d16m3EEqImK > > D4gcZl38x9zdUnD4hHyXkIPz1uCfuMuGgWEnaUk4Wbj41CBZr3O0ABue6regV15U > > jaes8r+B8iCcY+0yP2kse+3iaCaMqNv5FgQZ9+b2Cql8pFkZJVtBVUw4GW3DWZJi > > du0O/YrC9TgS+xY9ht/MD2qSHwjcK1sdImjqBO7xP8TIOwKeYyDvGKnSO3EJ/sSA > > UPGEPrkBDQRUCJbgAQgA0k/Qg67CCUJE2/zuxBEoK4wLJpDRJzh8CQPZpjWx8VP0 > > KL892jwfxymXn8KNhuy1SgCBFSeV9jg4VZNWDlUGJc2lo82ajr9PzIsrQwu4lf0B > > zrUWV5hWepKu/kb8uSjx58YYfx0SFz4+9akX3Wwu9TUHntzL5Gk3Q26nnsr1xEJ+ > > VEumvCH9AE0Tk0K7dQpJ2/JcLuO+uhrpd/lHFDYVN5NsG3P015uFOkDI6N/xNFCj > > v95XNR93QlfKpK3qWlFGescfG+o/7Ub6s67/i/JoNbw0XgPEHmQfXpD7IHO4cu+p > > +ETb11cz+1mmi96cy98ID+uTiToJ8G//yD9rmtyxoQARAQABiQElBBgBAgAPBQJU > > CJbgAhsMBQkB4TOAAAoJEGS7F7AHUajs6sQH/iKs6sPc0vkRJLfbwrijZeecwCWF > > blo/jzIQ8jPykAj9SLjV20Xwqg3XcJyko8ZU6/zuRJq9xjlv9pZr/oVudQAt6v+h > > 2Cf4rKEjmau483wjMV2xjTXQhZi9+ttDbia4fgdmGtKsOicn5ae2fFXcXNPu3RiW > > sZKifWdokA6xqMW6iIG9YjjI5ShxngHWp2xfPscBFMDRtFOMags/Yx+YvwoyEZ4A > > dURYMFHFqpwILEc8hIzhRg1gq40AHbOaEdczS1Rr3T7/gS6eBs4u6HuY5g2Bierm > > lLjpspFPjMXwJAa/XLOBjMF2vsHPrZNcouNKkumQ36yq/Pm6DFXAseQDxOk= > > =PGP9 > > -----END PGP PUBLIC KEY BLOCK----- > > _______________________________________________ > > Unison-hackers mailing list > > Unison-hackers at lists.seas.upenn.edu > > http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers > > _______________________________________________ > Unison-hackers mailing list > Unison-hackers at lists.seas.upenn.edu > http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers From bcpierce at cis.upenn.edu Mon Sep 28 22:17:41 2015 From: bcpierce at cis.upenn.edu (Benjamin C. Pierce) Date: Mon, 28 Sep 2015 22:17:41 -0400 Subject: [Unison-hackers] [FeatureReqest] auto-delete unneeded backup-files In-Reply-To: <20150928050049.GB31302@ofb.net> References: <3nNhY205ptz5vMw@dovecot03.posteo.de> <08AAA8B4-86DD-409B-9FD0-0F2B8B742D1D@cis.upenn.edu> <20150928050049.GB31302@ofb.net> Message-ID: <3DA64453-B9EB-4353-8706-6AA86FB812ED@cis.upenn.edu> > Experimenting > with my own 'backup' directory, it looks like the 'mtime' of backups > reflects when the backup was made, is this correct? In which case one > could do > > find .unison/backup/ -mtime +20 | xargs rm > > to remove backup files created more than 20 days ago. However, running > just 'find' I now see that there seem to be some files in this > directory whose 'mtime' predates the time when I started saving > backups, so maybe 'mtime' isn't the right metric, or perhaps the > Unison code needs to be modified so that 'mtime' works for this > purpose. IIRC, backups are done with ?mv?, so the mtime of the original file will be preserved. I?m not sure what?s the best way to implement this kind of tidying script. > Of course, I mostly use Unison backups to recover from typing the > wrong thing at the Unison prompt, so I could just delete the entire > backup directory as soon as I'm confident that I ran Unison correctly. This is what I do. (Not every time, but periodically when I notice my disk filling up.) - B > > Frederick > > On Sat, Sep 26, 2015 at 11:32:49PM -0400, Benjamin C. Pierce wrote: >> I?d be interested to hear opinions from others on the list. Personally, I think I probably would not want Unison ever to remove backups of deleted files (though, somewhat inconsistently, I do use the maxbackups feature). But if there are many others that would find this feature useful, I would not object strongly to its being added, as long as it is easy to control and off by default. >> >> - Benjamin >> >>> On Sep 26, 2015, at 4:27 PM, c.buhtz at posteo.jp wrote: >>> >>> This FR is related to this Thread on the user-List >>> >>> >>> I am asking for such a feature because I want to have the opinion of >>> the maintainers of that project. >>> I am aware of the current status (not in active developend) and of the >>> less resources of the project. >>> I don't ask about if you (the maintainers) have the resources to >>> implement that. I am just asking if you think it is a good idea and you >>> would accept a patch/pull-request. Maybe you hate this idea and would >>> never accept that feature in an official unison release. It is >>> important do know for any developer who try to implement that. >>> >>> The Feature: >>> Unison should check if the backup-files have a corrosponding original >>> file. If the original file doesn't exists anymore something should be >>> done (e.g. delete it). This could be depended on the age of the >>> bak-file, on the size of the complete backup-directorie, etc. >>> This would prevend the user for wasting storage space. When the user >>> delete an original file the backup-file is still there. Most of the use >>> cases it is in a hidden directory. So the user never think about that. >>> Using unison for some years this would produce a lot of files. >>> >>> In my opinion and habit of developing software this FR could be a >>> bug-report, too. >>> The use of bak-files is not fully implemented. Unison create them >>> automaticly but doesn't take care of there deletion. In german we say: >>> "When you say A you have to say B, too." >>> >>> Minimaly there should be a big-red-bold-hint in the manual about the >>> possible excessive use of storage space the user should take care about >>> himself. >>> >>> So what is your opinion about this feature and would you accept it no >>> matter about your own resources? >>> -- >>> -----BEGIN PGP PUBLIC KEY BLOCK----- >>> Version: GnuPG v1 >>> >>> mQENBFQIluABCACfPwAhRAwFD3NXgv5CtVUGSiqdfJGVViVBqaKd+14E0pASA0MU >>> G0Ewj7O7cGy/ZIoiZ0+lIEZmzJKHfuGwYhXjR/PhnUDrQIHLBvh9WuD6JQuULXfH >>> kXtVm/i9wm76QAcvr2pwYgNzhcJntUHl2GcgnInYbZDeVmg+p9yIPJjuq73/lRS3 >>> 0/McgNoFOBhKK/S6STQuFyjr9OyJyYd1shoM3hmy+kg0HYm6OgQBJNg92WV9jwGe >>> GzlipvEp2jpLwVsTxYir2oOPhfd9D1fC9F/l/3gXbfjd5GIIVrZFq2haZmoVeJ33 >>> LJxo3RA5Tf9LoUeels1b4s9kFz6h7+AHERUpABEBAAG0IUNocmlzdGlhbiBCdWh0 >>> eiA8YnVodHpAcG9zdGVvLmRlPokBPgQTAQIAKAUCVAiW4AIbAwUJAeEzgAYLCQgH >>> AwIGFQgCCQoLBBYCAwECHgECF4AACgkQZLsXsAdRqOxNUAf/V/hDA5zGDpySuCEj >>> DhjiVRK74J9Wd8gfH0WAf1Co5HZ24wZH8rgOIVIgXw8rWkOw/VA6xfdfT+64xjTY >>> Fhkpbrk199nDzp72F7Jc4NC+x8xac2e3rK5ifSWhZx7L5A32pGYE+d16m3EEqImK >>> D4gcZl38x9zdUnD4hHyXkIPz1uCfuMuGgWEnaUk4Wbj41CBZr3O0ABue6regV15U >>> jaes8r+B8iCcY+0yP2kse+3iaCaMqNv5FgQZ9+b2Cql8pFkZJVtBVUw4GW3DWZJi >>> du0O/YrC9TgS+xY9ht/MD2qSHwjcK1sdImjqBO7xP8TIOwKeYyDvGKnSO3EJ/sSA >>> UPGEPrkBDQRUCJbgAQgA0k/Qg67CCUJE2/zuxBEoK4wLJpDRJzh8CQPZpjWx8VP0 >>> KL892jwfxymXn8KNhuy1SgCBFSeV9jg4VZNWDlUGJc2lo82ajr9PzIsrQwu4lf0B >>> zrUWV5hWepKu/kb8uSjx58YYfx0SFz4+9akX3Wwu9TUHntzL5Gk3Q26nnsr1xEJ+ >>> VEumvCH9AE0Tk0K7dQpJ2/JcLuO+uhrpd/lHFDYVN5NsG3P015uFOkDI6N/xNFCj >>> v95XNR93QlfKpK3qWlFGescfG+o/7Ub6s67/i/JoNbw0XgPEHmQfXpD7IHO4cu+p >>> +ETb11cz+1mmi96cy98ID+uTiToJ8G//yD9rmtyxoQARAQABiQElBBgBAgAPBQJU >>> CJbgAhsMBQkB4TOAAAoJEGS7F7AHUajs6sQH/iKs6sPc0vkRJLfbwrijZeecwCWF >>> blo/jzIQ8jPykAj9SLjV20Xwqg3XcJyko8ZU6/zuRJq9xjlv9pZr/oVudQAt6v+h >>> 2Cf4rKEjmau483wjMV2xjTXQhZi9+ttDbia4fgdmGtKsOicn5ae2fFXcXNPu3RiW >>> sZKifWdokA6xqMW6iIG9YjjI5ShxngHWp2xfPscBFMDRtFOMags/Yx+YvwoyEZ4A >>> dURYMFHFqpwILEc8hIzhRg1gq40AHbOaEdczS1Rr3T7/gS6eBs4u6HuY5g2Bierm >>> lLjpspFPjMXwJAa/XLOBjMF2vsHPrZNcouNKkumQ36yq/Pm6DFXAseQDxOk= >>> =PGP9 >>> -----END PGP PUBLIC KEY BLOCK----- >>> _______________________________________________ >>> Unison-hackers mailing list >>> Unison-hackers at lists.seas.upenn.edu >>> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers >> >> _______________________________________________ >> Unison-hackers mailing list >> Unison-hackers at lists.seas.upenn.edu >> http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers > _______________________________________________ > Unison-hackers mailing list > Unison-hackers at lists.seas.upenn.edu > http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers From bcpierce at seas.upenn.edu Mon Sep 28 22:26:15 2015 From: bcpierce at seas.upenn.edu (bcpierce at seas.upenn.edu) Date: Mon, 28 Sep 2015 22:26:15 -0400 Subject: [Unison-hackers] [unison-svn] r546 - in trunk: doc src Message-ID: <201509290226.t8T2QFjc003728@yaws.seas.upenn.edu> Author: bcpierce Date: 2015-09-28 22:26:15 -0400 (Mon, 28 Sep 2015) New Revision: 546 Modified: trunk/doc/unison-manual.tex trunk/src/RECENTNEWS trunk/src/mkProjectInfo.ml Log: * Wording improvement in manual Modified: trunk/doc/unison-manual.tex =================================================================== --- trunk/doc/unison-manual.tex 2015-07-13 21:43:41 UTC (rev 545) +++ trunk/doc/unison-manual.tex 2015-09-29 02:26:15 UTC (rev 546) @@ -1537,7 +1537,7 @@ \SUBSECTION{Keeping Backups}{backups} -When Unison overwrites a file or directory by propagating a new version from +When Unison overwrites (or deletes) a file or directory while propagating changes from the other replica, it can keep the old version around as a backup. There are several preferences that control precisely where these backups are stored and how they are named. Modified: trunk/src/RECENTNEWS =================================================================== --- trunk/src/RECENTNEWS 2015-07-13 21:43:41 UTC (rev 545) +++ trunk/src/RECENTNEWS 2015-09-29 02:26:15 UTC (rev 546) @@ -1,3 +1,8 @@ +CHANGES FROM VERSION 2.49.2 + +* Wording improvement in manual + +------------------------------- CHANGES FROM VERSION 2.49.1 * Update to cygwin install instructions from Mario Emmenlauer Modified: trunk/src/mkProjectInfo.ml =================================================================== --- trunk/src/mkProjectInfo.ml 2015-07-13 21:43:41 UTC (rev 545) +++ trunk/src/mkProjectInfo.ml 2015-09-29 02:26:15 UTC (rev 546) @@ -104,3 +104,4 @@ + From c.buhtz at posteo.jp Wed Sep 30 22:54:30 2015 From: c.buhtz at posteo.jp (c.buhtz at posteo.jp) Date: Thu, 1 Oct 2015 04:54:30 +0200 Subject: [Unison-hackers] [FeatureReqest] auto-delete unneeded backup-files In-Reply-To: <3nNhY205ptz5vMw@dovecot03.posteo.de> References: <3nNhY205ptz5vMw@dovecot03.posteo.de> Message-ID: <3nRJvb6H1Cz5vMJ@dovecot03.posteo.de> Here is an improved and open version of my script "Unison-Walker" to take care of hidden and unnecessary bak-files created by Unison. -- -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v1 mQENBFQIluABCACfPwAhRAwFD3NXgv5CtVUGSiqdfJGVViVBqaKd+14E0pASA0MU G0Ewj7O7cGy/ZIoiZ0+lIEZmzJKHfuGwYhXjR/PhnUDrQIHLBvh9WuD6JQuULXfH kXtVm/i9wm76QAcvr2pwYgNzhcJntUHl2GcgnInYbZDeVmg+p9yIPJjuq73/lRS3 0/McgNoFOBhKK/S6STQuFyjr9OyJyYd1shoM3hmy+kg0HYm6OgQBJNg92WV9jwGe GzlipvEp2jpLwVsTxYir2oOPhfd9D1fC9F/l/3gXbfjd5GIIVrZFq2haZmoVeJ33 LJxo3RA5Tf9LoUeels1b4s9kFz6h7+AHERUpABEBAAG0IUNocmlzdGlhbiBCdWh0 eiA8YnVodHpAcG9zdGVvLmRlPokBPgQTAQIAKAUCVAiW4AIbAwUJAeEzgAYLCQgH AwIGFQgCCQoLBBYCAwECHgECF4AACgkQZLsXsAdRqOxNUAf/V/hDA5zGDpySuCEj DhjiVRK74J9Wd8gfH0WAf1Co5HZ24wZH8rgOIVIgXw8rWkOw/VA6xfdfT+64xjTY Fhkpbrk199nDzp72F7Jc4NC+x8xac2e3rK5ifSWhZx7L5A32pGYE+d16m3EEqImK D4gcZl38x9zdUnD4hHyXkIPz1uCfuMuGgWEnaUk4Wbj41CBZr3O0ABue6regV15U jaes8r+B8iCcY+0yP2kse+3iaCaMqNv5FgQZ9+b2Cql8pFkZJVtBVUw4GW3DWZJi du0O/YrC9TgS+xY9ht/MD2qSHwjcK1sdImjqBO7xP8TIOwKeYyDvGKnSO3EJ/sSA UPGEPrkBDQRUCJbgAQgA0k/Qg67CCUJE2/zuxBEoK4wLJpDRJzh8CQPZpjWx8VP0 KL892jwfxymXn8KNhuy1SgCBFSeV9jg4VZNWDlUGJc2lo82ajr9PzIsrQwu4lf0B zrUWV5hWepKu/kb8uSjx58YYfx0SFz4+9akX3Wwu9TUHntzL5Gk3Q26nnsr1xEJ+ VEumvCH9AE0Tk0K7dQpJ2/JcLuO+uhrpd/lHFDYVN5NsG3P015uFOkDI6N/xNFCj v95XNR93QlfKpK3qWlFGescfG+o/7Ub6s67/i/JoNbw0XgPEHmQfXpD7IHO4cu+p +ETb11cz+1mmi96cy98ID+uTiToJ8G//yD9rmtyxoQARAQABiQElBBgBAgAPBQJU CJbgAhsMBQkB4TOAAAoJEGS7F7AHUajs6sQH/iKs6sPc0vkRJLfbwrijZeecwCWF blo/jzIQ8jPykAj9SLjV20Xwqg3XcJyko8ZU6/zuRJq9xjlv9pZr/oVudQAt6v+h 2Cf4rKEjmau483wjMV2xjTXQhZi9+ttDbia4fgdmGtKsOicn5ae2fFXcXNPu3RiW sZKifWdokA6xqMW6iIG9YjjI5ShxngHWp2xfPscBFMDRtFOMags/Yx+YvwoyEZ4A dURYMFHFqpwILEc8hIzhRg1gq40AHbOaEdczS1Rr3T7/gS6eBs4u6HuY5g2Bierm lLjpspFPjMXwJAa/XLOBjMF2vsHPrZNcouNKkumQ36yq/Pm6DFXAseQDxOk= =PGP9 -----END PGP PUBLIC KEY BLOCK-----