[Unison-hackers] [unison-svn] r285 - in trunk: . doc src src/lwt
Benjamin C. Pierce
bcpierce at seas.upenn.edu
Sat Jun 21 11:18:01 EDT 2008
Author: bcpierce
Date: 2008-06-21 11:17:48 -0400 (Sat, 21 Jun 2008)
New Revision: 285
Removed:
trunk/COPYING
Modified:
trunk/doc/unison-manual.tex
trunk/src/Makefile
trunk/src/RECENTNEWS
trunk/src/common.ml
trunk/src/copy.ml
trunk/src/fileinfo.ml
trunk/src/fileinfo.mli
trunk/src/files.ml
trunk/src/lwt/lwt_unix.ml
trunk/src/mkProjectInfo.ml
trunk/src/os.ml
trunk/src/os.mli
trunk/src/remote.ml
Log:
This version introduces some pretty big changes, by BCP in
collaboration with Alan Schmitt. We've tested them minimally, but
this version should be considered "only for the adventurous" for the
moment.
* Added some more debugging code to remote.ml to give more informative
error messages when we encounter the (dreaded and longstanding)
"assert failed during file transfer" bug
* Experimental support for invoking an external file transfer tool for
whole-file copies instead of Unison's built-in transfer protocol.
Two new preferences have been added:
- copyprog is a string giving the name (and command-line switches,
if needed) of an external program that can be used to copy large
files efficiently. By default, rsync is invoked, but other
tools such as scp can be used instead by changing the value of
this preference. (Although this is not its primary purpose,
rsync is actually a pretty fast way of copying files that don't
already exist on the receiving host.) For files that do already
exist on (but that have been changed in one replica), Unison
will always use its built-in implementation of the rsync
algorithm.
- copythreshold is an integer (-1 by default), indicating above
what filesize (in megabytes) Unison should use the external
copying utility specified by copyprog. Specifying 0 will cause
ALL copies to use the external program; a negative number will
prevent any files from using it. (Default is -1.)
* If Unison is interrupted during a directory transfer, it will now
leave the partially transferred directory intact in a temporary
location. (This maintains the invariant that new files/directories
are transferred either completely or not at all.) The next time
Unison is run, it will continue filling in this temporary directory,
skipping transferring files that it finds are already there.
Deleted: trunk/COPYING
===================================================================
--- trunk/COPYING 2008-06-04 14:30:56 UTC (rev 284)
+++ trunk/COPYING 2008-06-21 15:17:48 UTC (rev 285)
@@ -1,341 +0,0 @@
- GNU GENERAL PUBLIC LICENSE
- Version 2, June 1991
-
- Copyright (C) 1989, 1991 Free Software Foundation, Inc.
- 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- Everyone is permitted to copy and distribute verbatim copies
- of this license document, but changing it is not allowed.
-
- Preamble
-
- The licenses for most software are designed to take away your
-freedom to share and change it. By contrast, the GNU General Public
-License is intended to guarantee your freedom to share and change free
-software--to make sure the software is free for all its users. This
-General Public License applies to most of the Free Software
-Foundation's software and to any other program whose authors commit to
-using it. (Some other Free Software Foundation software is covered by
-the GNU Library General Public License instead.) You can apply it to
-your programs, too.
-
- When we speak of free software, we are referring to freedom, not
-price. Our General Public Licenses are designed to make sure that you
-have the freedom to distribute copies of free software (and charge for
-this service if you wish), that you receive source code or can get it
-if you want it, that you can change the software or use pieces of it
-in new free programs; and that you know you can do these things.
-
- To protect your rights, we need to make restrictions that forbid
-anyone to deny you these rights or to ask you to surrender the rights.
-These restrictions translate to certain responsibilities for you if you
-distribute copies of the software, or if you modify it.
-
- For example, if you distribute copies of such a program, whether
-gratis or for a fee, you must give the recipients all the rights that
-you have. You must make sure that they, too, receive or can get the
-source code. And you must show them these terms so they know their
-rights.
-
- We protect your rights with two steps: (1) copyright the software, and
-(2) offer you this license which gives you legal permission to copy,
-distribute and/or modify the software.
-
- Also, for each author's protection and ours, we want to make certain
-that everyone understands that there is no warranty for this free
-software. If the software is modified by someone else and passed on, we
-want its recipients to know that what they have is not the original, so
-that any problems introduced by others will not reflect on the original
-authors' reputations.
-
- Finally, any free program is threatened constantly by software
-patents. We wish to avoid the danger that redistributors of a free
-program will individually obtain patent licenses, in effect making the
-program proprietary. To prevent this, we have made it clear that any
-patent must be licensed for everyone's free use or not licensed at all.
-
- The precise terms and conditions for copying, distribution and
-modification follow.
-
- GNU GENERAL PUBLIC LICENSE
- TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
-
- 0. This License applies to any program or other work which contains
-a notice placed by the copyright holder saying it may be distributed
-under the terms of this General Public License. The "Program", below,
-refers to any such program or work, and a "work based on the Program"
-means either the Program or any derivative work under copyright law:
-that is to say, a work containing the Program or a portion of it,
-either verbatim or with modifications and/or translated into another
-language. (Hereinafter, translation is included without limitation in
-the term "modification".) Each licensee is addressed as "you".
-
-Activities other than copying, distribution and modification are not
-covered by this License; they are outside its scope. The act of
-running the Program is not restricted, and the output from the Program
-is covered only if its contents constitute a work based on the
-Program (independent of having been made by running the Program).
-Whether that is true depends on what the Program does.
-
- 1. You may copy and distribute verbatim copies of the Program's
-source code as you receive it, in any medium, provided that you
-conspicuously and appropriately publish on each copy an appropriate
-copyright notice and disclaimer of warranty; keep intact all the
-notices that refer to this License and to the absence of any warranty;
-and give any other recipients of the Program a copy of this License
-along with the Program.
-
-You may charge a fee for the physical act of transferring a copy, and
-you may at your option offer warranty protection in exchange for a fee.
-
- 2. You may modify your copy or copies of the Program or any portion
-of it, thus forming a work based on the Program, and copy and
-distribute such modifications or work under the terms of Section 1
-above, provided that you also meet all of these conditions:
-
- a) You must cause the modified files to carry prominent notices
- stating that you changed the files and the date of any change.
-
- b) You must cause any work that you distribute or publish, that in
- whole or in part contains or is derived from the Program or any
- part thereof, to be licensed as a whole at no charge to all third
- parties under the terms of this License.
-
- c) If the modified program normally reads commands interactively
- when run, you must cause it, when started running for such
- interactive use in the most ordinary way, to print or display an
- announcement including an appropriate copyright notice and a
- notice that there is no warranty (or else, saying that you provide
- a warranty) and that users may redistribute the program under
- these conditions, and telling the user how to view a copy of this
- License. (Exception: if the Program itself is interactive but
- does not normally print such an announcement, your work based on
- the Program is not required to print an announcement.)
-
-These requirements apply to the modified work as a whole. If
-identifiable sections of that work are not derived from the Program,
-and can be reasonably considered independent and separate works in
-themselves, then this License, and its terms, do not apply to those
-sections when you distribute them as separate works. But when you
-distribute the same sections as part of a whole which is a work based
-on the Program, the distribution of the whole must be on the terms of
-this License, whose permissions for other licensees extend to the
-entire whole, and thus to each and every part regardless of who wrote it.
-
-Thus, it is not the intent of this section to claim rights or contest
-your rights to work written entirely by you; rather, the intent is to
-exercise the right to control the distribution of derivative or
-collective works based on the Program.
-
-In addition, mere aggregation of another work not based on the Program
-with the Program (or with a work based on the Program) on a volume of
-a storage or distribution medium does not bring the other work under
-the scope of this License.
-
- 3. You may copy and distribute the Program (or a work based on it,
-under Section 2) in object code or executable form under the terms of
-Sections 1 and 2 above provided that you also do one of the following:
-
- a) Accompany it with the complete corresponding machine-readable
- source code, which must be distributed under the terms of Sections
- 1 and 2 above on a medium customarily used for software interchange; or,
-
- b) Accompany it with a written offer, valid for at least three
- years, to give any third party, for a charge no more than your
- cost of physically performing source distribution, a complete
- machine-readable copy of the corresponding source code, to be
- distributed under the terms of Sections 1 and 2 above on a medium
- customarily used for software interchange; or,
-
- c) Accompany it with the information you received as to the offer
- to distribute corresponding source code. (This alternative is
- allowed only for noncommercial distribution and only if you
- received the program in object code or executable form with such
- an offer, in accord with Subsection b above.)
-
-The source code for a work means the preferred form of the work for
-making modifications to it. For an executable work, complete source
-code means all the source code for all modules it contains, plus any
-associated interface definition files, plus the scripts used to
-control compilation and installation of the executable. However, as a
-special exception, the source code distributed need not include
-anything that is normally distributed (in either source or binary
-form) with the major components (compiler, kernel, and so on) of the
-operating system on which the executable runs, unless that component
-itself accompanies the executable.
-
-If distribution of executable or object code is made by offering
-access to copy from a designated place, then offering equivalent
-access to copy the source code from the same place counts as
-distribution of the source code, even though third parties are not
-compelled to copy the source along with the object code.
-
- 4. You may not copy, modify, sublicense, or distribute the Program
-except as expressly provided under this License. Any attempt
-otherwise to copy, modify, sublicense or distribute the Program is
-void, and will automatically terminate your rights under this License.
-However, parties who have received copies, or rights, from you under
-this License will not have their licenses terminated so long as such
-parties remain in full compliance.
-
- 5. You are not required to accept this License, since you have not
-signed it. However, nothing else grants you permission to modify or
-distribute the Program or its derivative works. These actions are
-prohibited by law if you do not accept this License. Therefore, by
-modifying or distributing the Program (or any work based on the
-Program), you indicate your acceptance of this License to do so, and
-all its terms and conditions for copying, distributing or modifying
-the Program or works based on it.
-
- 6. Each time you redistribute the Program (or any work based on the
-Program), the recipient automatically receives a license from the
-original licensor to copy, distribute or modify the Program subject to
-these terms and conditions. You may not impose any further
-restrictions on the recipients' exercise of the rights granted herein.
-You are not responsible for enforcing compliance by third parties to
-this License.
-
- 7. If, as a consequence of a court judgment or allegation of patent
-infringement or for any other reason (not limited to patent issues),
-conditions are imposed on you (whether by court order, agreement or
-otherwise) that contradict the conditions of this License, they do not
-excuse you from the conditions of this License. If you cannot
-distribute so as to satisfy simultaneously your obligations under this
-License and any other pertinent obligations, then as a consequence you
-may not distribute the Program at all. For example, if a patent
-license would not permit royalty-free redistribution of the Program by
-all those who receive copies directly or indirectly through you, then
-the only way you could satisfy both it and this License would be to
-refrain entirely from distribution of the Program.
-
-If any portion of this section is held invalid or unenforceable under
-any particular circumstance, the balance of the section is intended to
-apply and the section as a whole is intended to apply in other
-circumstances.
-
-It is not the purpose of this section to induce you to infringe any
-patents or other property right claims or to contest validity of any
-such claims; this section has the sole purpose of protecting the
-integrity of the free software distribution system, which is
-implemented by public license practices. Many people have made
-generous contributions to the wide range of software distributed
-through that system in reliance on consistent application of that
-system; it is up to the author/donor to decide if he or she is willing
-to distribute software through any other system and a licensee cannot
-impose that choice.
-
-This section is intended to make thoroughly clear what is believed to
-be a consequence of the rest of this License.
-
- 8. If the distribution and/or use of the Program is restricted in
-certain countries either by patents or by copyrighted interfaces, the
-original copyright holder who places the Program under this License
-may add an explicit geographical distribution limitation excluding
-those countries, so that distribution is permitted only in or among
-countries not thus excluded. In such case, this License incorporates
-the limitation as if written in the body of this License.
-
- 9. The Free Software Foundation may publish revised and/or new versions
-of the General Public License from time to time. Such new versions will
-be similar in spirit to the present version, but may differ in detail to
-address new problems or concerns.
-
-Each version is given a distinguishing version number. If the Program
-specifies a version number of this License which applies to it and "any
-later version", you have the option of following the terms and conditions
-either of that version or of any later version published by the Free
-Software Foundation. If the Program does not specify a version number of
-this License, you may choose any version ever published by the Free Software
-Foundation.
-
- 10. If you wish to incorporate parts of the Program into other free
-programs whose distribution conditions are different, write to the author
-to ask for permission. For software which is copyrighted by the Free
-Software Foundation, write to the Free Software Foundation; we sometimes
-make exceptions for this. Our decision will be guided by the two goals
-of preserving the free status of all derivatives of our free software and
-of promoting the sharing and reuse of software generally.
-
- NO WARRANTY
-
- 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
-FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
-OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
-PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
-OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
-MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
-TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
-PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
-REPAIR OR CORRECTION.
-
- 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
-WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
-REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
-INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
-OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
-TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
-YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
-PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
-POSSIBILITY OF SUCH DAMAGES.
-
- END OF TERMS AND CONDITIONS
-
- How to Apply These Terms to Your New Programs
-
- If you develop a new program, and you want it to be of the greatest
-possible use to the public, the best way to achieve this is to make it
-free software which everyone can redistribute and change under these terms.
-
- To do so, attach the following notices to the program. It is safest
-to attach them to the start of each source file to most effectively
-convey the exclusion of warranty; and each file should have at least
-the "copyright" line and a pointer to where the full notice is found.
-
- <one line to give the program's name and a brief idea of what it does.>
- Copyright (C) 19yy <name of author>
-
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program; if not, write to the Free Software
- Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
-
-
-Also add information on how to contact you by electronic and paper mail.
-
-If the program is interactive, make it output a short notice like this
-when it starts in an interactive mode:
-
- Gnomovision version 69, Copyright (C) 19yy name of author
- Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
- This is free software, and you are welcome to redistribute it
- under certain conditions; type `show c' for details.
-
-The hypothetical commands `show w' and `show c' should show the appropriate
-parts of the General Public License. Of course, the commands you use may
-be called something other than `show w' and `show c'; they could even be
-mouse-clicks or menu items--whatever suits your program.
-
-You should also get your employer (if you work as a programmer) or your
-school, if any, to sign a "copyright disclaimer" for the program, if
-necessary. Here is a sample; alter the names:
-
- Yoyodyne, Inc., hereby disclaims all copyright interest in the program
- `Gnomovision' (which makes passes at compilers) written by James Hacker.
-
- <signature of Ty Coon>, 1 April 1989
- Ty Coon, President of Vice
-
-This General Public License does not permit incorporating your program into
-proprietary programs. If your program is a subroutine library, you may
-consider it more useful to permit linking proprietary applications with the
-library. If this is what you want to do, use the GNU Library General
-Public License instead of this License.
-
Modified: trunk/doc/unison-manual.tex
===================================================================
--- trunk/doc/unison-manual.tex 2008-06-04 14:30:56 UTC (rev 284)
+++ trunk/doc/unison-manual.tex 2008-06-21 15:17:48 UTC (rev 285)
@@ -2065,11 +2065,11 @@
To prevent accidents, Unison provides a preference called
\verb|mountpoint|. Including a line like
\begin{verbatim}
- mountpoint = /mnt/foo
+ mountpoint = foo
\end{verbatim}
in your preference file will cause Unison to check, after it finishes
detecting updates, that something actually exists at the path
-\verb|/mnt/foo| on both replicas; if it does not, the Unison run will
+\verb|foo| on both replicas; if it does not, the Unison run will
abort.
\SUBSECTION{Click-starting Unison}{click}
Modified: trunk/src/Makefile
===================================================================
--- trunk/src/Makefile 2008-06-04 14:30:56 UTC (rev 284)
+++ trunk/src/Makefile 2008-06-21 15:17:48 UTC (rev 285)
@@ -363,6 +363,11 @@
checkin:
$(MAKE) -C .. checkin
+installremote:
+ $(MAKE) UISTYLE=text
+ -unison eniac -path current/unison/trunk/src -batch
+ ssh eniac.seas.upenn.edu make -C current/unison/trunk/src installtext
+
####################################################################
# Documentation strings
Modified: trunk/src/RECENTNEWS
===================================================================
--- trunk/src/RECENTNEWS 2008-06-04 14:30:56 UTC (rev 284)
+++ trunk/src/RECENTNEWS 2008-06-21 15:17:48 UTC (rev 285)
@@ -1,3 +1,45 @@
+CHANGES FROM VERSION 2.29.0
+
+This version introduces some pretty big changes, by BCP in
+collaboration with Alan Schmitt. We've tested them minimally, but
+this version should be considered "only for the adventurous" for the
+moment.
+
+* Added some more debugging code to remote.ml to give more informative
+ error messages when we encounter the (dreaded and longstanding)
+ "assert failed during file transfer" bug
+
+* Experimental support for invoking an external file transfer tool for
+ whole-file copies instead of Unison's built-in transfer protocol.
+
+ Two new preferences have been added:
+
+ - copyprog is a string giving the name (and command-line switches,
+ if needed) of an external program that can be used to copy large
+ files efficiently. By default, rsync is invoked, but other
+ tools such as scp can be used instead by changing the value of
+ this preference. (Although this is not its primary purpose,
+ rsync is actually a pretty fast way of copying files that don't
+ already exist on the receiving host.) For files that do already
+ exist on (but that have been changed in one replica), Unison
+ will always use its built-in implementation of the rsync
+ algorithm.
+
+ - copythreshold is an integer (-1 by default), indicating above
+ what filesize (in megabytes) Unison should use the external
+ copying utility specified by copyprog. Specifying 0 will cause
+ ALL copies to use the external program; a negative number will
+ prevent any files from using it. (Default is -1.)
+
+* If Unison is interrupted during a directory transfer, it will now
+ leave the partially transferred directory intact in a temporary
+ location. (This maintains the invariant that new files/directories
+ are transferred either completely or not at all.) The next time
+ Unison is run, it will continue filling in this temporary directory,
+ skipping transferring files that it finds are already there.
+
+
+-------------------------------
CHANGES FROM VERSION 2.28.51
* Propagating changes from 2.27 branch
Modified: trunk/src/common.ml
===================================================================
--- trunk/src/common.ml 2008-06-04 14:30:56 UTC (rev 284)
+++ trunk/src/common.ml 2008-06-21 15:17:48 UTC (rev 285)
@@ -190,7 +190,7 @@
| _ -> false)
| _ -> false
-let rcType ((fi, _, _, _) as rc) =
+let rcType (fi, _, _, _) =
Fileinfo.type2string fi
let riFileType ri =
Modified: trunk/src/copy.ml
===================================================================
--- trunk/src/copy.ml 2008-06-04 14:30:56 UTC (rev 284)
+++ trunk/src/copy.ml 2008-06-21 15:17:48 UTC (rev 285)
@@ -36,7 +36,8 @@
in
Unix.out_channel_of_descr fd
end
- | `RESS len -> Osx.openRessOut fspath path len
+ | `RESS len ->
+ Osx.openRessOut fspath path len
let protect f g =
try
@@ -60,9 +61,7 @@
let localFile
fspathFrom pathFrom fspathTo pathTo realPathTo update desc ressLength ido =
-
let use_id f = match ido with Some id -> f id | None -> () in
-
Util.convertUnixErrorsToTransient
"copying locally"
(fun () ->
@@ -136,8 +135,6 @@
let startReceivingFile
fspath path realPath fileKind update srcFileSize id file_id =
- debug (fun() ->
- Util.msg "startReceivingFile: %s\n" (Fspath.concatToString fspath path));
(* We delay the opening of the file so that there are not too many
temporary files remaining after a crash *)
let outfd = ref None in
@@ -153,15 +150,14 @@
!decompressor;
if Prefs.read rsyncActivated then begin
match update with
- `Update (destFileDataSize, destFileRessSize) when
- let destFileSize =
- match fileKind with
- `DATA -> destFileDataSize
- | `RESS _ -> destFileRessSize
- in
- Transfer.Rsync.aboveRsyncThreshold destFileSize
- &&
- Transfer.Rsync.aboveRsyncThreshold srcFileSize ->
+ `Update (destFileDataSize, destFileRessSize)
+ when let destFileSize =
+ match fileKind with
+ `DATA -> destFileDataSize
+ | `RESS _ -> destFileRessSize
+ in
+ Transfer.Rsync.aboveRsyncThreshold destFileSize
+ && Transfer.Rsync.aboveRsyncThreshold srcFileSize ->
Util.convertUnixErrorsToTransient
"preprocessing file"
(fun () ->
@@ -247,7 +243,6 @@
let compressRemotely = Remote.registerServerCmd "compress" compress
-
let receiveRemBiLocally _ (file_id, bi) =
let bil =
try
@@ -281,7 +276,7 @@
(* We limit the size of the output buffers to about 512 KB
(we cannot go above the limit below plus 64) *)
-let transmitFileReg = Lwt_util.make_region 440
+let transferFileReg = Lwt_util.make_region 440
let bufferSize sz =
min 64 ((truncate (Uutil.Filesize.toFloat sz) + 1023) / 1024)
@@ -314,69 +309,77 @@
| None -> ()
end
-let reallyTransmitFile
+(* The ressOnly flag tells reallyTransferFile to skip transferring
+ the data fork (which has already been taken care of by some external
+ utility) and just transfer the resource fork (which external utilities
+ are not necessarily good at). *)
+let reallyTransferFile
connFrom fspathFrom pathFrom fspathTo pathTo realPathTo
- update desc ressLength id =
- debug (fun() -> Util.msg "getFile(%s,%s) -> (%s,%s,%s,%s)\n"
+ update desc ressLength ressOnly id =
+ debug (fun() -> Util.msg "reallyTransferFile(%s,%s) -> (%s,%s,%s,%s)\n"
(Fspath.toString fspathFrom) (Path.toString pathFrom)
(Fspath.toString fspathTo) (Path.toString pathTo)
(Path.toString realPathTo) (Props.toString desc));
-
let srcFileSize = Props.length desc in
- debug (fun () ->
- Util.msg "src file size = %s bytes\n"
- (Uutil.Filesize.toString srcFileSize));
let file_id = Remote.newMsgId () in
- (* Data fork *)
- startReceivingFile
- fspathTo pathTo realPathTo `DATA update srcFileSize id file_id
- >>= (fun (outfd, infd, firstBi, remBi) ->
- Lwt.catch (fun () ->
- Uutil.showProgress id Uutil.Filesize.zero "f";
- sendRemBi connFrom file_id remBi >>= (fun () ->
- compressRemotely connFrom
- (firstBi,
- fspathFrom, pathFrom, `DATA, srcFileSize, id, file_id)
- >>= (fun () ->
- decompressor :=
- Remote.MsgIdMap.remove file_id !decompressor; (* For GC *)
- close_all infd outfd;
- Lwt.return ())))
- (fun e ->
- decompressor :=
- Remote.MsgIdMap.remove file_id !decompressor; (* For GC *)
- close_all_no_error infd outfd;
- Lwt.fail e) >>= (fun () ->
- (* Ressource fork *)
- if ressLength > Uutil.Filesize.zero then begin
+
+ (if ressOnly then
+ (* Skip data fork *)
+ Lwt.return ()
+ else
+ (* Data fork *)
startReceivingFile
- fspathTo pathTo realPathTo
- (`RESS ressLength) update ressLength id file_id
- >>= (fun (outfd, infd, firstBi, remBi) ->
+ fspathTo pathTo realPathTo `DATA update srcFileSize id file_id
+ >>= (fun (outfd, infd, firstBi, remBi) ->
Lwt.catch (fun () ->
Uutil.showProgress id Uutil.Filesize.zero "f";
sendRemBi connFrom file_id remBi >>= (fun () ->
compressRemotely connFrom
- (firstBi, fspathFrom, pathFrom,
- `RESS ressLength, ressLength, id, file_id)
- >>= (fun () ->
+ (firstBi,
+ fspathFrom, pathFrom, `DATA, srcFileSize, id, file_id)
+ >>= (fun () ->
decompressor :=
Remote.MsgIdMap.remove file_id !decompressor; (* For GC *)
close_all infd outfd;
Lwt.return ())))
- (fun e ->
- decompressor :=
- Remote.MsgIdMap.remove file_id !decompressor; (* For GC *)
- close_all_no_error infd outfd;
- Lwt.fail e))
- end else
- Lwt.return ()) >>= (fun () ->
- begin match update with
- `Update _ -> Fileinfo.set fspathTo pathTo (`Copy realPathTo) desc
- | `Copy -> Fileinfo.set fspathTo pathTo (`Set Props.fileDefault) desc
- end;
- Lwt.return ()))
+ (* catch handler *)
+ (fun e ->
+ decompressor :=
+ Remote.MsgIdMap.remove file_id !decompressor; (* For GC *)
+ close_all_no_error infd outfd;
+ Lwt.fail e)
+ )) >>= (fun () ->
+ (* Resource fork *)
+ (if ressLength > Uutil.Filesize.zero then begin
+ startReceivingFile
+ fspathTo pathTo realPathTo
+ (`RESS ressLength) update ressLength id file_id
+ >>= (fun (outfd, infd, firstBi, remBi) ->
+ Lwt.catch (fun () ->
+ Uutil.showProgress id Uutil.Filesize.zero "f";
+ sendRemBi connFrom file_id remBi >>= (fun () ->
+ compressRemotely connFrom
+ (firstBi, fspathFrom, pathFrom,
+ `RESS ressLength, ressLength, id, file_id)
+ >>= (fun () ->
+ decompressor :=
+ Remote.MsgIdMap.remove file_id !decompressor; (* For GC *)
+ close_all infd outfd;
+ Lwt.return ())))
+ (fun e ->
+ decompressor :=
+ Remote.MsgIdMap.remove file_id !decompressor; (* For GC *)
+ close_all_no_error infd outfd;
+ Lwt.fail e))
+ end else
+ Lwt.return ()) >>= (fun () ->
+ begin match update with
+ `Update _ -> Fileinfo.set fspathTo pathTo (`Copy realPathTo) desc
+ | `Copy -> Fileinfo.set fspathTo pathTo (`Set Props.fileDefault) desc
+ end;
+ Lwt.return ()))
+
(****)
let tryCopyMovedFile fspathTo pathTo realPathTo update desc fp ress id =
@@ -391,7 +394,8 @@
the problem has to do with the RPC mechanism, which was used here to
make a call *back* from the server to the client inside Trace.log so that
the log message would be appended to the log file on the client. *)
- let loggit = if Prefs.read Globals.someHostIsRunningWindows then Util.msg "%s" else Trace.log in
+ let loggit = if Prefs.read Globals.someHostIsRunningWindows
+ then Util.msg "%s" else Trace.log in
match Xferhint.lookup fp with
None ->
false
@@ -432,7 +436,7 @@
(Path.toString candidatePath));
Xferhint.deleteEntry (candidateFspath, candidatePath);
false
- end
+ end
with
Util.Transient s ->
debug (fun () ->
@@ -445,40 +449,110 @@
false)
end
-let transmitFileLocal
- connFrom
- (fspathFrom, pathFrom, fspathTo, pathTo, realPathTo,
- update, desc, fp, ress, id) =
- if
- tryCopyMovedFile
- fspathTo pathTo realPathTo update desc fp ress id
- then
- Lwt.return ()
- else
- reallyTransmitFile
- connFrom fspathFrom pathFrom fspathTo pathTo realPathTo
- update desc (Osx.ressLength ress) id
+let transferFileLocal connFrom
+ (fspathFrom, pathFrom, fspathTo, pathTo, realPathTo,
+ update, desc, fp, ress, ressOnly, id) =
+ if (not ressOnly)
+ && tryCopyMovedFile fspathTo pathTo realPathTo update desc fp ress id
+ then Lwt.return ()
+ else reallyTransferFile
+ connFrom fspathFrom pathFrom fspathTo pathTo realPathTo
+ update desc (Osx.ressLength ress) ressOnly id
-let transmitFileOnRoot =
- Remote.registerRootCmdWithConnection "transmitFile" transmitFileLocal
+let transferFileOnRoot =
+ Remote.registerRootCmdWithConnection "transferFile" transferFileLocal
-let transmitFile
+let transferFile
rootFrom pathFrom rootTo fspathTo pathTo realPathTo
- update desc fp ress id =
+ update desc fp ress ressOnly id =
let bufSz = bufferSize (max (Props.length desc) (Osx.ressLength ress)) in
(* This must be on the client: any lock on the server side may result
in a deadlock under windows *)
- Lwt_util.run_in_region transmitFileReg bufSz (fun () ->
+ Lwt_util.run_in_region transferFileReg bufSz (fun () ->
Abort.check id;
- transmitFileOnRoot rootTo rootFrom
+ transferFileOnRoot rootTo rootFrom
(snd rootFrom, pathFrom, fspathTo, pathTo, realPathTo,
- update, desc, fp, ress, id))
+ update, desc, fp, ress, ressOnly, id))
(****)
-let file
- rootFrom pathFrom rootTo fspathTo pathTo realPathTo
- update desc fp ress id =
+let copyprog =
+ Prefs.createString "copyprog" "rsync --partial --inplace"
+ "External program for copying large files"
+ ("A string giving the name (and command-line switches, if needed) of an "
+ ^ "external program that can be used to copy large files efficiently.")
+
+let copythreshold =
+ Prefs.createInt "copythreshold" (-1)
+ "If nonnegative, use copyprog to transfer files larger than this"
+ ("A number indicating above what filesize (in megabytes) Unison should "
+ ^ "use the external "
+ ^ "copying utility specified by {\\tt copyprog}. Specifying 0 will cause "
+ ^ "{\\em all} copies to use the external program; "
+ ^ "a negative number will prevent any files from using it. (Default is -1.)")
+
+let tryCopyMovedFileLocal connFrom
+ (fspathTo, pathTo, realPathTo, update, desc, fp, ress, id) =
+ Lwt.return (tryCopyMovedFile fspathTo pathTo realPathTo update desc fp ress id)
+
+let tryCopyMovedFileOnRoot =
+ Remote.registerRootCmdWithConnection "tryCopyMovedFile" tryCopyMovedFileLocal
+
+let setFileinfoLocal connFrom (fspathTo, pathTo, desc) =
+ Lwt.return (Fileinfo.set fspathTo pathTo (`Set Props.fileDefault) desc)
+
+let setFileinfoOnRoot =
+ Remote.registerRootCmdWithConnection "setFileinfo" setFileinfoLocal
+
+let targetExistsWithCorrectSize fspathTo pathTo desc =
+ Os.exists fspathTo pathTo
+ && Props.length (Fileinfo.get false fspathTo pathTo).Fileinfo.desc = Props.length desc
+
+let targetExistsWithCorrectSizeLocal connFrom (fspathTo, pathTo, desc) =
+ Lwt.return (targetExistsWithCorrectSize fspathTo pathTo desc)
+
+let targetExistsWithCorrectSizeOnRoot =
+ Remote.registerRootCmdWithConnection
+ "targetExistsWithCorrectSize" targetExistsWithCorrectSizeLocal
+
+let transferFileUsingExternalCopyprog
+ rootFrom pathFrom rootTo fspathTo pathTo realPathTo
+ update desc fp ress id =
+ tryCopyMovedFileOnRoot rootTo rootFrom
+ (fspathTo, pathTo, realPathTo, update, desc, fp, ress, id)
+ >>= (fun b ->
+ if b then Lwt.return ()
+ else begin
+ Trace.log (Printf.sprintf "Using external program to copy %s\n"
+ (Path.toString pathTo));
+ Uutil.showProgress id Uutil.Filesize.zero "ext";
+ let fromSpec =
+ (match fst rootFrom with Common.Local -> "" | Common.Remote h -> h ^ ":")
+ ^ (Fspath.concatToString (snd rootFrom) pathFrom) in
+ let toSpec =
+ (match fst rootTo with Common.Local -> "" | Common.Remote h -> h ^ ":")
+ ^ (Fspath.concatToString fspathTo pathTo) in
+ let cmd = (Prefs.read copyprog) ^ " "
+ ^ (Os.quotes fromSpec) ^ " "
+ ^ (Os.quotes toSpec) in
+ Trace.log (Printf.sprintf "%s\n" cmd);
+ (* FIX: When possible, we should run this in a Lwt, not sequentially. *)
+ let _,log = Os.runExternalProgram cmd in
+ debug (fun() -> Util.msg
+ "transferFileUsingExternalCopyprog: returned\n------\n%s\n-----\n"
+ (Util.trimWhitespace log));
+ targetExistsWithCorrectSizeOnRoot rootTo rootFrom (fspathTo, pathTo, desc)
+ >>= (fun b ->
+ if not b then
+ raise (Util.Transient (Printf.sprintf
+ "External copy program did not create target file (or bad length): %s"
+ (Path.toString pathTo)));
+ Uutil.showProgress id (Props.length desc) "ext";
+ Lwt.return ())
+ end)
+
+let file rootFrom pathFrom rootTo fspathTo pathTo realPathTo
+ update desc fp ress id =
debug (fun() -> Util.msg "copyRegFile(%s,%s) -> (%s,%s,%s,%s,%s)\n"
(Common.root2string rootFrom) (Path.toString pathFrom)
(Common.root2string rootTo) (Path.toString realPathTo)
@@ -492,10 +566,46 @@
update desc (Osx.ressLength ress) (Some id);
Lwt.return ()
| _ ->
- transmitFile
- rootFrom pathFrom rootTo fspathTo pathTo realPathTo
- update desc fp ress id
- end >>= (fun () ->
+ (* Check whether we actually need to copy the file (or whether it
+ already exists from some interrupted previous transfer) *)
+ targetExistsWithCorrectSizeOnRoot
+ rootTo rootFrom (fspathTo, pathTo, desc) >>= (fun b ->
+ if b then begin
+ debug (fun () ->
+ Util.msg "Skipping transfer of %s/%s because it's already there\n"
+ (Fspath.toString fspathTo) (Path.toString pathTo));
+ Lwt.return ()
+ (* Check whether we should use an external program to copy the
+ file *)
+ end else if
+ Prefs.read copyprog <> ""
+ && Prefs.read copythreshold >= 0
+ && Props.length desc >= Uutil.Filesize.ofInt64 (Int64.mul
+ (Int64.of_int 1000000)
+ (Int64.of_int (Prefs.read copythreshold)))
+ && update = `Copy
+ then begin
+ (* First use the external program to copy the data fork *)
+ transferFileUsingExternalCopyprog
+ rootFrom pathFrom rootTo fspathTo pathTo realPathTo
+ update desc fp ress id >>= (fun () ->
+ (* Now use the regular transport mechanism to copy the resource
+ fork *)
+ begin if (Osx.ressLength ress) > Uutil.Filesize.zero then
+ transferFile
+ rootFrom pathFrom rootTo fspathTo pathTo realPathTo
+ update desc fp ress true id
+ else Lwt.return ()
+ end >>= (fun() ->
+ (* Finally, initialize the file info *)
+ setFileinfoOnRoot rootTo rootFrom (fspathTo, pathTo, desc)))
+ end else
+ (* Just transfer the file in the usual way with Unison's
+ built-in facilities *)
+ transferFile
+ rootFrom pathFrom rootTo fspathTo pathTo realPathTo
+ update desc fp ress false id
+ ) end >>= (fun () ->
Trace.showTimer timer;
Lwt.return ())
Modified: trunk/src/fileinfo.ml
===================================================================
--- trunk/src/fileinfo.ml 2008-06-04 14:30:56 UTC (rev 284)
+++ trunk/src/fileinfo.ml 2008-06-21 15:17:48 UTC (rev 285)
@@ -36,9 +36,9 @@
(fun () ->
try
let stats = statFn fromRoot fspath path in
-debugV (fun () ->
- Util.msg "%s: %b %f %f\n" (Fspath.concatToString fspath path)
- fromRoot stats.Unix.LargeFile.st_ctime stats.Unix.LargeFile.st_mtime);
+ debugV (fun () ->
+ Util.msg "%s: %b %f %f\n" (Fspath.concatToString fspath path)
+ fromRoot stats.Unix.LargeFile.st_ctime stats.Unix.LargeFile.st_mtime);
let typ =
match stats.Unix.LargeFile.st_kind with
Unix.S_REG -> `FILE
Modified: trunk/src/fileinfo.mli
===================================================================
--- trunk/src/fileinfo.mli 2008-06-04 14:30:56 UTC (rev 284)
+++ trunk/src/fileinfo.mli 2008-06-21 15:17:48 UTC (rev 285)
@@ -8,10 +8,9 @@
desc : Props.t; osX : Osx.info}
val get : bool -> Fspath.t -> Path.local -> t
-val set :
- Fspath.t -> Path.local ->
- [`Set of Props.t | `Copy of Path.local | `Update of Props.t] ->
- Props.t -> unit
+val set : Fspath.t -> Path.local ->
+ [`Set of Props.t | `Copy of Path.local | `Update of Props.t] ->
+ Props.t -> unit
(* IF THIS CHANGES, MAKE SURE TO INCREMENT THE ARCHIVE VERSION NUMBER! *)
type stamp =
Modified: trunk/src/files.ml
===================================================================
--- trunk/src/files.ml 2008-06-04 14:30:56 UTC (rev 284)
+++ trunk/src/files.ml 2008-06-21 15:17:48 UTC (rev 285)
@@ -128,8 +128,15 @@
Remote.registerRootCmd
"mkdir"
(fun (fspath,(workingDir,path)) ->
- Os.createDir workingDir path Props.dirDefault;
- Lwt.return (Fileinfo.get false workingDir path).Fileinfo.desc)
+ let createIt() = Os.createDir workingDir path Props.dirDefault in
+ if Os.exists workingDir path then
+ if (Fileinfo.get false workingDir path).Fileinfo.typ <> `DIRECTORY then begin
+ Os.delete workingDir path;
+ createIt()
+ end else ()
+ else
+ createIt();
+ Lwt.return (Fileinfo.get false workingDir path).Fileinfo.desc)
let mkdir onRoot workingDir path = mkdirRemote onRoot (workingDir,path)
@@ -301,7 +308,7 @@
let setupTargetPathsLocal (fspath, path) =
let localPath = Update.translatePathLocal fspath path in
let (workingDir,realPath) = Fspath.findWorkingDir fspath localPath in
- let tempPath = Os.tempPath workingDir realPath in
+ let tempPath = Os.tempPath ~fresh:false workingDir realPath in
Lwt.return (workingDir, realPath, tempPath, localPath)
let setupTargetPaths =
@@ -413,185 +420,37 @@
| Update.NoArchive ->
assert false
in
+ (* BCP (6/08): We used to have an unwindProtect here that would *always* do the
+ final performDelete. This was removed so that failed partial transfers can
+ be restarted. *)
+ Update.transaction (fun id ->
+ (* Update the archive on the source replica (but don't commit
+ the changes yet) and return the part of the new archive
+ corresponding to this path *)
+ Update.updateArchive rootFrom pathFrom uiFrom id
+ >>= (fun (localPathFrom, archFrom) ->
+ let make_backup =
+ (* Perform (asynchronously) a backup of the destination files *)
+ Update.updateArchive rootTo pathTo uiTo id
+ in
+ copyRec localPathFrom tempPathTo realPathTo archFrom >>= (fun () ->
+ make_backup >>= (fun _ ->
+ (* BCP: We put the unwindProtect here instead, so that we clean everything
+ up if there is a failure during the paranoid checking phase. *)
Remote.Thread.unwindProtect
(fun () ->
- Update.transaction (fun id ->
- (* Update the archive on the source replica (but don't commit
- the changes yet) and return the part of the new archive
- corresponding to this path *)
- Update.updateArchive rootFrom pathFrom uiFrom id
- >>= (fun (localPathFrom, archFrom) ->
- let make_backup =
- (* Perform (asynchronously) a backup of the destination files *)
- Update.updateArchive rootTo pathTo uiTo id
- in
- copyRec localPathFrom tempPathTo realPathTo archFrom >>= (fun () ->
- make_backup >>= (fun _ ->
- Update.replaceArchive
- rootTo pathTo (Some (workingDir, tempPathTo))
- archFrom id true >>= (fun _ ->
- rename rootTo pathTo localPathTo workingDir tempPathTo realPathTo uiTo ))))))
+ Update.replaceArchive
+ rootTo pathTo (Some (workingDir, tempPathTo))
+ archFrom id true >>= (fun _ ->
+ rename rootTo pathTo localPathTo workingDir tempPathTo realPathTo uiTo))
(fun _ ->
- performDelete rootTo (Some workingDir, tempPathTo)))
+ debug (fun() -> Util.msg "Removing temp files\n");
+ performDelete rootTo (Some workingDir, tempPathTo) ))))))
-(* A PARTIALLY COMPLETE HACKED VERSION THAT COPIES DIRECTORIES IN PLACE -- Jan 14, 2006
-
-(This is left here in case Jerome gets a chance to finish it...)
-
-let copy
- update
- rootFrom pathFrom (* copy from here... *)
- uiFrom (* (and then check that this updateItem still
- describes the current state of the src replica) *)
- rootTo pathTo (* ...to here *)
- uiTo (* (but, before committing the copy, check that
- this updateItem still describes the current
- state of the target replica) *)
- id = (* for progress display *)
- debug (fun() ->
- Util.msg
- "copy %s %s ---> %s %s \n"
- (root2string rootFrom) (Path.toString pathFrom)
- (root2string rootTo) (Path.toString pathTo));
- (* Inner loop for recursive copy... *)
- let rec copyRec pFrom (* Path to copy from *)
- pTo (* (Temp) path to copy to *)
- f = (* Source archive subtree for this path *)
- debug (fun() ->
- Util.msg "copyRec %s --> %s\n"
- (Path.toString pFrom) (Path.toString pTo));
- (* Calculate target paths *)
- setupTargetPaths rootTo pTo
- >>= (fun (workingDir, realPathTo, tempPathTo, _) ->
- match f with
- Update.ArchiveFile (desc, dig, stamp, ress) ->
- Lwt_util.run_in_region copyReg 1 (fun () ->
- Abort.check id;
- Copy.file
- rootFrom pFrom rootTo workingDir tempPathTo realPathTo
- update desc dig ress id
- >>= (fun () ->
- checkContentsChange rootFrom pFrom desc dig stamp ress))
- | Update.ArchiveSymlink l ->
- Lwt_util.run_in_region copyReg 1 (fun () ->
- debug (fun() -> Util.msg "Making symlink %s/%s -> %s\n"
- (root2string rootTo) (Path.toString pTo) l);
- Abort.check id;
- makeSymlink rootTo (workingDir, tempPathTo, l))
- | Update.ArchiveDir (desc, children) ->
- Lwt_util.run_in_region copyReg 1 (fun () ->
- debug (fun() -> Util.msg "Creating directory %s/%s\n"
- (root2string rootTo) (Path.toString pTo));
- mkdir rootTo workingDir realPathTo) >>= (fun initialDesc ->
- Abort.check id;
- let runningThreads = ref [] in
- Lwt.catch
- (fun () ->
- Update.NameMap.iter
- (fun name child ->
- let thread =
- copyRec (Path.child pFrom name)
- (Path.child pTo name)
- child
- in
- runningThreads := thread :: !runningThreads)
- children;
- Lwt_util.join !runningThreads)
- (fun e ->
- (* If one thread fails (in a non-fatal way), we wait for
- all other threads to terminate before continuing *)
- if not (Abort.testException e) then Abort.file id;
- match e with
- Util.Transient _ ->
- let e = ref e in
- Lwt_util.iter
- (fun act ->
- Lwt.catch
- (fun () -> act)
- (fun e' ->
- match e' with
- Util.Transient _ ->
- if Abort.testException !e then e := e';
- Lwt.return ()
- | _ ->
- Lwt.fail e'))
- !runningThreads >>= (fun () ->
- Lwt.fail !e)
- | _ ->
- Lwt.fail e) >>= (fun () ->
- Lwt_util.run_in_region copyReg 1 (fun () ->
- (* We use the actual file permissions so as to preserve
- inherited bits *)
- Abort.check id;
- setPropRemote rootTo
- (workingDir, realPathTo, `Set initialDesc, desc))))
- | Update.NoArchive ->
- assert false)
- in
- Update.transaction (fun id ->
- (* Update the archive on the source replica (but don't commit
- the changes yet) and return the part of the new archive
- corresponding to this path *)
- Update.updateArchive rootFrom pathFrom uiFrom id
- >>= (fun (localPathFrom, archFrom) ->
- (* Perform (asynchronously) a backup of the destination files *)
- Update.updateArchive rootTo pathTo uiTo id >>= (fun _ ->
-(*
- Remote.Thread.unwindProtect
- (fun () ->
-*)
- copyRec localPathFrom pathTo archFrom >>= (fun () ->
-(*
- rename rootTo pathTo workingDir tempPathTo realPathTo uiTo ))
- (fun _ ->
- performDelete rootTo (Some workingDir, tempPathTo)) >>= (fun () ->
-*)
- Update.replaceArchive
- rootTo pathTo (Some (workingDir, tempPathTo))
- archFrom id true))))
-
-*)
-
-
(* ------------------------------------------------------------ *)
-let readChannelTillEof c =
- let rec loop lines =
- try let l = input_line c in
- loop (l::lines)
- with End_of_file -> lines in
- String.concat "\n" (Safelist.rev (loop []))
-
-let readChannelTillEof_lwt c =
- let rec loop lines =
- let lo =
- try
- Some(Lwt_unix.run (Lwt_unix.input_line c))
- with End_of_file -> None
- in
- match lo with
- Some l -> loop (l :: lines)
- | None -> lines
- in
- String.concat "\n" (Safelist.rev (loop []))
-
let (>>=) = Lwt.bind
-let readChannelsTillEof l =
- let rec suckitdry lines c =
- Lwt.catch
- (fun() -> Lwt_unix.input_line c >>= (fun l -> return (Some l)))
- (fun e -> match e with End_of_file -> return None | _ -> raise e)
- >>= (fun lo ->
- match lo with
- None -> return lines
- | Some l -> suckitdry (l :: lines) c) in
- Lwt_util.map
- (fun c ->
- suckitdry [] c
- >>= (fun res -> return (String.concat "\n" (Safelist.rev res))))
- l
-
let diffCmd =
Prefs.createString "diff" "diff -u CURRENT2 CURRENT1"
"*command for showing differences between files"
@@ -603,16 +462,6 @@
^ "diffed. If not, the two filenames will be appended to the command. In both "
^ "cases, the filenames are suitably quoted.")
-(* Using single quotes is simpler under Unix but they are not accepted
- by the Windows shell. Double quotes without further quoting is
- sufficient with Windows as filenames are not allowed to contain
- double quotes. *)
-let quotes s =
- if Util.osType = `Win32 && not Util.isCygwin then
- "\"" ^ s ^ "\""
- else
- "'" ^ Util.replacesubstring s "'" "'\''" ^ "'"
-
let tempName s = Os.tempFilePrefix ^ s
let rec diff root1 path1 ui1 root2 path2 ui2 showDiff id =
@@ -625,12 +474,12 @@
let cmd =
if Util.findsubstring "CURRENT1" (Prefs.read diffCmd) = None then
(Prefs.read diffCmd)
- ^ " " ^ (quotes (Fspath.toString fspath1))
- ^ " " ^ (quotes (Fspath.toString fspath2))
+ ^ " " ^ (Os.quotes (Fspath.toString fspath1))
+ ^ " " ^ (Os.quotes (Fspath.toString fspath2))
else
Util.replacesubstrings (Prefs.read diffCmd)
- ["CURRENT1", quotes (Fspath.toString fspath1);
- "CURRENT2", quotes (Fspath.toString fspath2)] in
+ ["CURRENT1", Os.quotes (Fspath.toString fspath1);
+ "CURRENT2", Os.quotes (Fspath.toString fspath2)] in
(* Doesn't seem to work well on Windows!
let c = Lwt_unix.run (Lwt_unix.open_process_in cmd) in *)
let c = Unix.open_process_in
@@ -640,7 +489,7 @@
"\"" ^ cmd ^ "\""
else
cmd) in
- showDiff cmd (readChannelTillEof c);
+ showDiff cmd (Os.readChannelTillEof c);
ignore (Unix.close_process_in c) in
let (desc1, fp1, ress1, desc2, fp2, ress2) = Common.fileInfos ui1 ui2 in
match root1,root2 with
@@ -887,47 +736,15 @@
let dig2 = Os.fingerprint workingDirForMerge working2 info2 in
let cmd = formatMergeCmd
path
- (quotes (Fspath.concatToString workingDirForMerge working1))
- (quotes (Fspath.concatToString workingDirForMerge working2))
- (match arch with None -> None | Some f -> Some(quotes (Fspath.toString f)))
- (quotes (Fspath.concatToString workingDirForMerge new1))
- (quotes (Fspath.concatToString workingDirForMerge new2))
- (quotes (Fspath.concatToString workingDirForMerge newarch)) in
+ (Os.quotes (Fspath.concatToString workingDirForMerge working1))
+ (Os.quotes (Fspath.concatToString workingDirForMerge working2))
+ (match arch with None -> None | Some f -> Some(Os.quotes (Fspath.toString f)))
+ (Os.quotes (Fspath.concatToString workingDirForMerge new1))
+ (Os.quotes (Fspath.concatToString workingDirForMerge new2))
+ (Os.quotes (Fspath.concatToString workingDirForMerge newarch)) in
Trace.log (Printf.sprintf "Merge command: %s\n" cmd);
- let returnValue, mergeResultLog =
- if Util.osType = `Win32 && not Util.isCygwin then
- let c = Unix.open_process_in ("\"" ^ cmd ^ "\"") in
- let mergeLog = readChannelTillEof c in
- let returnValue = Unix.close_process_in c in
-
- let mergeResultLog =
- cmd ^
- (if mergeLog <> "" then "\n\n" ^ mergeLog else "") ^
- (if returnValue <> Unix.WEXITED 0 then
- "\n\n" ^ Util.process_status_to_string returnValue
- else
- "") in
- (returnValue,mergeResultLog)
- else Lwt_unix.run (
- Lwt_unix.open_process_full cmd (Unix.environment ())
- >>= (fun (out, ipt, err) ->
- readChannelsTillEof [out;err]
- >>= (function [mergeLogOut;mergeLogErr] ->
- Lwt_unix.close_process_full (out, ipt, err)
- >>= (fun returnValue ->
- return (returnValue, (
- cmd
- ^ "\n\n" ^
- (if mergeLogOut = "" || mergeLogErr = ""
- then mergeLogOut ^ mergeLogErr
- else mergeLogOut ^ "\n\n" ^ ("Error Output:"^mergeLogErr))
- ^"\n\n"
- ^ (if returnValue = Unix.WEXITED 0
- then ""
- else Util.process_status_to_string returnValue))))
- (* Stop typechechecker from complaining about non-exhaustive pattern above *)
- | _ -> assert false))) in
+ let returnValue, mergeResultLog = Os.runExternalProgram cmd in
Trace.log (Printf.sprintf "Merge result (%s):\n%s\n"
(showStatus returnValue) mergeResultLog);
Modified: trunk/src/lwt/lwt_unix.ml
===================================================================
--- trunk/src/lwt/lwt_unix.ml 2008-06-04 14:30:56 UTC (rev 284)
+++ trunk/src/lwt/lwt_unix.ml 2008-06-21 15:17:48 UTC (rev 285)
@@ -406,7 +406,15 @@
Unix.close in_write;
Lwt.return (inchan, outchan)))
-let open_proc_full cmd env proc input output error toclose =
+(* FIX: Subprocesses that use /dev/tty to print things on the terminal
+ will NOT have this output captured and returned to the caller of this
+ function. There's an argument that this is correct, but if we are
+ running from a GUI the user may not be looking at any terminal and it
+ will appear that the process is just hanging. This can be fixed, in
+ principle, by writing a little C code that opens /dev/tty and then uses
+ the TIOCNOTTY ioctl control to detach the terminal. *)
+
+let open_proc_full cmd env proc output input error toclose =
match Unix.fork () with
0 -> Unix.dup2 input Unix.stdin; Unix.close input;
Unix.dup2 output Unix.stdout; Unix.close output;
@@ -420,15 +428,15 @@
Lwt.bind (pipe ()) (fun (in_read, in_write) ->
Lwt.bind (pipe ()) (fun (out_read, out_write) ->
Lwt.bind (pipe ()) (fun (err_read, err_write) ->
- let inchan = Unix.in_channel_of_descr in_read in
- let outchan = Unix.out_channel_of_descr out_write in
+ let inchan = Unix.out_channel_of_descr in_write in
+ let outchan = Unix.in_channel_of_descr out_read in
let errchan = Unix.in_channel_of_descr err_read in
- open_proc_full cmd env (Process_full(inchan, outchan, errchan))
- out_read in_write err_write [in_read; out_write; err_read];
- Unix.close out_read;
- Unix.close in_write;
+ open_proc_full cmd env (Process_full(outchan, inchan, errchan))
+ out_write in_read err_write [in_write; out_read; err_read];
+ Unix.close out_write;
+ Unix.close in_read;
Unix.close err_write;
- Lwt.return (inchan, outchan, errchan))))
+ Lwt.return (outchan, inchan, errchan))))
let find_proc_id fun_name proc =
try
@@ -453,9 +461,9 @@
close_in inchan; close_out outchan;
Lwt.bind (waitpid [] pid) (fun (_, status) -> Lwt.return status)
-let close_process_full (inchan, outchan, errchan) =
+let close_process_full (outchan, inchan, errchan) =
let pid =
find_proc_id "close_process_full"
- (Process_full(inchan, outchan, errchan)) in
- close_in inchan; close_out outchan; close_in errchan;
+ (Process_full(outchan, inchan, errchan)) in
+ close_out inchan; close_in outchan; close_in errchan;
Lwt.bind (waitpid [] pid) (fun (_, status) -> Lwt.return status)
Modified: trunk/src/mkProjectInfo.ml
===================================================================
--- trunk/src/mkProjectInfo.ml 2008-06-04 14:30:56 UTC (rev 284)
+++ trunk/src/mkProjectInfo.ml 2008-06-21 15:17:48 UTC (rev 285)
@@ -5,8 +5,8 @@
let projectName = "unison"
let majorVersion = 2
-let minorVersion = 28
-let pointVersionOrigin = 229 (* Revision that corresponds to point version 0 *)
+let minorVersion = 29
+let pointVersionOrigin = 284 (* Revision that corresponds to point version 0 *)
(* Documentation:
This is a program to construct a version of the form Major.Minor.Point,
@@ -70,3 +70,4 @@
+
Modified: trunk/src/os.ml
===================================================================
--- trunk/src/os.ml 2008-06-04 14:30:56 UTC (rev 284)
+++ trunk/src/os.ml 2008-06-21 15:17:48 UTC (rev 285)
@@ -30,6 +30,20 @@
(*****************************************************************************)
+(* ESCAPING SHELL PARAMETERS *)
+(*****************************************************************************)
+
+(* Using single quotes is simpler under Unix but they are not accepted
+ by the Windows shell. Double quotes without further quoting is
+ sufficient with Windows as filenames are not allowed to contain
+ double quotes. *)
+let quotes s =
+ if Util.osType = `Win32 && not Util.isCygwin then
+ "\"" ^ s ^ "\""
+ else
+ "'" ^ Util.replacesubstring s "'" "'\''" ^ "'"
+
+(*****************************************************************************)
(* QUERYING THE FILESYSTEM *)
(*****************************************************************************)
@@ -108,10 +122,18 @@
Util.startswith file tempFilePrefix
then begin
if Util.endswith file !tempFileSuffix then begin
- let newPath = Path.child path filename in
- debug (fun()-> Util.msg "deleting old temp file %s\n"
- (Fspath.concatToString fspath newPath));
- delete fspath newPath
+ let p = Path.child path filename in
+ let i = Fileinfo.get false fspath p in
+ let secondsinaweek = 604800.0 in
+ if Props.time i.Fileinfo.desc +. secondsinaweek < Util.time()
+ then begin
+ debug (fun()-> Util.msg "deleting old temp file %s\n"
+ (Fspath.concatToString fspath p));
+ delete fspath p
+ end else
+ debug (fun()-> Util.msg
+ "keeping temp file %s since it is less than a week old\n"
+ (Fspath.concatToString fspath p));
end;
false
end else
@@ -289,7 +311,7 @@
(*****************************************************************************)
(* Generates an unused fspath for a temporary file. *)
-let freshPath fspath path prefix suffix =
+let genTempPath fresh fspath path prefix suffix =
let rec f i =
let s =
if i=0 then suffix
@@ -299,11 +321,11 @@
(Path.addSuffixToFinalName path s)
prefix
in
- if exists fspath tempPath then f (i + 1) else tempPath
+ if fresh && exists fspath tempPath then f (i + 1) else tempPath
in f 0
-let tempPath fspath path =
- freshPath fspath path tempFilePrefix !tempFileSuffix
+let tempPath ?(fresh=true) fspath path =
+ genTempPath fresh fspath path tempFilePrefix !tempFileSuffix
(*****************************************************************************)
(* INTERRUPTED SYSTEM CALLS *)
@@ -318,3 +340,81 @@
try Unix.accept fd
with Unix.Unix_error(Unix.EINTR,_,_) -> loop() in
loop()
+
+
+(*****************************************************************************)
+(* RUNNING EXTERNAL PROGRAMS *)
+(*****************************************************************************)
+
+let (>>=) = Lwt.bind
+open Lwt
+
+let readChannelTillEof c =
+ let rec loop lines =
+ try let l = input_line c in
+ loop (l::lines)
+ with End_of_file -> lines in
+ String.concat "\n" (Safelist.rev (loop []))
+
+let readChannelTillEof_lwt c =
+ let rec loop lines =
+ let lo =
+ try
+ Some(Lwt_unix.run (Lwt_unix.input_line c))
+ with End_of_file -> None
+ in
+ match lo with
+ Some l -> loop (l :: lines)
+ | None -> lines
+ in
+ String.concat "\n" (Safelist.rev (loop []))
+
+let readChannelsTillEof l =
+ let rec suckitdry lines c =
+ Lwt.catch
+ (fun() -> Lwt_unix.input_line c >>= (fun l -> return (Some l)))
+ (fun e -> match e with End_of_file -> return None | _ -> raise e)
+ >>= (fun lo ->
+ match lo with
+ None -> return lines
+ | Some l -> suckitdry (l :: lines) c) in
+ Lwt_util.map
+ (fun c ->
+ suckitdry [] c
+ >>= (fun res -> return (String.concat "\n" (Safelist.rev res))))
+ l
+
+let runExternalProgram cmd =
+ if Util.osType = `Win32 && not Util.isCygwin then begin
+ debug (fun()-> Util.msg "Executing external program windows-style\n");
+ let c = Unix.open_process_in ("\"" ^ cmd ^ "\"") in
+ let mergeLog = readChannelTillEof c in
+ let returnValue = Unix.close_process_in c in
+ let mergeResultLog =
+ cmd ^
+ (if mergeLog <> "" then "\n\n" ^ mergeLog else "") ^
+ (if returnValue <> Unix.WEXITED 0 then
+ "\n\n" ^ Util.process_status_to_string returnValue
+ else
+ "") in
+ (returnValue,mergeResultLog)
+ end else Lwt_unix.run (
+ debug (fun()-> Util.msg "Executing external program unix-style\n");
+ Lwt_unix.open_process_full cmd (Unix.environment ())
+ >>= (fun (out, ipt, err) ->
+ readChannelsTillEof [out;err]
+ >>= (function [mergeLogOut;mergeLogErr] ->
+ Lwt_unix.close_process_full (out, ipt, err)
+ >>= (fun returnValue ->
+ return (returnValue, (
+ cmd
+ ^ "\n\n" ^
+ (if mergeLogOut = "" || mergeLogErr = ""
+ then mergeLogOut ^ mergeLogErr
+ else mergeLogOut ^ "\n\n" ^ ("Error Output:"^mergeLogErr))
+ ^"\n\n"
+ ^ (if returnValue = Unix.WEXITED 0
+ then ""
+ else Util.process_status_to_string returnValue))))
+ (* Stop typechechecker from complaining about non-exhaustive pattern above *)
+ | _ -> assert false)))
Modified: trunk/src/os.mli
===================================================================
--- trunk/src/os.mli 2008-06-04 14:30:56 UTC (rev 284)
+++ trunk/src/os.mli 2008-06-21 15:17:48 UTC (rev 285)
@@ -3,7 +3,7 @@
val myCanonicalHostName : string
-val tempPath : Fspath.t -> Path.local -> Path.local
+val tempPath : ?fresh:bool -> Fspath.t -> Path.local -> Path.local
val tempFilePrefix : string
val includeInTempNames : string -> unit
@@ -54,3 +54,6 @@
((Fspath.t * Path.local) -> (Fspath.t * Path.local) -> unit) ->
unit
+val readChannelTillEof : in_channel -> string
+val runExternalProgram : string -> Unix.process_status * string
+val quotes : string -> string
Modified: trunk/src/remote.ml
===================================================================
--- trunk/src/remote.ml 2008-06-04 14:30:56 UTC (rev 284)
+++ trunk/src/remote.ml 2008-06-21 15:17:48 UTC (rev 285)
@@ -343,11 +343,22 @@
else
String.sub s p len
+(* An integer just a little smaller than the maximum representable in 30 bits *)
+let hugeint = 1000000000
+
let safeMarshal marshalPayload tag data rem =
let (rem', length) = marshalPayload data rem in
+ if length > hugeint then begin
+ let start = first_chars (min length 10) rem' in
+ let start = if length > 10 then start ^ "..." else start in
+ let start = String.escaped start in
+ Util.msg "Fatal error in safeMarshal: sending too many (%d) bytes with tag %s and contents [%s]\n" length tag start;
+ raise (Util.Fatal ((Printf.sprintf
+ "Message payload too large (%d, %s, [%s]). \n" length tag start)
+ ^ "This is a bug in Unison; if it happens to you in a repeatable way, \n"
+ ^ "please post a report on the unison-users mailing list."))
+ end;
let l = String.length tag in
- assert (length > 0); (* tracking down an assert failure in receivePacket... *)
- assert (l > 0);
debugE (fun() ->
let start = first_chars (min length 10) rem' in
let start = if length > 10 then start ^ "..." else start in
@@ -585,7 +596,7 @@
type msgId = int
module MsgIdMap = Map.Make (struct type t = msgId let compare = compare end)
let ids = ref 1
-let newMsgId () = incr ids; if !ids = 1000000000 then ids := 2; !ids
+let newMsgId () = incr ids; if !ids = hugeint then ids := 2; !ids
(* Threads waiting for a response from the other side *)
let receivers = ref MsgIdMap.empty
More information about the Unison-hackers
mailing list