[Unison-hackers] [unison-svn] r305 - in trunk: . doc src src/lwt
Benjamin C. Pierce
bcpierce at seas.upenn.edu
Thu Aug 21 11:39:08 EDT 2008
Author: bcpierce
Date: 2008-08-21 11:39:05 -0400 (Thu, 21 Aug 2008)
New Revision: 305
Modified:
trunk/Makefile
trunk/doc/unison-manual.tex
trunk/src/Makefile
trunk/src/Makefile.OCaml
trunk/src/NEWS
trunk/src/RECENTNEWS
trunk/src/TODO.txt
trunk/src/copy.ml
trunk/src/lwt/Makefile
trunk/src/main.ml
trunk/src/mkProjectInfo.ml
trunk/src/os.ml
trunk/src/pty.c
trunk/src/strings.ml
trunk/src/update.ml
Log:
* Updated documentation with recently added preferences.
* Applied patch from Antoine Reilles for NetBSD compilation
* Makefile tidying
Modified: trunk/Makefile
===================================================================
--- trunk/Makefile 2008-07-05 17:43:33 UTC (rev 304)
+++ trunk/Makefile 2008-08-21 15:39:05 UTC (rev 305)
@@ -186,9 +186,9 @@
exportnative:
-$(RM) -r $(EXPORTTMP)
cp -r src $(EXPORTTMP)
- make realexportnative
+ $(MAKE) realexportnative
ifeq ($(OSARCH),linux)
- make realexportnative EXPORTSTATIC=true KIND=-static
+ $(MAKE) realexportnative EXPORTSTATIC=true KIND=-static
endif
$(RM) -r $(EXPORTTMP)
Modified: trunk/doc/unison-manual.tex
===================================================================
--- trunk/doc/unison-manual.tex 2008-07-05 17:43:33 UTC (rev 304)
+++ trunk/doc/unison-manual.tex 2008-08-21 15:39:05 UTC (rev 305)
@@ -2025,15 +2025,16 @@
\SUBSECTION{Making Unison Faster on Large Files}{speeding}
Unison's built-in implementation of the rsync algorithm makes transferring
-updates to existing files pretty fast. However, for whole-file copies, the
-built-in transfer method is not highly optimized. Also, if Unison is
-interrupted in the middle of transferring a large file, it will attempt to
-retransfer the whole thing on the next run.
+updates to existing files pretty fast. However, for whole-file copies of
+newly created files, the built-in transfer method is not highly optimized.
+Also, if Unison is interrupted in the middle of transferring a large file,
+it will attempt to retransfer the whole thing on the next run.
-These shortcomings can be addressed by telling Unison to use an external
-file copying utility for whole-file transfers. The recommended one is the
-standalone {\tt rsync} tool, which is available by default on most Unix
-systems and can easily be installed on Windows systems using Cygwin.
+These shortcomings can be addressed with a little extra work by telling
+Unison to use an external file copying utility for whole-file transfers.
+The recommended one is the standalone {\tt rsync} tool, which is available
+by default on most Unix systems and can easily be installed on Windows
+systems using Cygwin.
If you have {\tt rsync} installed on both hosts, you can make Unison use it
simply by setting the {\tt copythreshold} flag to something non-negative.
@@ -2044,15 +2045,35 @@
this size (which is given in kilobytes, so setting it to 1000 will cause the
external tool to be used for all transfers larger than a megabyte).
-If you want to use a different external copy utility, set the {\tt copyprog}
-preference.
+If you want to use a different external copy utility, set both the {\tt
+ copyprog} and {\tt copyprogpartial} preferences---the former is used for
+the first transfer of a file, while the latter is used when Unison sees a
+partially transferred temp file on the receiving host. Be careful here:
+Your external tool needs to be instructed to copy files in place (otherwise
+if the transfer is interrupted Unison will not notice that some of the data
+has already been transferred, the next time it tries). The default values
+are:
+\begin{verbatim}
+ copyprog = rsync --inplace --compress
+ copyprogrest = rsync --partial --inplace --compress
+\end{verbatim}
+You may also need to set the {\tt copyquoterem} preference. When it is set
+to {\tt true}, this causes Unison to add an extra layer of quotes to
+the remote path passed to the external copy program. This is is needed by
+rsync, for example, which internally uses an ssh connection, requiring an
+extra level of quoting for paths containing spaces. When this flag is set to
+{\tt default}, extra quotes are added if the value of {\tt copyprog}
+contains the string {\tt rsync}. The default value is {\tt default},
+naturally.
If a {\em directory} transfer is interrupted, the next run of Unison will
-skip any files that were completely transferred before the interruption.
-Note, though, that the new directory will not appear in the destination
-filesystem until everything has been transferred---partially transferred
-directories are kept in a temporary location (with names like {\tt
- .unison.DIRNAME....}) until the transfer is complete.
+automatically skip any files that were completely transferred before the
+interruption. (This behavior is always on: it does not depend on the
+setting of the {\tt copythreshold} preference.) Note, though, that the new
+directory will not appear in the destination filesystem until everything has
+been transferred---partially transferred directories are kept in a temporary
+location (with names like {\tt .unison.DIRNAME....}) until the transfer is
+complete.
\SUBSECTION{Fast Update Detection}{fastcheck}
Modified: trunk/src/Makefile
===================================================================
--- trunk/src/Makefile 2008-07-05 17:43:33 UTC (rev 304)
+++ trunk/src/Makefile 2008-08-21 15:39:05 UTC (rev 305)
@@ -74,10 +74,10 @@
install: doinstall
installtext:
- make -C .. installtext
+ $(MAKE) -C .. installtext
text:
- make -C .. text
+ $(MAKE) -C .. text
doinstall: $(NAME)$(EXEC_EXT)
-mv $(INSTALLDIR)/$(NAME)$(EXEC_EXT) /tmp/$(NAME)-$(shell echo $$$$)
Modified: trunk/src/Makefile.OCaml
===================================================================
--- trunk/src/Makefile.OCaml 2008-07-05 17:43:33 UTC (rev 304)
+++ trunk/src/Makefile.OCaml 2008-08-21 15:39:05 UTC (rev 305)
@@ -29,9 +29,13 @@
else
ifeq ($(shell uname),OpenBSD)
OSARCH=OpenBSD
+else
+ifeq ($(shell uname),NetBSD)
+ OSARCH=NetBSD
endif
endif
endif
+endif
ETAGS=etags
endif
endif
@@ -261,6 +265,10 @@
endif
endif
+ifeq ($(OSARCH), NetBSD)
+ OCAMLDOT=false
+endif
+
# Rebuild dependencies (must be invoked manually)
.PHONY: depend
depend::
Modified: trunk/src/NEWS
===================================================================
--- trunk/src/NEWS 2008-07-05 17:43:33 UTC (rev 304)
+++ trunk/src/NEWS 2008-08-21 15:39:05 UTC (rev 305)
@@ -1,5 +1,5 @@
-Changes in Version 2.30.4
+Changes in Version 2.31.3
Changes since 2.17:
* Major rewrite and cleanup of the whole Mac OS X graphical user
Modified: trunk/src/RECENTNEWS
===================================================================
--- trunk/src/RECENTNEWS 2008-07-05 17:43:33 UTC (rev 304)
+++ trunk/src/RECENTNEWS 2008-08-21 15:39:05 UTC (rev 305)
@@ -1,3 +1,12 @@
+CHANGES FROM VERSION 2.31.3
+
+* Updated documentation with recently added preferences.
+
+* Applied patch from Antoine Reilles for NetBSD compilation
+
+* Makefile tidying
+
+-------------------------------
CHANGES FROM VERSION 2.31.2
* Added a bit of debugging code for Alan.
Modified: trunk/src/TODO.txt
===================================================================
--- trunk/src/TODO.txt 2008-07-05 17:43:33 UTC (rev 304)
+++ trunk/src/TODO.txt 2008-08-21 15:39:05 UTC (rev 305)
@@ -30,7 +30,29 @@
* See if there are other hacks that should be propagated to 2.27 (the
directory transfer throttle for sure!), and Jerome's recent suggested fix
+* Rsync debugging
+ - R can't run with debugging (even in 2.13) -- Alan cannot reproduce
+
+ - when using socket mode under windows, upon completion of the first
+ external rsync call, the connection to the server is dropped (the
+ server gets an EOF and closes the connection; the client sees a
+ broken connection)
+
+ - only with rsync, not scp
+ - only with socket mode connection by Unison, not ssh mode
+ - seems to have nothing to do with ssh tunneling
+
+ - calling Unix.open_process_in instead of
+ Lwt_unix.open_process_full seems to make no difference
+
+ - one difference we can see is that, at the end of the transfer,
+ the ssh started by rsync (when run with with -v -v) says
+ something like "FD1 clearing O_NONBLOCK". The similar call to
+ ssh from scp does not print this.
+
+ We're running under Cygwin (which is needed to have rsync)
+
###########################################################################
SOON
Modified: trunk/src/copy.ml
===================================================================
--- trunk/src/copy.ml 2008-07-05 17:43:33 UTC (rev 304)
+++ trunk/src/copy.ml 2008-08-21 15:39:05 UTC (rev 305)
@@ -529,7 +529,7 @@
"!add quotes to remote file name for copyprog (true/false/default)"
("When set to {\\tt true}, this flag causes Unison to add an extra layer "
^ "of quotes to the remote path passed to the external copy program. "
- ^ "This is needed by rsync, for example, which internal uses an ssh "
+ ^ "This is needed by rsync, for example, which internally uses an ssh "
^ "connection requiring an extra level of quoting for paths containing "
^ "spaces. When this flag is set to {\\tt default}, extra quotes are "
^ "added if the value of {\\tt copyprog} contains the string "
Modified: trunk/src/lwt/Makefile
===================================================================
--- trunk/src/lwt/Makefile 2008-07-05 17:43:33 UTC (rev 304)
+++ trunk/src/lwt/Makefile 2008-08-21 15:39:05 UTC (rev 305)
@@ -51,4 +51,4 @@
rm -f *.cmi *.cmo *.cmx *.cma *.cmxa *.a *.o *~ *.bak
clean::
- cd example && make clean
+ cd example && $(MAKE) clean
Modified: trunk/src/main.ml
===================================================================
--- trunk/src/main.ml 2008-07-05 17:43:33 UTC (rev 304)
+++ trunk/src/main.ml 2008-08-21 15:39:05 UTC (rev 305)
@@ -13,17 +13,17 @@
A non-functor interface is provided to allow the Mac GUI to reuse the
startup code for non-GUI options.
-*)
+ *)
(* ---------------------------------------------------------------------- *)
(* Some command-line arguments are handled specially during startup, e.g.,
- -doc
- -help
- -version
- -server
- -socket
- -ui
+ -doc
+ -help
+ -version
+ -server
+ -socket
+ -ui
They are expected to appear on the command-line only, not in a
profile. In particular, -version and -doc will print to the
standard output, so they only make sense if invoked from the
@@ -36,8 +36,8 @@
without loading a profile; and then we can't do command-line
parsing because it is intertwined with profile loading.
- NB: the Mac GUI handles these options and needs to change if they
- any more are added.
+ NB: the Mac GUI handles these options itself and needs to change
+ if any more are added.
*)
let versionPrefName = "version"
@@ -185,7 +185,7 @@
begin try
let i = List.hd (Util.StringMap.find socketPrefName argv) in
catch_all (fun () ->
- Os.createUnisonDir();
+ Os.createUnisonDir();
Remote.waitOnPort
(begin try
match Util.StringMap.find serverHostName argv with
Modified: trunk/src/mkProjectInfo.ml
===================================================================
--- trunk/src/mkProjectInfo.ml 2008-07-05 17:43:33 UTC (rev 304)
+++ trunk/src/mkProjectInfo.ml 2008-08-21 15:39:05 UTC (rev 305)
@@ -89,3 +89,4 @@
+
Modified: trunk/src/os.ml
===================================================================
--- trunk/src/os.ml 2008-07-05 17:43:33 UTC (rev 304)
+++ trunk/src/os.ml 2008-08-21 15:39:05 UTC (rev 305)
@@ -357,6 +357,7 @@
let readChannelTillEof c =
let rec loop lines =
try let l = input_line c in
+ (* Util.msg "%s\n" l; *)
loop (l::lines)
with End_of_file -> lines in
String.concat "\n" (Safelist.rev (loop []))
Modified: trunk/src/pty.c
===================================================================
--- trunk/src/pty.c 2008-07-05 17:43:33 UTC (rev 304)
+++ trunk/src/pty.c 2008-08-21 15:39:05 UTC (rev 305)
@@ -15,7 +15,7 @@
#define HAS_OPENPTY 1
#endif
-#ifdef __APPLE__
+#if defined(__APPLE__) || defined(__NetBSD__)
#include <util.h>
#define HAS_OPENPTY 1
#endif
Modified: trunk/src/strings.ml
===================================================================
--- trunk/src/strings.ml 2008-07-05 17:43:33 UTC (rev 304)
+++ trunk/src/strings.ml 2008-08-21 15:39:05 UTC (rev 305)
@@ -4,7 +4,7 @@
let docs =
("about", ("About Unison",
"Unison File Synchronizer\n\
- Version 2.30.4\n\
+ Version 2.31.3\n\
\n\
"))
::
@@ -1148,8 +1148,6 @@
\032-contactquietly suppress the 'contacting server' message during startup\n\
\032-copyprog xxx external program for copying large files\n\
\032-copyprogrest xxx variant of copyprog for resuming partial transfers\n\
- \032-copyquoterem xxx add quotes to remote file name for copyprog (true/false/def\n\
- ault)\n\
\032-copythreshold n use copyprog on files bigger than this (if >=0, in Kb)\n\
\032-debug xxx debug module xxx ('all' -> everything, 'verbose' -> more)\n\
\032-diff xxx command for showing differences between files\n\
@@ -1307,14 +1305,6 @@
\032 will just be copyprog with one extra option (e.g., -partial,\n\
\032 for rsync). The default setting invokes rsync with appropriate\n\
\032 options--most users should not need to change it.\n\
- \032 copyquoterem xxx\n\
- \032 When set to true, this flag causes Unison to add an extra layer\n\
- \032 of quotes to the remote path passed to the external copy\n\
- \032 program. This is needed by rsync, for example, which internal\n\
- \032 uses an ssh connection requiring an extra level of quoting for\n\
- \032 paths containing spaces. When this flag is set to default,\n\
- \032 extra quotes are added if the value of copyprog contains the\n\
- \032 string rsync.\n\
\032 copythreshold n\n\
\032 A number indicating above what filesize (in kilobytes) Unison\n\
\032 should use the external copying utility specified by copyprog.\n\
@@ -2346,16 +2336,16 @@
\n\
\032 Unison's built-in implementation of the rsync algorithm makes\n\
\032 transferring updates to existing files pretty fast. However, for\n\
- \032 whole-file copies, the built-in transfer method is not highly\n\
- \032 optimized. Also, if Unison is interrupted in the middle of\n\
- \032 transferring a large file, it will attempt to retransfer the whole\n\
+ \032 whole-file copies of newly created files, the built-in transfer method\n\
+ \032 is not highly optimized. Also, if Unison is interrupted in the middle\n\
+ \032 of transferring a large file, it will attempt to retransfer the whole\n\
\032 thing on the next run.\n\
\n\
- \032 These shortcomings can be addressed by telling Unison to use an\n\
- \032 external file copying utility for whole-file transfers. The\n\
- \032 recommended one is the standalone rsync tool, which is available by\n\
- \032 default on most Unix systems and can easily be installed on Windows\n\
- \032 systems using Cygwin.\n\
+ \032 These shortcomings can be addressed with a little extra work by\n\
+ \032 telling Unison to use an external file copying utility for whole-file\n\
+ \032 transfers. The recommended one is the standalone rsync tool, which is\n\
+ \032 available by default on most Unix systems and can easily be installed\n\
+ \032 on Windows systems using Cygwin.\n\
\n\
\032 If you have rsync installed on both hosts, you can make Unison use it\n\
\032 simply by setting the copythreshold flag to something non-negative. If\n\
@@ -2367,16 +2357,33 @@
\032 it to 1000 will cause the external tool to be used for all transfers\n\
\032 larger than a megabyte).\n\
\n\
- \032 If you want to use a different external copy utility, set the copyprog\n\
- \032 preference.\n\
+ \032 If you want to use a different external copy utility, set both the\n\
+ \032 copyprog and copyprogpartial preferences--the former is used for the\n\
+ \032 first transfer of a file, while the latter is used when Unison sees a\n\
+ \032 partially transferred temp file on the receiving host. Be careful\n\
+ \032 here: Your external tool needs to be instructed to copy files in place\n\
+ \032 (otherwise if the transfer is interrupted Unison will not notice that\n\
+ \032 some of the data has already been transferred, the next time it\n\
+ \032 tries). The default values are:\n\
+ \032 copyprog = rsync --inplace --compress\n\
+ \032 copyprogrest = rsync --partial --inplace --compress\n\
\n\
+ \032 You may also need to set the copyquoterem preference. When it is set\n\
+ \032 to true, this causes Unison to add an extra layer of quotes to the\n\
+ \032 remote path passed to the external copy program. This is is needed by\n\
+ \032 rsync, for example, which internally uses an ssh connection, requiring\n\
+ \032 an extra level of quoting for paths containing spaces. When this flag\n\
+ \032 is set to default, extra quotes are added if the value of copyprog\n\
+ \032 contains the string rsync. The default value is default, naturally.\n\
+ \n\
\032 If a directory transfer is interrupted, the next run of Unison will\n\
- \032 skip any files that were completely transferred before the\n\
- \032 interruption. Note, though, that the new directory will not appear in\n\
- \032 the destination filesystem until everything has been\n\
- \032 transferred--partially transferred directories are kept in a temporary\n\
- \032 location (with names like .unison.DIRNAME....) until the transfer is\n\
- \032 complete.\n\
+ \032 automatically skip any files that were completely transferred before\n\
+ \032 the interruption. (This behavior is always on: it does not depend on\n\
+ \032 the setting of the copythreshold preference.) Note, though, that the\n\
+ \032 new directory will not appear in the destination filesystem until\n\
+ \032 everything has been transferred--partially transferred directories are\n\
+ \032 kept in a temporary location (with names like .unison.DIRNAME....)\n\
+ \032 until the transfer is complete.\n\
\n\
Fast Update Detection\n\
\n\
@@ -2553,8 +2560,8 @@
\n\
"))
::
- ("news", ("Changes in Version 2.30.4",
- "Changes in Version 2.30.4\n\
+ ("news", ("Changes in Version 2.31.3",
+ "Changes in Version 2.31.3\n\
\n\
\032 Changes since 2.17:\n\
\032 * Major rewrite and cleanup of the whole Mac OS X graphical user\n\
Modified: trunk/src/update.ml
===================================================================
--- trunk/src/update.ml 2008-07-05 17:43:33 UTC (rev 304)
+++ trunk/src/update.ml 2008-08-21 15:39:05 UTC (rev 305)
@@ -1746,7 +1746,7 @@
let dig' = Os.fingerprint fspath path info in
let ress' = Osx.stamp info.Fileinfo.osX in
if dig' <> dig then begin
-(* if deleteBadTempFiles then Os.delete fspath path; *)
+ if deleteBadTempFiles then Os.delete fspath path;
raise (Util.Transient (Printf.sprintf
"The file %s was incorrectly transferred (fingerprint mismatch in %s)%s"
(Path.toString path)
More information about the Unison-hackers
mailing list