<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">On 23/11/2024 23:01, Greg Troxel wrote:<br>
</div>
<blockquote type="cite" cite="mid:rmiplmlwtoa.fsf@s1.lexort.com">
<pre wrap="" class="moz-quote-pre">Michael von Glasow <a class="moz-txt-link-rfc2396E" href="mailto:michael@vonglasow.com"><michael@vonglasow.com></a> writes:
</pre>
<blockquote type="cite">
<pre wrap="" class="moz-quote-pre">Switching profiles should be sufficient – this created a new unison
process on the server, while the old one gradually freed up its memory
(but kept running).
</pre>
</blockquote>
<pre wrap="" class="moz-quote-pre">
Sorry, I guess I was unclear. I am not really looking for just
sufficient to make your case, but the simplest possible way to reproduce
the problem, expressed programmatically so that others can run it (after
reading the code to feel it is safe). So that's no GUI, no persistent
server, and everything in the profile to be synced created by the
script.</pre>
</blockquote>
I‘ll look into it, but reproducing would also involve creating a
huge set of files to sync.<br>
<blockquote type="cite" cite="mid:rmiplmlwtoa.fsf@s1.lexort.com">
<pre wrap="" class="moz-quote-pre">For comparing with other people, I think it will be more useful to talk
about KB or MB of memory usage vs %. Lots of people will have different
amounts of RAM and other loads, although 1 GB such as RPI3 is pretty
common.</pre>
</blockquote>
In principle, +1. However, the tools I have spit out percentages.
One % is roughly 10 MB, so the math is fairly easy.<br>
<blockquote type="cite" cite="mid:rmiplmlwtoa.fsf@s1.lexort.com">
<pre wrap="" class="moz-quote-pre">I personally am not interested in debugging anything older than 2.53.7
(or 6, but there is no reason to use 6 if you are compiling).</pre>
</blockquote>
You do you, but the trouble is that distro repositories are a bit
slow to catch up. Part of the whole update exercise was so I could
finally return to Unison from the distro repos and not have to build
my own any more. Especially since building for non-Intel platforms
is quite a hassle – I tried to get that on CI some time ago, but my
attempts stalled as setting up a cross-compilation environment
proved challenging.<br>
<blockquote type="cite" cite="mid:rmiplmlwtoa.fsf@s1.lexort.com">
<pre wrap="" class="moz-quote-pre">So it sort of sounds like memory is allocated proportional to the size
of the transferred file, and not freed.
And, that other memory uesd for scan/etc. is reused.</pre>
</blockquote>
It is roughly proportional to the size of the transferred file,
although 16000M uses more in comparison than 12800M does (roughly
55M vs. 34M). Also, archive file size seems to matter.<span
style="white-space: pre-wrap"> </span>
<blockquote type="cite" cite="mid:rmiplmlwtoa.fsf@s1.lexort.com">
<blockquote type="cite">
<pre wrap="" class="moz-quote-pre">Looking at the docs, what comes to mind is:
- copyprog, copyprogthreshold (use external program <copyprog> for
copying files larger than <copyprogthreshold> kB)
</pre>
</blockquote>
<pre wrap="" class="moz-quote-pre">
That is about to be deleted.</pre>
</blockquote>
That would be too bad, because that seems to be a valid workaround.
I added `copythreshold = 163840` (1% of 16G, roughly where the
issues started) to my profile and was able to sync.<br>
<blockquote type="cite" cite="mid:rmiplmlwtoa.fsf@s1.lexort.com">
<blockquote type="cite">
<pre wrap="" class="moz-quote-pre">Or is there a way to tell Unison to stop being smart and just copy the
damn thing (which is presumably less memory-hungry) if a file is larger
than a certain size?
</pre>
</blockquote>
<pre wrap="" class="moz-quote-pre">
I don't think so, but really that should not be necessary. If there is
code that uses memory when it shouldn't, we should find that and fix it.</pre>
</blockquote>
For “cut the smartness above file size X”, `copythreshold` (not
`copyprogthreshold`, that was a typo) seems to do the trick. Please
keep that feature until memory efficiency is improved, unless there
is an alternative.
</body>
</html>