[Unison-hackers] Memory exhaustion issue (#1068)

[Tõivo Leedjärv]

On Sun, 24 Nov 2024 at 12:48, Greg Troxel <gdt at lexort.com> wrote:
>
> It is probably a good idea to force gc when a unison server finishes
> servicing a request and is going to sleep.  I don't think it's good to
> poke gc all the time; there are reasons it was designed how it was.

All true. We definitely want to let the GC do its thing. Just about
the only case would be exactly this one, when we know a major activity
has finished and there is no more activity coming up to trigger GC.

We could consider a patch like this if Michael's testing shows that it
could make a real difference for his case (not just -20 MB!).
Otherwise, no.

> Cleaning up before sleeping sounds good and I don't expect big
> downsides.  There will be some memory references that might provoke
> paging, but that's more or less taking the pain earlier.
>
> This could be related to:
>
>   https://urldefense.com/v3/__https://github.com/bcpierce00/unison/issues/399__;!!IBzWLUs!TxdPcz379DjA7uWzp_BrNk6IbY5qmGxRAu8wVRNFz6fbTFUF-ERI2VYe65RsmQ2OHkf_9X4xWVKQhnhRMeUu8vnwcxs$ 

Ah! Thank you for reminding me of this. My suspicion is indeed that
the real issue Michael is facing is not the copy/rsync but the holding
the entire archive in the memory. And forcing GC at end of sync will
not solve that because the archive is entirely cached.

We have a long time ago discussed if/how we could improve this cache
but I don't remember the details of those discussions. In any case,
I'd like to wait for some more results from Michael to pinpoint what
exactly is consuming the memory.