Thanks. I’ve spent some time figuring out when the server of our website and swish tend to grow pretty much forever. To do so I’ve replayed a couple of days worth of log data against a local copy of the website server using 4 concurrent clients as quick as the server wants to handle the requests. This gives some interesting insights.
- Running under heaptrack, the server is claimed to be using 200Mb at the end of the run. After some initial growing it also stays pretty stable. So there are no serious memory leaks in the conventional meaning.
- Using glibc ptmalloc implementation it uses at the end of the run 1.3Gb RSS and 16.3 Gb virtual memory.
- Using tcmalloc (v4.3.0) it uses at the end 400Mb RSS and 680Mb virtual memory.
- Using jemalloc (v3) it uses at the end 630Mb RSS and 5Gb virtual memory.
An interesting comparison of the performance of these allocators is here. This isn’t very interesting to us. Unlike C++. very few Prolog programs depend critically on malloc() performance. Most of SWI-Prolog’s fine grained heap memory management is either not very time critical (e.g., allocating a clause) or SWI-Prolog allocates a larger buffer and allocates from that (for example findall/3), such that it reduces stress on malloc() and can release memory to the OS for large findall/3 answer sets. The current tabling implementation is an exception, but this too should be handled in a more clever way such that table space is cheaper and can be handed to the OS.
I’m going to have a look in exploiting tcmalloc(), possible also providing some of its advanced features.