And the access time shouldn’t go down with more clauses. Variations could be due to different hash distribution.
Some of these libraries you can simply try using LD_PRELOAD= (on Linux). I don’t really think there is a one size fits all. It all depends on the workload and different Prolog programs can exhibit quite different workloads wrt. malloc(). There are a lot of aspects on which malloc implementations can be optimized.
That said, multi-threaded SWI-Prolog does seem to suffer from poor memory reuse, at least when we look at the SWISH stats page. I’m still not 100% sure whether these stats are correct. There might be a problem because the various worker threads (Pengines and HTTP) do the allocations of (notably) atoms and clauses while the gc thread does all the deallocations. I’ve been doing some investigation work on this, but it is fairly hard to figure out what is going on. The SWI-Prolog site suffers from similar issues. Ideas are welcome. I’m particularly interested to figure out where not reused memory was allocated for and with which size distribution.