Preallocating storage for assoc

Hello,

I’d like to try to use an assoc instead of the fact store, to store a graph structure. I am wondering, is it possible to preallocate more memory space for an assoc – so that memory allocation isn’t performed piece by piece during the construction of a larger graph.

thank you,

Dan

It’s not clear that you’ll gain anything. Here’s a fairly succinct description of how most memory allocators work:

A typical allocator implementation will first call the operating system to get huge block of memory, and then to satisfy your request it will give you a piece of that memory, this is known as suballocation . If it runs out of memory, it will get more from the operating system.

The allocator must keep track of both all the big blocks it got from the operating system and also all the small blocks it handed out to its clients. It also must accept blocks back from clients.

A typical suballocation algorithm keeps a list of returned blocks of each size called a freelist and always tries to satisfy a request from the freelist, only going to the main block if the freelist is empty. This particular implementation technique is extremely fast and quite efficient for average programs, though it has woeful fragmentation properties if request sizes are all over the place (which is not usual for most programs).

Modern allocators like GNU’s malloc implementation are complex, but have been built with many decades of experience and should be considered so good that it is very rare to need to write your own specialised suballocator.

Hi Peter,

Thank you for your comments - this is indeed very interesting;

Dan

The assoc data lives on the Prolog (global) stack. That is managed using malloc/realloc as a big chunk, which typically means it is a memory mapped region. The biggest price you pay is Prolog garbage collection.

Malloc() doesn’t always do a good job. Notable SWISH seems to suffer from poor reuse of memory. See https://swish.swi-prolog.org/example/stats.swinb, last chart. I’m still looking for tools that can tell me what happens …

1 Like