Tabling: memory not being returned to the OS?

Help!
1

With this program:

:- table words/2.

words([],0).
words([W|Ws],N) :-
    N #> 0,
    N1 #= N - 1,
    word(W),
    words(Ws,N1).

word("hi").
word("ho").

table_mem(Bytes) :-
    aggregate_all(sum(Bytes1),
                  (  current_table(_,Table),
                     trie_property(Table,size(Bytes1))
		  ),
		  Bytes).

And the following queries:

2 ?- table_mem(Bytes).
Bytes = 0.

3 ?- shell("free -h").
              total        used        free      shared  buff/cache   available
Mem:           23Gi       4.6Gi        10Gi       602Mi       8.2Gi        17Gi
Swap:         4.0Gi          0B       4.0Gi
true.

4 ?- words(W,20),fail.
false.

5 ?- table_mem(Bytes).
Bytes = 788527096.

6 ?- shell("free -h").
              total        used        free      shared  buff/cache   available
Mem:           23Gi       5.6Gi       9.6Gi       602Mi       8.2Gi        16Gi
Swap:         4.0Gi          0B       4.0Gi
true.

7 ?- abolish_all_tables.
true.

8 ?- table_mem(Bytes).
Bytes = 0.

9 ?- shell("free -h").
              total        used        free      shared  buff/cache   available
Mem:           23Gi       5.6Gi       9.6Gi       602Mi       8.2Gi        16Gi
Swap:         4.0Gi          0B       4.0Gi
true.

abolish_all_tables is not returning all the memory to the OS. The table memory seems to be realeased (table_mem(0) is true), but it doesn’t seem to be returned to the OS. Am I missing something?

P.S. This is with the latest commit including dynamic tabling (8.1.8-128-g076e59f24).

2

This is the normal situation as currently all memory is allocated using malloc(). GNU malloc (and probably most modern malloc implementation) only return large chunks back to the OS, but the table space is fragmented in many small allocations. At some point this will probably be enhanced by allocating small related chunks from a larger pool such that they are freed together and be returned to the OS. For now there are other priorities wrt. tabling :slight_smile:

Anyone interested in this topic is invited to help. Similar problems are related to atom and predicate allocation (clauses and indexes). Does require a good deal of understanding C and memory allocation :slight_smile:

3

I see, thanks, that explains it.

Great! with dynamic tabling working, what is the next target?

4

Probably view consistency for incremental tabling, which relates to enumeration of modifying tables (tries) which relates to the fact that enumerating from tries is far too slow. For short, some redesign of the trie implementation. The YAP community has done some interesting work here. Studying that is probably the best start.