Memory usage

Help!
24

What is the difference between concurrent_and/2 and concurrent_forall/2? From the documentation, the only difference seems to be that concurrent_and/2 puts the generator in a separate thread but concurrent_forall/2 doesn’t – it’s not clear to me why that would be of any benefit because the main thread would just be waiting for termination of all the workers anyway.

Is there a way to gather the results from concurrent_and/2? I presume assert/1 or similar could be used; but something like concurrent_bagof/3 would seem to get the effect of concurrent_maplist/2 but with less memory usage (assuming the Goal in maplist is transformed to a generator).

25

Consider this program:

a(1).
a(2).
b(1).
b(2).

Now

?- concurrent_and(a(X),b(X)).
X = 1 ;
X = 2 ;
false.

?- concurrent_forall(a(X),b(X)).
true.
1 Like
26

Interesting thought. Probably concurrent_findall/3 as you can (and have to) layer variable preservation on top anyway. Roughly, this seems what @friguzzi is doing by asserting the successes. The alternative would be to send the successes back using a queue and create a list from that.

28

Some more comments: the big time reduction comes from the management of the set of rules, rather than from concurrent_forall/2 vs concurrent_maplist/2.
In fact, with these two predicates:

main:-
  findall(rank(G,A,Ans),i(G,A,Ans),TestAtoms),
  out(R00),
  maplist(generate_cl,R00),
  concurrent_maplist(rank_ans,TestAtoms).

and

main:-
  out(R00),
  maplist(generate_cl,R00),
  concurrent_forall(i(G,A,Ans),rank_ans(rank(G,A,Ans))).

I get the same timing.

29

By comparison, the approach that was much slower and memory hungry was this one:

main:-
  findall(rank(G,A,Ans),i(G,A,Ans),TestAtoms),
  out(R00),
  maplist(generate_cl,R00,Prog),
  concurrent_maplist(rank_ans(Prog),TestAtoms).

where Prog is a list containing several thousand terms.

30

This has been proposed and there’s some discussion: Suggestion: concurrent_findall - #11 by EricGT

1 Like
31

Memory … In retrospect, the gain is probably marginal compared to findall/3 over concurrent_and/2. It merely avoids one copy for each result.