Overhead of concurrent_maplist

General
1

Hi!

The manual page of concurrent_maplist says:

Note that the the overhead of this predicate is considerable and therefore Goal must be fairly expensive before one reaches a speedup.

What do you mean with “considerable” and “fairly expensive”?

We have goals that take from 60 to 110 seconds when executed sequentially. Are these fairly expensive to be run concurrently using concurrent_maplist?

I mean, say p(x) takes 60 seconds to complete and p(y) takes 110 seconds, if I do:

concurrent_maplist(p,[x,y])

would it take nearly 110 seconds to finish (in a two or more core machine)?

I guess it should but maybe the overhead of concurrent_maplist is much higher than I think.

Is it possible that it takes 120 (i.e. the overhead is around 10 second for a 2 element list)? If I have an 8 element list of goals taking from 60 to 230, is it possible that concurrent_maplist takes around 400 seconds in a 8 core machine?

Thanks in advance!

Maxi

2

Here’s the overhead for concurrent_maplist/2 with 100 and 1000 items on my 4-core machine:

[user].
|: m(Xs,X) :- member(X,Xs).
|: 

?- findall(X,between(1,100,X),Xs), time(maplist(m(Xs), Xs)).
% 5,350 inferences, 0.001 CPU in 0.001 seconds (99% CPU, 8248675 Lips)
Xs = [1, 2, 3, 4, 5, 6, 7, 8, 9|...] 

?- findall(X,between(1,100,X),Xs), time(concurrent_maplist(m(Xs), Xs)).
% 17,901 inferences, 0.024 CPU in 0.011 seconds (226% CPU, 733731 Lips)
Xs = [1, 2, 3, 4, 5, 6, 7, 8, 9|...].

?- findall(X,between(1,1000,X),Xs), time(concurrent_maplist(m(Xs), Xs)).
% 525,697 inferences, 0.253 CPU in 0.105 seconds (241% CPU, 2075994 Lips)
Xs = [1, 2, 3, 4, 5, 6, 7, 8, 9|...].
1 Like
3

Thanks Peter! Good way of measuring the overhead. The overhead is in the order of magnitude I thought it would be… so our problem is in some other place.