I’m new to this forum and have no idea what most of the thread titles mean. Of course the overall completion time for my Prolog program depends on what I put in it. But I’m surprised that it only uses 13% of the CPU capacity when nothing else is running. Could this have something to do with my workstation having two Xeon chips installed?
What are you using to measure CPU utilization? And on what system (Linux, Windows, something else?)
Some tools show percentage of a single CPU; some show a percentage of all the CPUs.
(FWIW, I can get 100% utilization of all 4 CPUs on my Ubuntu system, using the SWI-Prolog threads library … this shows as 400% using top(1) on Ubuntu 18.04.)
1 Like
Thanks. I’ll have to look into that threads library. I’m using Windows 7. System can’t be upgraded to Windows 10, and 7 no longer gets security updates, so this interaction is on my little laptop and my workstation stays off the web…
I used task manager for the 13% figure.
The easiest ways to use threads are
concurrent_maplist/2, concurrent_maplist/3, etc.
concurrent_forall/2
By default, Windows task manager shows usage for all cores. You can right-click on the graph and display “logical processors”.
SWI-Prolog, in general, doesn’t use multiple threads unless you explicitly use threads.
1 Like
Thanks again for your help. I am finding the documentation of concurrent_maplist/2 to be pretty thoroughly opaque. Goals is a list of callable terms and callable terms are bound to atoms… I know what an atom is, but not what is bound to it.
Can I just use a predicate as a callable term? Most of my work is in basic list decapitation with comparisons of multiple lists which may be a few hundred items long, all in failure driven loops. I was hoping Prolog could just split up the lists and then pull the results together when done, but I don’t see how to get there from here.
I’m sorry. Things always become clearer right after I leave a post. I will plan on partitioning my lists, running concurrent tasks on the pieces and then putting them back together. This should work on the predicates that only look at the first element. I don’t know about the ones that use the first two.
I still don’t understand the manual pages.
If you can convert your code to use forall/2 or maplist/3 (or similar), then you can easily switch to using the concurrent versions of those, provided your code has no side-effects such as assert or retract.
For example, if your code is something like this:
process :-
generate_value(X),
process_value(X),
fail
; true.
you can convert it to:
process :-
forall(generate_value(X),
process_value(X)).
and thence to:
process :-
concurrent_forall(
generate_value(X),
process_value(X)).
(depending on your situation, concurrent_and/2 might be better).
Or you can convert it to:
process :-
bagof(X, generate_value(X), Xs),
maplist(process_value, Xs).
and thence to:
process :-
bagof(X, generate_value(X), Xs),
concurrent_maplist(process_value, Xs).
There is some overhead in setting up the concurrency queues, so you might or might not see an overall speedup. (For some of my code, where I was processing millions of items, the speed-up was almost 4x with 4 CPUs.)
1 Like
My question is what causes a 13% CPU usage in the first place? Anything that is single threaded and CPU bound should get 100% or a little more due to concurrent garbage collection. If it doesn’t get 100%, it typically waits for something, often I/O. Could that be the case?
Hi!
This 13% looks like 1/8 to me. A processor with eight cores and that prolog program using one. Does the system monitor program count the CPU as 100% or as 800%, int the latter case meaning that each core counts as 100%?
bye
Volker
1 Like
Right-click on the “performance” graph and select “Logical processors” (default is “Overall utilization”). You should see “N” graphs, where N is the number of logical CPUs. You might also want to turn on “Show kernel times”.
That system (not the laptop I am using for this reply) is not connected to the internet (or any network). The Prolog program gets its input from the command line and writes results to a file. There is nothing other than essential system tasks running on the system.
I’ll check that the next time I have that system in use. It has two Xeon chips with 4 cores each. I’ve also thought that 13% looks a lot like 1/8.
I have converted my failure driven loop to forall/2. That works, and I get the same results that I did in the failure driven version (which is good). When I change to concurrent_forall/2, it says the predicate concurrent_forall/2 is not defined. I added “use_module(library(thread)),” at the top, but that didn’t help.
If I enter “use-module(library(thread)).” at the ? prompt, it says there is no such library. Do I need to download something? I can’t find that in the documentation so far.
Quick guess, you are on an older version of SWI-Prolog. Even a stable version is to old as concurrent_forall/2 is relatively new.
See: SWI-Prolog 8.3.3
Thanks, that was exactly it.
I tried to play it safe by always loading the stable build, but that was too old.
Now, all I have to do is figure out why it is doing nothing to make my code run faster.
In the documentation of forall, the second argument is listed as a side effect. That works fine, my side effect is just writing the current solution to a file.
In the documentation for concurrent_forall, it is stated that the syntax is the same as forall, but then it shows the second argument to be a test. That has me confused.
In Prolog there really isn’t a difference between an “Action” and a “Test”, unless it’s explicitly stated that the success/failure of the “Action” is ignored (e.g.: Action->true;true). But the documentation is confusing. @jan - is there a reason for having “Action” for forall/2 and “Test” for concurrent_forall/2?
In my experience, the “development” build is very reliable; if there’s a problem, it’s usually reported and fixed within a day of the release. (I use the PPA to automatically upgrade under Ubuntu.)
Yes, by now I am also a firm believer in a “rolling release” model for open source software. Any bugs still present in the latest development version are almost certainly present in any older “stable” release. @jan has so far been careful enough to leave experimental changes on branches and only merge once they are ripe. The occasional bugs coming from new features haven’t caused any memorable regressions in existing features. Most importantly, the more users have the latest development release, the sooner any problems are spotted and fixed.
Bottom line: for open source, a rolling release model is a pragmatic choice.
2 Likes
No. Pushed a patch to the concurrent_forall/2 docs to use Action.
1 Like
Now that I am taking the doc. at its word (necessitated by my ignorance of the shared meaning of test and action), I can make my program run with concurrent_forall/3, with the condition [threads(4)]. I expected this to start using additional threads, but my CPU usage has not changed. Nor, has the completion time for my program. I’m wondering if I need to put the concurrency further down, after some things are already instantiated?