Engines and thread safety when using engine_next_reified

General
1

Given the following sample code:

test :-
  engine_create(_, runloop, E),
  forall(between(0,100,_), thread_create(worker(E), _, [detached(true)])).

runloop :- engine_fetch(test), engine_yield(ok), runloop.

worker(E) :- engine_post(E, test), engine_next_reified(E, Result), assertion(Result = the(ok)), worker(E).

Very soon almost all threads die with Thread running "worker(<engine>(3,0x600001630510))" died on exception: engine_post/2: No permission to post_to engine (3,0x600001630510)'`

Now when I change the two step engine_post/engine_next to engine_post/3, this seems to work, the access to the engine seems synchronized?

worker_b(E) :- engine_post(E, test, X), assertion(X == ok), worker_b(E).

I was under the impression that thread issuing first engine_post/2 would also obtain some kind of lock and others would wait. In any case, is there a way to get engine_post/3 behaviour but get the “reified” result, perhaps engine_post_reified/3?

2

Nobody says that the thread calling next must be the same as the one that did the post, so I guess this is to be expected. Would be nice if the docs are more explicit about what is safe and what not though …

Not sure about the general usability. engine_next_reified/2 is defined in Prolog on top of engine_next/2, so you can do the same for an engine_post_reified/3

3

Thanks for the hint. But is there something special about engine_post/3 that makes it thread-safe (in my scenario, meaning that post and next happen in a “transaction”)? Or is it just a side effect of the implementation?

4

Post and next do need to lock the engine and engine_post/3 does not release it in between.

5

How would I reproduce this behaviour using just engine_post/2 and engine_next, for example doing something in between the two while still having the engine locked for other threads’ engine_posts? Is there a simple way to “lock” engine? I guess creating a mutex and then using something like blackboard to associate the mutex, or even using blackboard itself to coordinate engine access would work…

6

A simple binary predicate that associates engines with mutexes and using these mutexes should work. Why would you do something in between though? Engines do not run concurrently. Instead, engine_next/2 temporarily replaces the current thread engine with the engine on which you do the next. When the answer is ready, the active engine is swapped back. In other words, an engine is a complete Prolog prover (stacks, virtual machine instance, etc.). A Prolog thread is an OS thread that is running an engine. SWI-Prolog allows engines to hop between OS threads (a bit like Pythons (<3.13) GIL, except that classical Python has only one engine and SWI-Prolog can handle many engines). Engines can also not be associated with any OS thread, which implies they cannot make progress.