# Multi-threading when using swi prolog as embeded engine **URL:** \ **Category:** Help!\ **Tags:** how-to\ **Created:** [March 31, 2020, 10:12pm UTC](https://swi-prolog.discourse.group/t/multi-threading-when-using-swi-prolog-as-embeded-engine/2085 "2020-03-31T22:12:36Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![freedafeng](https://avatars.discourse-cdn.com/v4/letter/f/ecc23a/32.png) [@freedafeng](https://swi-prolog.discourse.group/u/freedafeng)\ **Post date:** [March 31, 2020, 10:12pm UTC](https://swi-prolog.discourse.group/t/multi-threading-when-using-swi-prolog-as-embeded-engine/2085/1 "2020-03-31T22:12:36Z")
I’m using: SWI-Prolog version 8.03. We need to run our application in parallel. We used “Multithreaded mixed C and Prolog applications” – [https://www.swi-prolog.org/pldoc/man?section=foreignthread](https://www.swi-prolog.org/pldoc/man?section=foreignthread). I want the code to share some big relations across the threads. Meanwhile, we also need to create smaller and unique relations in each thread. Each thread needs to run the queries against all the relations, small and big. I tried to allocate threads through C (Python actually, but under the hood, it’s the same). I then assign each thread a separate prolog engine using PL\_thread\_attach\_engine(NULL) function. However, when I run the query, I always get the “nested query” error - “NestedQueryError: The last query was not closed”. Is it possible to do this? TIA. Jackson ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [April 1, 2020, 6:59am UTC](https://swi-prolog.discourse.group/t/multi-threading-when-using-swi-prolog-as-embeded-engine/2085/2 "2020-04-01T06:59:31Z")
SWI-Prolog can definitely do this. The _NestedQueryError_ should come from the Python interface as this term doesn’t exist in the SWI-Prolog source. This smells like the Python interface is using global variables. Note that where SWI-Prolog is fully multi-threaded, Python is a bit half way, using a central lock. At least that was the situation when I asked one of the early developers of Python about locking requirements when calling Python from C. ---
**Author:** ![freedafeng](https://avatars.discourse-cdn.com/v4/letter/f/ecc23a/32.png) [@freedafeng](https://swi-prolog.discourse.group/u/freedafeng)\ **Post date:** [April 1, 2020, 4:31pm UTC](https://swi-prolog.discourse.group/t/multi-threading-when-using-swi-prolog-as-embeded-engine/2085/3 "2020-04-01T16:31:51Z")
Thanks Jan! You are right. I found the central lock in the python interface and disabled it for now. Everything works perfectly now. Please let me know if I do need to add some mutex to the application to coordinate the prolog engines. Otherwise, all is good. Thanks! ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [April 1, 2020, 8:37pm UTC](https://swi-prolog.discourse.group/t/multi-threading-when-using-swi-prolog-as-embeded-engine/2085/4 "2020-04-01T20:37:54Z")
Prolog engines should be fine. I’d be more worried about disabling the Python global lock. You might consider running Prolog as a web server using either plain HTTP, plain sockets or Pengines. All have their pros and cons. Separating Prolog has mostly cons except if you cannot tolerate the network latency. ---
**Author:** ![peter.ludemann](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/peter.ludemann/32/48_2.png) [@peter.ludemann](https://swi-prolog.discourse.group/u/peter.ludemann)\ **Post date:** [April 1, 2020, 9:38pm UTC](https://swi-prolog.discourse.group/t/multi-threading-when-using-swi-prolog-as-embeded-engine/2085/5 "2020-04-01T21:38:30Z")
> [@freedafeng](#): > > I found the central lock in the python interface and disabled it for now. Is this the Python Global Interpreter Lock (GIL) or something specific for your Python interface? ---
**Author:** ![freedafeng](https://avatars.discourse-cdn.com/v4/letter/f/ecc23a/32.png) [@freedafeng](https://swi-prolog.discourse.group/u/freedafeng)\ **Post date:** [April 1, 2020, 10:10pm UTC](https://swi-prolog.discourse.group/t/multi-threading-when-using-swi-prolog-as-embeded-engine/2085/6 "2020-04-01T22:10:13Z")
No. It’s not the GIL, but a specific thing in this specific python wrapper. ---
**Author:** ![freedafeng](https://avatars.discourse-cdn.com/v4/letter/f/ecc23a/32.png) [@freedafeng](https://swi-prolog.discourse.group/u/freedafeng)\ **Post date:** [April 1, 2020, 10:17pm UTC](https://swi-prolog.discourse.group/t/multi-threading-when-using-swi-prolog-as-embeded-engine/2085/7 "2020-04-01T22:17:46Z")
Thanks Jan for the advice. To be more informative to the people in similar situation, I am using pyswip, a python wrapper(bridge) of swi-prolog. Pyswip is based on ctypes. I made some changes to pyswip to add the support to attach a prolog engine to each thread. For the “nested query” problem I was talking about, I disabled the class variable of Prolog. It’s not the GIL. Disabling GIL even temporarily is too risky. ---
**Author:** ![michaelby](https://avatars.discourse-cdn.com/v4/letter/m/b3f665/32.png) [@michaelby](https://swi-prolog.discourse.group/u/michaelby)\ **Post date:** [April 2, 2020, 9:57am UTC](https://swi-prolog.discourse.group/t/multi-threading-when-using-swi-prolog-as-embeded-engine/2085/8 "2020-04-02T09:57:01Z")
There may be a good reason why `pyswip` has a lock to enforce single-threaded Prolog querying. (I ran into a problem with an earlier pyswip version once when I tried to use it with Python `multiprocessing`.) To be safe, you might want to pass goals to Prolog asynchronously and have Prolog create threads and send goals to them through message queues, for example. When a result gets sent back, your main Prolog thread can call back into Python with the result. In that way `pyswip` is only “aware” of a single thread. ---
**Author:** ![freedafeng](https://avatars.discourse-cdn.com/v4/letter/f/ecc23a/32.png) [@freedafeng](https://swi-prolog.discourse.group/u/freedafeng)\ **Post date:** [April 2, 2020, 3:44pm UTC](https://swi-prolog.discourse.group/t/multi-threading-when-using-swi-prolog-as-embeded-engine/2085/9 "2020-04-02T15:44:26Z")
Thanks for the info. I admit that I haven’t tested my solution thoroughly. There might be some pitfalls that I am not aware of. I will have to wait for that to happen before I can address them. I am pretty sure multi-processing is totally different from my multi-threading approach. Processes don’t share any data (unless you specifically let them), while threads do. So the problem that you ran into maybe different. However, without more details I can not say for sure. I don’t know how to use swi prolog’s C interface to let prolog create threads, except that writing special directives in prolog program and then use “consult” or sth like that. I will dig deeper if I can find more info about that. So far I have been using python to create threads, which I believe is the same as using C to create OS threads. ---
**Author:** ![ferdasonmez](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/ferdasonmez/32/3578_2.png) [@ferdasonmez](https://swi-prolog.discourse.group/u/ferdasonmez)\ **Post date:** [June 22, 2021, 8:44am UTC](https://swi-prolog.discourse.group/t/multi-threading-when-using-swi-prolog-as-embeded-engine/2085/10 "2021-06-22T08:44:57Z")
Hi, @freedafeng can you please share a few lines of code? I am also trying to modiy the prolog.py file to not to get nested query error. Which lines did you commented/changed? I have been trying to solve this problem for a few days. This is very urgent to me. I would appreciate if you can help. Many thanks in advance… ---
**Author:** ![ferdasonmez](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/ferdasonmez/32/3578_2.png) [@ferdasonmez](https://swi-prolog.discourse.group/u/ferdasonmez)\ **Post date:** [June 22, 2021, 8:54am UTC](https://swi-prolog.discourse.group/t/multi-threading-when-using-swi-prolog-as-embeded-engine/2085/11 "2021-06-22T08:54:19Z")
> [@jan](#): > > _NestedQueryError_ Hello Jan, Can you please clarify how to fix this problem? Many thanks ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [June 22, 2021, 9:19am UTC](https://swi-prolog.discourse.group/t/multi-threading-when-using-swi-prolog-as-embeded-engine/2085/12 "2021-06-22T09:19:56Z")
> [@ferdasonmez](#): > > Can you please clarify how to fix this problem? I’m afraid not. I have nothing to do with the Python interface and merely explained the error is not from Prolog (and thus from the interface). Hope one of the other can clarify.