Possible deadlock after updating to >8.4

Hello,

our project KnowRob doesn’t finish starting since updating SWI-Prolog from 8.24 to 8.4.

I, unfortunately, am stuck at the moment in finding the exact issue/a minimal example, but I could pinpoint the problem to this thread_get_message/3 call. I could also find the exact commit to swipl-devel with which the problem starts to occur: https://github.com/SWI-Prolog/swipl-devel/commit/5dca96d4f6a03e26d828e0e072fc6317ddd0ccad

We start KnowRob with the C Interface in our project rosprolog. I could observe that, if I remove the init calls that result in the thread_get_message call and call it after everything loaded, KnowRob will start up and run the init call without problems.

If someone has some how the commit could result in such a problem, it could help me greatly in further investigating the error.

Thanks,
Sascha Jongebloed

Thanks for the clear report. I doubt I can give a precise answer, but possibly a hint. The referred Prolog update runs compiling a file in signal atomic mode. This is needed to avoid signals interrupting loading a file. Ideally this should be possible, reverting anything that has already been done wrt. loading the file. This revert is (as it stands) practically impossible though. As a result though, goals started using initialization/1 (or as straight :- goal. run with signals disabled. Could it be you are running non-terminating goals using any of the above?

Two things that makes we wonder is the referred thread_get_message/3 call which uses timeout(0) and thus never blocks (should never block) and the fact that you seem to start using --signals=false and (thus) there is little signal handling anyway, although this flag doesn’t stop the alert signal that makes Prolog thread signals work while the receiver is in a blocking system call.

Hope this gives a few hints.

Thanks for your help. I will have a deeper look into it and report back if it is an issue with our code or swipl. At the moment I found a workaround (moving the code out of the initialization and calling it after the initialization).

Best, Sascha Jongebloed

1 Like

That is more or less what I expected. There are limits on what you can do during initialization as it may lead to deadlocks as a result of concurrent program loading. Actually, there are not that many good ways of using initialization/1. You should not “run your program” from such a hook unless it concerns a single file script. SWI-Prolog has initializiation(Goal,main) for that, which is executed after the entire program has been loaded. You could use it to initialize some state that is relevant to the module. In most cases there are better ways to do so though.

I do wonder why you inhibit signals in the init? Things have changed there. By default SWI-Prolog sets up crash handlers for the crash signals such as SEGV, sets up SIGINT for the debugger and a signal to unblock system calls. That is all. Old versions used two more signals to synchronise atom and clause garbage collection that could interfere with other software (notably a problem with Java).

I always thought that is exactly what it is for in ISO Prolog, since the standard doesn’t specify any other way to define an entry point into a program. It says:

A directive initialization(T) converts the term T to a goal G and includes it in a set of goals which shall be executed immediately after the Prolog text has been prepared for execution. The order in which any such goals will be executed shall be implementation defined.

I suppose you are interpreting “Prolog text” as a single file, but the standard also uses it to denote the whole program, e.g.

A directive ensure_loaded(P_text) specifies that the Prolog text being prepared for execution shall include the Prolog text denoted by P_text […]

Sicstus documentarion says (emphasis mine):

Declares that Goal is to be run when the file in which the declaration appears is loaded into a running system, or when a stand-alone program or runtime system that contains the file is started up.

Is initialization/1 not meant to be used like main() in C?

Its hard for me to say how it is meant to be :slight_smile: AFAIK, all implementations execute the goals in the order they appear after loading a file and a stand-alone program shall normally execute them in the same order as during loading but with the entire program being loaded. I (now) understand ISO doesn’t demand this ordering.

First of all, this means there is not just a single main(), but potentially a lot of them. This is a bit problematic. What if one doesn’t complete?

Another way to see them is as their name suggests: initialization steps that initialize some state that cannot be done statically. Compare to shared object linking where some defined entry point is called when the object it attached and detached from the process. This is exploited by e.g. C++ global constructors (if that is the right term). I guess there is no way to enforce, but these constructors are supposed to initialize things at runtime that cannot be done statically. For Prolog you can use it to generate some predicates at runtime, load or initialize some foreign resource, etc. The first can in most Prolog systems also be achieved using term_expansion/2, which has the advantage that the data is already precomputed in stand alone programs. The second is often better done lazily.

Another problem of initialization/1 is what needs to happen on dynamic reloading of the file in which they appear during development? Just re-running is often problematic (resource it needs to create already exists, etc.)

SWI-Prolog complicates these issues by providing lazy and concurrent loading of source files. For that reason it has initialization/2 for a while, where the second argument indicates when the goal should be executed. So far the available options cover nicely the use cases.

1 Like

Just thinking out loud.

This reminds me of the JavaScript promise which many other languages have in some form.