Pengines: inconsistent series of events received for sequence of ask requests to the same pengine

I started a local Pengines server with an ancestorsapp defined as in

I then load the script test_client.plwith the code below and make the query `test_genealogist_queries`.

The trace I get shows that

  1. For all the pengine_ask queries except the last one, I get a series of success events where all** of them have More = true** (including the last success event for the query). The very last query is the only one for which I get a last success event with More = false.
  2. For the first pengine_ask query, (q1 in predicate test_query), I get a failure event after I ask for an additional event past the last success (which has a More=true). That failure event is expected. However, right after that failure event I get an error event. This only happens for the first query.
  3. The code shows the only workaround I found which consists of always asking for an additional event after a failure event, using a timeout. If that additional event comes and is an error it is ignored; if it doesn’t come we proceed after a timing out.

Questions, should anybody know:

  1. Why do I get a success event with More=true even for the last answer to the queries?
  2. Why does this happen for all but the last query?
  3. Why do I get an error event after a failure, but only for the first query?
  4. Am I dong something fundamentally wrong that I should fix?
  5. Is there a better workaround, should all this be bug-related?

Thanks in advance.

% test_client.pl

:- use_module(library(pengines)).

% Test specific queries
test_genealogist_queries :-
    pengine_create([
        server('http://localhost:3030'),  % not required if on the same host
        application(genealogist),
        id(ID),
        destroy(false)  % keep pengine alive for multiple queries
    ]),
    
    % Test multiple queries
    test_query(ID, q1, 'Find all ancestors of sally', ancestor_descendant(X, sally)),
    test_query(ID, q2, 'Find all descendants of mike', ancestor_descendant(mike, X)),
    test_query(ID, q3, 'Find all siblings', siblings(X, Y)),
    test_query(ID, q4, 'Find all parent-child relationships', parent_child(X, Y)),
    
    pengine_destroy(ID),
    writeln('<--- Destroy pengine after all queries').

test_query(ID, QueryName, Description, Query) :-
    format('~n--- ~w: ~w ---~n', [QueryName, Description]),
    pengine_ask(ID, Query, []),
    format('<--- Ask: ~w: ~w~n', [QueryName, Query]),
    collect_all_solutions(ID, QueryName).

% in seconds
timeout(0.1).

collect_all_solutions(ID, QueryName) :-
    timeout(Timeout),
    pengine_event(Event, [listen(ID), timeout(Timeout)]),
    format('---> ~w Event: ~w~n', [QueryName, Event]),
    (   Event = create(ID, Features)
    ->  format('  Pengine created: ~w: ~w~n', [ID, Features]),
        pengine_next(ID, []),
        writeln('<--- Next'),
        collect_all_solutions(ID, QueryName)
    ;   (   Event = success(_, Bindings, _Projection, _Time, More)
        ;   Event = success(_, Bindings, More)
        )
    ->  format('  Result: ~w, More: ~w~n', [Bindings, More]),
        (   More = true
        ->  pengine_next(ID, []),
            writeln('<--- Next'),
            collect_all_solutions(ID, QueryName)
        ;   true
        )
    ;   Event = failure(Term, _Time)
    ->  format('  No more solutions [~w].~n', [Term]),
        % pengine_next(ID, []),
        % writeln('<--- Next'),
        collect_all_solutions(ID, QueryName)    % check for any following (error) event
    ;   Event = error(_, Error)
    ->  format('  Error: ~w~n', [Error])        % without further event collection or processing
    ;   Event = output(_, Output)
    ->  format('  Output: ~w~n', [Output])      % without further event collection or processing
    ;   Event = prompt(_, Prompt)
    ->  format('  Prompt: ~w~n', [Prompt])      % without further event collection or processing
    ;   Event = timeout
    ->  format('  Timeout~n', [])               % without further event collection or processing
    ;   format('  Unknown event: ~w~n', [Event])
    ).

% Main test runner
main :-
    format('Testing pengines server...~n'),
    test_genealogist_queries,
    format('~nTest completed.~n').

Without having looked at all details, here are some observations.

  • The pengines repository is sleeping. Pengines as protocol are part of the packages/pengines in the main SWI-Prolog source. That is fairly well debugged and maintained as it underpins SWISH. SWISH could be seen as a follow-up of the pengines repo, though its focus is different.
  • Typically you can end up with More=true followed by fail in the same way as the toplevel. More=true only indicates there is a choicepoint left and thus there may be another solution.
  • You should only ask for a next on More=true. A any other context is a violation of the protocol.
  • You may want to look at the various clients. A Prolog client is in the library(pengines). Other libraries are available from swish/client at master · SWI-Prolog/swish · GitHub

@Jan (or someone else who might know):

What puzzles me is that after I receive the failure event closing the first query, I also get an error event, right after, and I don’t understand what might be causing it.

Here is a simplified code snippet (I realize the original was unnecessarily long).

Trace:

?- main.
Testing pengines server...

=== q1: Find all ancestors of sally ===
<--- Ask: q1: ancestor_descendant(_86,sally)
---> q1 Event: create(78e8cfb9-9045-46f3-981e-876e76304faf,[slave_limit(3)])
<--- Next
---> q1 Event: success(78e8cfb9-9045-46f3-981e-876e76304faf,[ancestor_descendant(trude,sally)],[],2.1746000000000005e-05,true)
<--- Next
---> q1 Event: success(78e8cfb9-9045-46f3-981e-876e76304faf,[ancestor_descendant(tom,sally)],[],1.7559e-05,true)
<--- Next
---> q1 Event: success(78e8cfb9-9045-46f3-981e-876e76304faf,[ancestor_descendant(mike,sally)],[],3.746300000000006e-05,true)
<--- Next
**---> q1 Event: failure(78e8cfb9-9045-46f3-981e-876e76304faf,1.053800000000004e-05)**
=== End of q1 ===

=== q2: Find all children of tom ===
**<--- Ask: q2: father_child(tom,_84)**
**---> q2 Event: error(78e8cfb9-9045-46f3-981e-876e76304faf,error(protocol_error,_4344))**
=== End of q2 ===
<--- Destroy pengine after all queries

Test completed.
true.

?-

Notice the error event received after the second ask.
What might be causing that extra error event?

An ask after a /failure is correct protocol as far as I understand.

Code:

% One pengine to last for several asks
% with events handled via explicit recursive predicate call

:- use_module(library(pengines)).

% Test specific queries
test_genealogist_queries :-
pengine_create([
server('http://localhost:3030'),
application(genealogist),
id(ID),
destroy(false) % keep pengine alive for multiple queries
]),

% Test multiple queries
test_query(ID, q1, 'Find all ancestors of sally', ancestor_descendant(X, sally)),
test_query(ID, q2, 'Find all children of tom', father_child(tom, X)),

pengine_destroy(ID),
writeln('<--- Destroy pengine after all queries').

test_query(ID, QueryName, Description, Query) :-
format('~n=== ~w: ~w ===~n', [QueryName, Description]),
pengine_ask(ID, Query, []),
format('<--- Ask: ~w: ~w~n', [QueryName, Query]),
collect_all_solutions(ID, QueryName),
format('=== End of ~w ===~n', [QueryName]).

collect_all_solutions(ID, QueryName) :-
pengine_event(Event, [listen(ID)]),
format('---> ~w Event: ~w~n', [QueryName, Event]),
( Event = create(ID, Features)
-> pengine_next(ID, []),
writeln('<--- Next'),
collect_all_solutions(ID, QueryName)
; Event = success(_, Bindings, _Projection, _Time, More)
-> ( More = true
-> pengine_next(ID, []),
writeln('<--- Next'),
collect_all_solutions(ID, QueryName)
; true
)
; true
).

% Main test runner
main :-
format('Testing pengines server...~n'),
test_genealogist_queries,
format('~nTest completed.~n').

genealogist.pl app contains:

ancestor_descendant(X, Y) :- parent_child(X, Y).
ancestor_descendant(X, Z) :- parent_child(X, Y), ancestor_descendant(Y, Z).

parent_child(X, Y) :- mother_child(X, Y).
parent_child(X, Y) :- father_child(X, Y).

mother_child(trude, sally).

father_child(tom, sally).
father_child(tom, erica).
father_child(mike, tom).

I’m using SWI-Prolog version 9.3.29 for x86_64-linux.

Thanks.

The problem is here. You act on the feedback of the pengine create using a next. You should (in this case) not act at all on this. As a result, you send one next too many and get a protocol error. If you remove this pengine_next/2 (and writeln/1), all works fine (for me).

Indeed! Thanks a lot! :grinning_face: