Threaded queries? Rulebase independence?

Help!
1

Hi,
I’m interested in using SWI prolog as a knowledge base and reasoning system for an online medical webservice.

Suppose I’m interested in serving multiple queries in parallel:

  • can I use a single SWI prolog instance?
  • can the knowledge base be efficiently and independently forked – i.e. can I load a very initial large knowledge base, assert a few extra facts for query 1, assert facts for query 2, and have the knowledge bases for query 1 and query 2 be independent and also have the queries execute simultaneously?)

(am happy to get stuck into the C/C++ library)

Thanks

  • Stuart
1 Like
2

Using a many worlds design pattern give you query independence by separating the facts for each query with the initial knowledge base being shared among all queries. I have a pattern example here. In this case, you have individual learning problems instead of queries with as many learning problems loaded simultaneously as needed (look into the SCRIPT.txt file for an example run). With the query facts separated, using SWI-Prolog support for multi-threading allows you to easily have concurrent queries. No need to resort to C/C++.

3

Roughly, you can separate knowledge by using different modules or use thread local predicates. A quick look suggests using modules is close to what Paulo suggests.

Next you can represent the combination of a large stable model and small variations using a shared stable KB and differential KBs with a some simple code that combines the two (e.g., something is true if it is in the shared or local KB and something is true if it is in the shared and its negation is not in the local KB.

Current work on tabling may come handy for certain reasoning tasks, providing better termination, less dependency on order for performance, well founded semantics for negation and incremental truth maintenance.

4

Perfect. Many thanks!

5

Inheritance naturally provides that functionality without requiring writing any additional code. The shared KB can provide default definitions for anything that may be overridden by query-specific facts (as described by the OP). A very simple example:

:- object(shared).

    :- public(a/1).
    a(1).

    :- public(b/1).
    b(one).

:- end

and then e.g.

:- object(query1, extends(shared)).

    a(2).

:- end_object.

or if a query needs to be created dynamically at runtime:

?- create_object(query2, [extends(shared)], [], [b(two)]).
true.

we then get:

?- query1::(a(X), b(Y)).
X = 2, Y = one.

?- query2::(a(X), b(Y)).
X = 1, Y = two.

As Jan suggested, you can combine this sketched solution with tabling, threading, and other notable SWI-Prolog features.

1 Like
6

I’ve explored the many worlds pattern. It does seems to work, but my use of it doesn’t seem safe to me.

More or less, this is the type of thing I’m trying to get to run (from Python with the foreign language interface):

def test1():
   module_name = "test"

   module_t_ctx = PL_new_module(module_name)
   try:
      prolog_assert("f(1)", module_t_ctx)          # Assert into module. PL_open_query(module_t_ctx,...)
      #.... potentially may other assertions into module test
      result = prolog_query("f(X)", module_t_ctx)  # PL_open_query(module_t_ctx,...)
      # Just one match please
      # len(result)==1?  --- should not be more
   finally:
      # How can I return prolog to its original state, whatever names the tests assert
      # Can retract/abolish  test1:f/1   ... what about the things I missed?

test1()
test1()  # Should start from the same state

I’d like to ensure that new assertions are independent to subsequent queries (I’d like long running queries to be idempotent).

One way to do this is to retractall or abolish the new assertions to restore the knowledge base to its original state. This seems unsafe because its hard to be sure that I didn’t miss a predicate – how can I list all of the predicates programmatic ally to see what is remaining?

Another way might be for a queries to place its working data into a uniquely named module:

mod_123:assert(f(1)).

which has the benefit (I think) of being thread-safe (the new fact doesn’t interfere with facts in other threads since they’re all in a separate namespace).

At cleanup, I can do:

abolish(mod_123:f/1).

but what I’d like is:

abolish(mod_123:X).

It is possible to abolish all facts in a module.

Also, if create very many modules mod_1 … mod_n, will there be a memory leak?
Is see the foreign language interface has PL_new_module, but no PL_delete_module.

Regards,

  • Stuart
7

See also The "many worlds" design pattern - Logtalk for a discussion on two alternative implementations for the pattern.

I would avoid asserting and retracting predicates. With either of the alternatives described in the blog post linked above, you can simply add the dynamic data as an object that extends the static part of the knowledge base. When you want to move to a different set of dynamic data, yo can simply use another object for the new set. If you use dynamic objects, you can also simply abolish the old objects.

8

Modules are independent, so you can indeed create multiple worlds. Neither modules nor predicates are subject to garbage collection though. You abolish a predicate, but that will merely unlink its clauses and leave these for being collected by the clause garbage collector. The (not so big) predicate instance itself is never reclaimed. Same applies to a module. You can empty a module, but you cannot destroy the module.

There in_temporary_module/3 which does reclaim (almost) the whole lot. Using temporary modules has some restrictions though, notably not to make any other module dependent on this module and make sure all goals related to the module have finished before the module is being deleted. The temporary module mechanism is what SWISH uses to isolate users and make sure almost all data is reclaimed after a query terminates.

I guess you do not use all these modules at the same time, so you should be able to empty and reuse modules.

You can do this as

?- forall(current_predicate(mod_123:P), abolish(mod_123:P)).

Another, somewhat clumsy, way is to add an additional argument to each predicate that represents the world. With some work you can achieve that using program transformation, so your code remains nice. That can fully and safely reclaim all data as clauses are subject to garbage collection. It may harm clause indexing (and thus performance) though.

I don’t know what Logtalk does at its core when using @pmoura’s suggestions. Eventually, I guess , it should be one of the above. Higher levels of abstraction can never make something possible, it can only make it (sometimes a lot) easier.

2 Likes
9

An object (or a protocol or a category) is abolished by abolishing all its predicates, including those that define it as an entity. Thus, after calling abolish_object/1. After, any attempts to access the abolished object (using e.g. curent_object/1 will throw an existence error.

True. But also a bit misleading in this case as Logtalk objects are not an abstraction over Prolog modules. They are not implemented using modules or require in any way that the backend Prolog compiler supports a module system. But you can do with objects everything you can do with modules. What makes Logtalk objects a technologically superior solution compared with Prolog modules is that you can natively and elegantly express a richer variety of programming idioms and patterns without having to resort to lower levels of abstraction to hack your way out or be forced to build your own custom abstractions. Thus, it’s not as much about making some tasks easier but more about what you can express at the same level of abstraction.

10

So, given broken SWI-Prolog, Logtalk objects cannot be fully deleted. Yes, you can abolish the object and verify Logtalk says it doesn’t exist. In the end creating the objects added some predicates and clauses. Abolishing the predicates cause also SWI-Prolog to claim they no longer exist, but in fact they do. As for the immediate consequences this also holds for the clauses that are deleted but if they are not reachable (executing, active references, etc.) their memory will eventually be reclaimed to be reused for something else (depending on malloc()/free() policies). Eventually this may change.

11

Many thanks both – is very helpful. I’m still struggling to bend modules to my will.

Ideally I’d have a common set of rules that see an independent set of rules (>1000) loaded into a thread’s query. Let’s sat we want a set of rules shared between many threads:

% thread_rules.pl
rule(X) :- fact(X).

we have the problem that rule/1 needs to find fact/1 (to be later defined by thread).
The facts can’t be entered into user: or thread_rules: because it would interfere with parallel threads.

I think what is needed is to achieve something like:

thread1:rule(X) :- thread1:fact(X).

that is, rules that have references to facts must be duplicated per thread. (The ugly alternative
is to pass the set of facts as parameter to rule(FACTS, X) … and its dependency predicates, which may be many.

What seems to work (sort of) is using load_files.

% thread_setup.pl
% Define:    thread1:rule(X) :- thread1:fact(X)
thread1:setup:-
    thread1:load_files(thread_rules, [module(thread1), register(false)]),
    print("Setup thread1 rules").

% Define:    thread2:rule(X) :- thread1:fact(X)
thread2:setup :-
    thread2:load_files(thread_rules, [module(thread2), register(false)]),
    print("Setup thread2 rules").

% Define facts for testing
thread1:fact(1).
thread2:fact(2).

Which I can use as:

%example.pl    -- run as cat example.pl | swipl
load_files(thread_setup, []).
thread1:setup.  % Prints setup thread1 rules
thread2:setup.  % Prints setup thread2 rules


findall(X, thread2:rule(X), Y).   % --> [1]
findall(X, thread1:rule(X), Y).   % undefine procedure thread1:rule/1

This works great for the findall(X, thread2:rule(X), Y).. It matches thread2:fact(2), but
for the second query (the last line) thread1:rule doesn’t exist! (ERROR: Undefined procedure: thread1:rule/1).

I’m guessing that the second call to load_files unloaded whatever was loaded in the first call.
(I was hoping register(false) would avoid this).

So…

  1. Is there a better way to attach a common set of rule sourcefiles to per-thread facts? Or perhaps somehow copy the contents of a module into another and rebind the references to that facts to the current module? (I briefly tried import/1, but couldn’t make it work).

  2. Does my problem mean that load_files is not thread safe? This docs mention that
    "Reloading a previously loaded file is safe, ". Is it so? In my case we’ve had the
    effect of abolish/unload_file (since thread1:rule has disappeared). The docs
    say abolish/unload_file are not thread safe.

  3. To get my example to work, I had to place

thread1:fact(1).
thread2:fact(2).

into thread_setup.pl. If I move them to setup.pl

% Define facts for testing
thread1:fact(1).
thread2:fact(2).

load_files(thread_setup, []).
thread1:setup.
thread2:setup.

findall(X, thread2:rule(X), Y).
findall(X, thread1:rule(X), Y).

… thread2:rule(X) finds thread2:fact/1 is undefined. Why?

Thanks in advance,

  • Stuart
12

At least for (3), I found that I needed if the file was run as cat example.pl | swipl

% example.pl
assertz(thread1:fact(1)).
assertz(thread2:fact(2)).
...

It worked.
Seems the contents are interpretted differently via stdin, than load_files. That makes sense.

13

Seems you are associating a module with a thread? I may have misread, but most likely the solution is this:

  • Put all your rules in a module (e.g., rules) and nicely export them,

  • Put all your facts for some thread in a module threadN

  • Import rules into threadN by calling

    use_module(threadN:rules).

Does this make sense?

14

Hi Jan, Thank you. Yes – this was the first solution I had, I couldn’t make it work:

% thread_rules.pl 
% Put all your rules in a module (e.g., rules) and nicely export them,
:- module(thread_rules, [rule/1]).
rule(X) :- fact(X).

% example.pl

% Put all your facts for some thread in a module threadN:
assertz(thread1:fact(1)).
assertz(thread2:fact(2)).

% Import rules into threadN by calling
use_module(thread1:thread_rules).
use_module(thread2:thread_rules).

findall(X, thread2:rule(X), Y).
findall(X, thread1:rule(X), Y).

cat example.pl | swipl:

ERROR: Undefined procedure: thread_rules:fact/1
ERROR: In:
ERROR:   [14] thread_rules:fact(_4748)
ERROR:   [13] thread_rules:rule(_4778) at ..../thread_rules.pl:2
15

Sorry. Late … Indeed, this doesn’t work. I think Logtalk has extends for that. In plain SWI-Prolog you can get this to work, but the rules look a bit ugly. Sure you can write term expansion rules to deal with that though.

edit (this is not the only way)

rules.pl

:- module(rules,
          [ rule/1
          ]).
:- meta_predicate
    rule(:).

rule(M:X) :- M:fact(X).

fact1.pl

:- module(fact1,
          []).
:- use_module(rules).

fact(sure).

Running …

swipl rules.pl fact1.pl
?- 1 ?- fact1:rule(X).
X = sure.
16

Why the word “broken”? My understanding of any language featuring a garbage collector is that the memory used by what’s collected will eventually be reclaimed, not that memory is recovered on the spot. As long as, in this case, predicate clauses don’t come back haunting the user visible code, all should work fine. That have always been my experience with SWI-Prolog garbage collector (which is one of the best among Prolog systems). A simple demonstrative example in my case would be:

?- create_object(foo, [], [public(a/1)], [a(1),a(2),a(3)]).
true.

?- foo::a(N).
N = 1 ;
N = 2 ;
N = 3.

?- abolish_object(foo).
true.

?- current_object(foo).
false.

?- foo::a(N).
!     Existence error: object foo does not exist
...

?- create_object(foo, [], [public(a/1)], [a(4),a(5),a(6)]).
true.

?- foo::a(N).
N = 4 ;
N = 5 ;
N = 6.

Thus, all working as expected. Same when dealing with dynamic clauses in static objects.

17

@stuz5000 Is the “many worlds” design pattern described at The "many worlds" design pattern - Logtalk a good fit for your problem? I only had time to quickly skim over the details you posted. I will try to do that later.

18

@pmoura Yes. I does look like a good fit. I like the features that logtalk offers. However, I’m a little apprehensive to use it. Its like asking a C programming to use C++ – it makes more sense to bite off as complexity grows. (In my imagination only) I expect logtalk is a leaky abstraction the same way as C++ is - every strange problem can be self-solved IFF you understand how the abstraction works (vanilla Prolog has this issue too, and results in a steep learning curve). I’m relearning Prolog (after a 25 year hiatus). Baby steps for me.

19

My suggestion is for you to take a look to short tutorial and see if you’re comfortable with the basics described there.

1 Like
20

This is an interesting use (or abuse, depending on the perspective :smile:) of the meta_predicate/1 directive. The main issue of your solution is scalability, however. For example, the following will not work:

:- module(rules, [rule/1]).

:- meta_predicate(rule(:)).

rule(X) :-
	transform1(X).

transform1(X) :- 
	Y is X * 2,
	transform2(Y).

transform2(M:X) :-
	Y is abs(X),
	M:fact(Y).

We will get:

?- fact1:rule(1).
ERROR: Arithmetic: `fact1/0' is not a function
ERROR: In:
ERROR:   [12] _6690 is (fact1:1)*2
...

I.e. is not enough to make all exported predicates that may, directly or indirectly, calling fact/1 meta-predicates (thus forcing you to track these dependencies). You also need to make all intermediate predicates (transform1/1 in my example) aware that one or more arguments are (or may be!) module qualified. In this example, being forced to write instead:

transform1(M:X) :- 
	Y is X * 2,
	transform2(M:Y).

With this change, we do get:

?- fact1:rule(1).
true.

With a set of non-trivial rules, each with multiple arguments, you risk ending up with module prefixing in most predicates. Notably, as I exemplified, in intermediate predicates. Elegance of the minimal example you provided is thus quickly lost and you inherit (a little pun :stuck_out_tongue:; bad Paulo!) a bunch of maintenance hurdles.

The Logtalk version of my example is arguably cleaner:

:- object(rules).

	:- public([rule/1, fact/1]).

	rule(X) :-
		transform1(X).

	transform1(X) :- 
		Y is X * 2,
		transform2(Y).

	transform2(X) :-
		Y is abs(X),
		% call fact/1 in "self"
		::fact(Y).

:- end_object.

The only place in the code where you need to be context aware is when calling fact/1 (which could also be declared protected or private).