Inspecting programs with attributed variables using clause/2

General
1

I have this code that uses library(clpBNR):

edge(T, A, B) :-
        seg(A1, B1),
        point(A,PA),
        point(B,PB),
        dist(A1, PA, T),
        dist(B1, PB, T).

dist(p(X1, Y1), p(X2, Y2), D) :-
% clpBNR constraint, X1/Y1/X2/Y2/D don't have to be ground
        { sqrt((X2 - X1)^2 + (Y2 - Y1)^2) =< D }.

seg(..., ...) % Not important, omitted for brevity
point(..., ...)

Now, when I try to get the body of edge/3 with clause/2 I get … silent death:

122 ?- listing(edge/3).
experiment_file:edge(T, A, B) :-
    seg(A1, B1),
    point(A, PA),
    point(B, PB),
    dist(A1, PA, T),
    dist(B1, PB, T).

true.

123 ?- clause(edge(A,B,C), Bod).
false.

Is this normal? Is there an alternative to clause/2 to get the body of a clause when there are attributed variables in one of the body goals?

2

Try ?- clause(experiment_file:edge(A,B,C), Bod). … listing/1 uses clause/2, so it should work. It is only more flexible to find what you are trying to list. I.e., if there is no module specified try in every module.

3

Not relevant to your problem, but there will be no attributes attached to any of the variables in the results of listing/1 or clause/2. They don’t get defined (via side effect) until dist/3 is actually called, and ?- listing(edge/3). won’t do that.

4

Thanks, yes that works.

That means I have to complicate my code a bit.

This was causing an error in a predicate I use to collect the definitions of background knowledge predicates in an ILP system. I haven’t had this error with other background knowledge predicates, I only saw the error when I added a predicate that uses attributed variables to the background knowledge.

What is the difference? Do attributed variables work differently in modules than ordinary variables?

5

Thanks. I don’t yet understand what the problem is exactly. As you say maybe it’s not the attributed variables at all. To clarify, I first got the error when trying to get the clauses of dist/3 from the module experiment_file where they are originally defined as part of a background knowledge base for an ILP system.

I collect the definitions of predicates in the background knowledge (BK) with the following program:

program(F/A,M,Ps):-
	!
	,program([F/A],M,Ps).
program(Ss,M,Ps):-
	findall(P
       ,(member(F/A,Ss)
		,functor(H,F,A)
		,M:clause(H,B)
		,(   B == true
		 ->  P = H
		 ;   P = (H:-B)
		 )
		)
	       ,Ps).

The call to clause/2 failed because the module name in M was passed as user. The BK predicates are defined in the module experiment_file but by the time program/3 is called they should be exported to user - that’s specifically to make them simpler to find. It seems that dist/3 is somehow special and whatever I do in my code to export the other BK predicates to user doesn’t work for it. Since dist/3 uses clp(BNR) operators and clp(BNR) uses attributed variables I suspected attributed variables, but I don’t understand either subject well (because I haven’t used attributed variables and CLP in general much) so I’m still not sure I understand what is the problem and how I should fix it.

6

The SWI-Prolog compiler knows nothing about attributed variables. assert/1 and friends simply ignore attributes, handling an attributed variable exactly the same as a normal variable. This is actually used as term/goal expansion uses attributes to track the status of variables (freshness as well as their names).

clp(BNR) probably rewrites its input though. Normally listing/1 or clause/2 tell you how the input is modified. If results look strange first try predicate_property/2 and check for any unexpected properties.

7

clpBNR does not rewrite anything that I’m aware of. If

:- module(dist,[dist/3]).

dist(p(X1, Y1), p(X2, Y2), D) :-
% clpBNR constraint, X1/Y1/X2/Y2/D don't have to be ground
        { sqrt((X2 - X1)^2 + (Y2 - Y1)^2) =< D }.

then

?- listing(dist).
dist:dist(p(X1, Y1), p(X2, Y2), D) :-
    { sqrt((X2-X1)^2+(Y2-Y1)^2)=<D
    }.

true.

?- functor(H,dist,3), clause(dist:H,B).
H = dist(p(_A, _B), p(_C, _D), _E),
B = {sqrt((_C-_A)^2+(_D-_B)^2)=<_E}.

So I think the problem is in program/3:

		,M:clause(H,B)

The module qualification is on clause/2 not on H (as in example above).

8

I can’t say I remember why I’m module-qualifying clause/2 rather than is argument.

On the bright side I tracked the error more precisely to a bit of code that tries to decide whether a predicate is a built-in or autoloaded:

built_in_or_library_predicate(H):-
	predicate_property(H, built_in)
	,!.
built_in_or_library_predicate(H):-
	predicate_property(H, autoload(_)).

This is called before calling program/3 to stop program/3 from collecting clauses of built-ins or predicates from libraries, which shouldn’t be further processed. In particular, I want to avoid meta-interpreting those predicates (especially since trying to do that with built-ins raises an error) which is what eventually happens to the predicates collected by program/3 (that is, they are passed to a meta-interpreter).

This is one of the calls that fail and ultimately cause an error to be raised (btw this error is not a SWI-Prolog error but an exception thrown by my system to guard against things like that specifically. Hey, defensive programming works):

211 ?- predicate_property(addConstraints_(_11594,_11596,_11598), P).
false.

Obviously addConstraints_/3 is neither a built-in nor autoloaded but I’m not sure how to catch code from external libraries.

predicate_property/1 keeps confusing me. For example, if I call it with a functor/arity it succeeds but it always thinks the predicate is defined in librar(yall) and I don’t know why.

212 ?- predicate_property(addConstraints_/3, P).
P = interpreted ;
P = visible ;
P = static ;
P = imported_from(yall) ;
P = transparent ;
P = (meta_predicate? / 0) ;
P = file('C:/Program Files/swipl/library/yall.pl') ;
P = line_count(275) ;
P = number_of_clauses(1) ;
P = number_of_rules(1) ;
P = last_modified_generation(25213) ;
P = defined ;
P = size(408) ;
false.

I mean it does that for every predicate.

213 ?- predicate_property(member/2, P).
P = interpreted ;
P = visible ;
P = static ;
P = imported_from(yall) ;
P = transparent ;
P = (meta_predicate? / 0) ;
P = file('C:/Program Files/swipl/library/yall.pl') ;
P = line_count(275) ;
P = number_of_clauses(1) ;
P = number_of_rules(1) ;
P = last_modified_generation(25213) ;
P = defined ;
P = size(408) ;
false.

On the other hand, if I module-qualify the goal like @ridgeworks suggests I get what I expect:

214 ?- predicate_property(M:addConstraints_(_11594,_11596,_11598), P).
M = clpBNR,
P = interpreted ;
M = clpBNR,
P = visible ;
M = clpBNR,
P = static ;
M = clpBNR,
P = file('C:/Users/YeGoblynQueenne/AppData/Local/swi-prolog/pack/clpBNR/prolog/clpBNR.pl') ;
M = clpBNR,
P = line_count(999) ;
M = clpBNR,
P = number_of_clauses(4) ;
M = clpBNR,
P = number_of_rules(4) ;
M = clpBNR,
P = last_modified_generation(28180) ;
M = clpBNR,
P = defined ;
M = clpBNR,
P = size(1344) ;
M = clpBNR,
P = primary_index(1) ;
false.

I can do hack something on top of that but what I would really like to have is a clean and simple way to keep built-ins and library predicates, including but not restricted to auto-loaded ones, out of my meta-interpreter. Btw, even when an error is not raised, meta-interpreting some of those just causes unnecessary overhead.

On the other hand I think that may just need a better design on my part. My system is trying to guess what predicates should and shouldn’t be meta-interpreted and that’s always going to cause stuff to fail. Maybe I should just ask the user to flag those predicates up. But that adds more work for the user so I tried to avoid it.

9

That’s great. I was hoping that attributed variables would not be special and magickal.

To give some more exposition, I’m for the first time trying to use one of my ILP systems to learn using background knowledge that includes clp(BNR) predicates. That is to make it easier to learn programs with numeric arguments. Instead of doing complicated things to handle arithmetic specially, I’m just going to use CLP to handle it for me and I’ll let my system learn a Prolog progam with CLP terms in it. That’s the plan. I’m very excited about this because if I can make it work it’s really going to be a bit of an unlock. So far ILP has sucked in learning programs that handle numbers at all, unless we’re talking integers in Peano notation.

My learning engine is just a second-order Prolog meta-interpreter so I’m pretty confident there won’t be any weird issues around it handling attributed variables. However, I do need to pre-process background knowledge predicates and to do that I need to get their definitions. This is where clause/2 comes in but it seems the snag is that clp(BNR) is a pack rather than a built-in or autoloaded, and it seems I hadn’t had to deal with that before.

Hey, progress.

10

built_in is fine. To see whether a predicate is from a library, get the implementation module (using predicate_property(Head, imported_from(M)) and on failure the module qualifier of Head) and use module_property/2, property class. I think you are only interested in modules of class user.

11

This confused me as well, but I think the penny finally dropped. addConstraints_/3 isn’t a functor/arity Head to predicate_property. It’s the term /(addConstraints_,3) which happens to match a predicate head in library yall which explains the result.

Also see @LogicalCaptain’s note at the end of predicate_property/2 .

Packs don’t get installed in the library:

?- module_property(clpBNR,class(C)).
C = user.

You might be able to use the module file property to detect a path containing ‘/swi-prolog/pack/’ but that feels a little hacky.

12

Yeah, I’d prefer not to have to check library names and/or paths to library source files, that’s way too hacky.

But it seems there are some deeper issues in my system with the way it accesses code in modules. I have this concept of an “experiment file” where all the data needed for training is exported from a single module, named “experiment_file” so that the rest of the system always knows where to go and find examples, background knowledge and anything else.

However, I want background predicates in the experiment file module to be able to call predicates outside of it, and that’s where the fun begins because now we have predicates with their head in one module and their body spread over possibly more than one others (I be that must feel woozy).

That’s what’s going on here, where dist/3 is defined in the experiment file module but {}/1 eventually has to find addConstraints_/3 in the clpBNR module. Somehow my code is getting lost between the modules and can’t find addConstraints_/3 (and the other clpBNR implementation predicates) when it needs it.

It doesn’t help that it’s been a while since I designed and coded all that so I don’t remember exactly how it works, but I’ll figure it out eventually.

13

Well I was hoping that I can access everything through user but that hasn’t worked in the past. The “experiment file” module mechanism I describe above has always been a constant source of bugs in my systems (all of them; I use it all the time). I’ve longed hoped for a simple, clean and portable way to declare data with the same format and the same interface across multiple files with different names without causing errors but I’ve never quite managed to make it work.

You’ve suggested a few different things in the past, like records etc. including the mechanism I use now, where predicates from a source file are loaded into the experiment_file module via the redefine_module/1 option in load_files/2. This still ends up very fiddly and you can find several posts in this forum where I ask how to tidy up over the years. It’s still a bit of a fiddly mess.

14

I clearly misunderstood what you meant by “class user”. I thought you meant the user module but you mean one of the values for the module_property/2 option class. OK.

I gave this a try but I can’t seem to make it work. Unless I module-qualify e.g. the predicate {}/1 in clpBNR predicate_property/2 just fails silently. When I module-qualify it it doesn’t return a module although I can maybe work with the file name as discussed above. The problem is I don’t know ahead of time what module a predicate is loaded from; that’s part of what I’m trying to figure out programmatically:

271 ?- predicate_property({_}, P).
false.

272 ?- predicate_property(clpBNR:{_}, P).
P = interpreted ;
P = visible ;
P = exported ;
P = static ;
P = file('C:/Users/YeGoblynQueenne/AppData/Local/swi-prolog/pack/clpBNR/prolog/clpBNR.pl') ;
P = line_count(964) ;
P = number_of_clauses(1) ;
P = number_of_rules(1) ;
P = last_modified_generation(27929) ;
P = defined ;
P = size(544) ;
false.
15

You have to import it into the calling context of course:

101 ?- [library(clpBNR)].
% *** clpBNR v0.12.0 ***.
%   Arithmetic global flags will be set to prefer rationals and IEEE continuation values.
true.

102 ?- predicate_property({_}, P).
P = interpreted ;
P = visible ;
P = static ;
P = imported_from(clpBNR) ;
P = file('/home/jan/.local/share/swi-prolog/pack/clpBNR/prolog/clpBNR.pl') 
...
16

I can understand why you need information about {}/1. It’s exported from clpBNR which presumably the experiment file module (let’s call it exM) uses. But addConstraints/3 is private (not exported) and that’s just the tip of the iceberg. Why is that of interest?

But I think the crux of the problem is that if external predicates are used in exM, you need to discover where they’re implemented, i.e., what are the module dependencies of exM. E.g., if something in exM uses {}/1 and it’s not implemented locally, then it must come from one of the modules in its list of dependencies (presumably clpBNR). And you need this from outside the context of exM, i.e., the current calling context is not exM, i.e., it’s somewhere else in your system.

If this is the issue, I’m not aware of a solution; hopefully @jan has an idea.

17

If you want to know about exM, you need that as context. But, you can use extM:Value for meta-arguments to pass this context and there is also @(Goal,Context) to call a goal given a particular context. I’d look at library such as library(prolog_codewalk) to get the full picture to find out what is called. Also library(prolog_xref) may help.

That said, I think @stassa.p needs some way to specify which files/modules are subject to ILP rather than trying to find rules on things that are not subject to ILP. Somehow I fear you will never get this fool proof.

18

I think I’ve sorted out my issue by understanding that an imported predicate becomes part of the context of the importing module. Using stripped down dist module (in place of exM) with `clpBNR:

?- dist:{X==Y+1}. 
X::real(-1.0Inf, 1.0Inf),
Y::real(-1.0Inf, 1.0Inf).

And predicate property will tell me where it’s imported from:

?- predicate_property(dist:{_},imported_from(M)).
M = clpBNR.

Apologies if this was spelled out before and it went over my head.

But it sounds like that isn’t @stassa.p 's problem so I’ll get out of the way.

19

The reason is meta-interpretation. If you think of how a classic Prolog meta-interpreter works, it uses clause/2 to get the body literals of the latest goal popped from its call-stack.

prove(true).
prove((L,Ls)):-
    prove(L)
   ,prove(Ls).
prove((L)):-
    clause(L,Bs) % Here
   ,prove(Bs)

So when clause/2 finds {}/1 it will place each of its body literals on the call stack. For example:

289 ?- clause({T >= 0}, Bs).
Bs = (g_read('$clpBNR:thread_init_done', _), term_variables(T>=0, _A), declare_vars_(_A), addConstraints_(T>=0, _B/_B, _C), stable_(_C)).

So that’s where addConstraints_/3 finds its way in the meta-interpretation loop.

As you suggest, I don’t actually want to meta-interpret addConstraints_/3. In fact, I don’t even want to meta-interpret {}/1! I can hand its execution off to Prolog. So in my meta-interpreter I have a guard that in theory should stop {}/1 and each of the predicates in its deductive closure from being called. At a very high-level and skipping cuts etc, this is the skeleton:

prove(true).
prove((L,Ls)):-
    prove(L)
   ,prove(Ls).
% Note this new clause
prove(L):-
   built_in_or_library_predicate(L)
  ,call(L)
  ,prove(true).
prove(L):-
    clause(L,Bs) % Here
   ,prove(Bs)

So when the new clause finds that the literal L is e.g. {T =< 0}, it hands it off to call/1 and doesn’t meta-interpret its goals.

But that only works if built_in_or_library_predicate/1 can reliably detect that {}/1 is from a library.

There’s also an earlier stage in the pipeline that pre-processes predicates that have to be meta-interpreted and the same logic applies.

TBH the simplest thing is to pass a list of predicate indicators (functor/arity terms) to the meta-interpreter so that it knows which goals it should pass to Prolog and which it should continue to meta-interpret but uh… I think I wanted to avoid the overhead of scanning a possibly large list of predicate indicators in the middle of the most resource-intensive loop of my engine. I’m not sure.

20

Yes, that’s exactly it. As I say above I could stick the predicate indicators of every ILP predicate in a list given as an argument to my meta-interpreter and scan the list when I have to make a decision ILP/not-ILP but I was worried about the overhead. There’s more efficient ways to do it than member/2 of course.

I think the real problem is somewhat lazy design on my part, or too conservative design maybe. For example the set of background predicates (the ones “subject to ILP”) are declared by the user as a list and I could simply take each predicate indicator from that list and assert a super-secret bk term in the dynamic database at the start of learning. So if I have a list like:

background_knowledge(p/2, [q/2,r/3,s/1,t/4]).

I could assert a set of facts:

'$background_predicate'(q/2)
'$background_predicate'(r/3)
'$background_predicate'(s/1)
'$background_predicate'(t/4)

And then a guard to clause/2 so that only predicates in that set are passed to it.

But… that wasn’t my original design and now I need to think about it. Also, I always try to not use the dynamic database if I can help it.

EDIT: there’s a further issue that my newest system is in two parts: a stand-alone learning engine on the one hand, and several learning systems implemented on top of that engine on the other hand. I want to keep the abstractions used in the systems from leaking to the engine so that the engine is not coupled to any particular system and it can be used to develop any new system with maximum flexibility. So for example the engine itself doesn’t know anything about “experiment files” not even about “background knowledge” because it shouldn’t. It’s just a second-order Prolog meta-intepreter and it only knows what a second-order Prolog meta-intepreter needs to know. And that of course makes design fiddlier.