No, that’s fine, I think that’s potentially useful. Thanks for pointing it out.
What about saying the background knowledge is in (a) particular module(s)?
To avoid recomputation, you can use the dynamic database as cache or using tabling. Unfortunately, currently the calling overhead of a tabled predicate is (still) relatively high 
That will break the separation of concerns I’m trying to keep between the engine and the learning systems. The engine shouldn’t know anything about “background knowledge”- it’s just a second-order meta-interpreter. It should also not know anything about any specially named modules, or any modules at all for that matter. Many ILP systems don’t use modules (I think mine may be the only ones that insist on keeping everything in modules) and it’s possible that someone will want to create a system that doesn’t use them. I don’t want to make that harder.
That’s what I thought you were doing. What I couldn’t understand was why you were meta-interpreting {}/1. And it appears you really don’t want to. But the check built_in_or_library_predicate(L) isn’t sufficiently strong to prevent it.
Now you have the “experiment files” which contains the content that you do want to meta-interpret, yes? If those are loaded into, or by, a module, e.g., exM, you now have a context for a stronger test, i.e., don’t meta-interpret any goals which are imported into that context. That should include any built-in’s, SWIP libraries or imported packs. Examples from the trivial dist module:
?- predicate_property(dist:is(_,_),imported_from(M)).
M = system.
?- predicate_property(dist:member(_,_),imported_from(M)).
M = lists.
?- predicate_property(dist:{}(_),imported_from(M)).
M = clpBNR.
?- predicate_property(dist:dist(_,_,_),imported_from(M)).
false.
so your test in the meta-interpreter could be:
do_not_interpret(L) :-
predicate_property(exM:L,imported_from(_)).
So the trick is to get everything meta-interpretable into context/module exM while not meta-interpreting anything that’s imported.
Am I missing something?
Yeah, sorry. It’s my obsession with keeping the learning engine separate from the learning systems. What you suggest is the thing that Jan suggested and (not trying to be disparaging) it’s the obvious thing to do but it bothers me that it ends up coupling the engine to the experiment file module, which is just this abstraction I happen to use. See my previous comment answering to Jan.
Pardon my ignorance of ILP systems, but that’s a little too abstract for my engineering brain to understand. If the “data” is the experiment file module it can certainly be separated from the meta interpreter. But at some level you have to tie the two together. The context/module name, e.g., exM needn’t be hard coded anywhere; it can be an input, i.e., you could have several “data” modules and pass the appropriate name to the “engine”?
It all seems to depend on how the “data” is structured, but the raw data doesn’t have to be in a module. You could create static generic encapsulation module which just “include”'s the data, or I think you can create a temporary module and load the data into it, but you need to end up with a module for context at runtime before the real work happens.
To clarify the project I’m working on has two parts: a learning engine and four learning systems implemented on top of that engine. The intention is for the engine to be stand-alone so that anyone can roll out their own ILP system on top of it. The four systems are there as examples or templates and for historical reasons (to tie up the engine with existing work).
Now, in the four currently implemented systems the data is in the experiment file module, but there’s nothing to say that future systems will follow the same convention, or that they will use modules at all. For example, other ILP systems (i.e. not one of the four discussed above) load their data from a traditional file (i.e. a Prolog file that does not declare a module) or from a non-Prolog text file.
So I don’t want to force a convention like modules, or not modules, on the user of the engine.
The simplest way to do that is for the meta-interpreter to find definitions of predicates in the user module. This is what it does now. Or more specifically it accesses predicates in the program database via the interface of clause/2. Anything that’s visible to clause/2 can be used. That is the most generic and free-form access scheme I can think of and I don’t want to restrict it. If a user wants to implement a module interface on top of that I give four examples of how to do that in the four systems included with the engine. I don’t have any examples of any non-module schemes though which our discussion reminds me I should try to include exactly because some users will prefer not to use modules.
But as I said, if you want to meta-interpret some code and not others you need to provide a context/module for the meta-interpreter. That doesn’t necessarily mean the input file(s) have to be modules but somehow you have to establish that context before interpreting.
So how are you loading your “systems”? have you considered load_files/2? In particular the module(M) option. If I understand it correctly:
- If the file is not a module it will load it into a module
M. - If the file is a module, it will rename it to ‘M’.
Since M is a name of your (or the user’s) choosing, it can be used as an input parameter to the meta-interpreter and be used to discriminate between predicate calls that are to be meta-interpreted (from module M) or not.
The problem with just doing a consult on modules is that private predicates in the module will not be visible in the user context, e.g., addConstraints_/3. But I think you do want private predicates in one of your systems that happens to be a module, to be visible for meta-interpretation.
Stop me if this isn’t useful.
I can think of various schemes to do that for sure but nothing that doesn’t end up tying the engine down to a specific abstraction.
You mean the files holding the data, not the systems? The systems are implementations of learning algorithms that learn from data files.
The data files are indeed loaded into a unique module with the same name always (experiment_file) but again that is incidental to the design of each current system.
I could, like you suggest, do that a common pattern but given the inconsistencies of Prolog modules that will end up causing all sorts of problems that the user will not be able to fix unless they modify their data format, which is not something I want.
That’s right, but that’s the reason why the meta-interpreter must have a mechanism to decide which predicates to try and meta-interpret. If it tries to meta-interpret a private goal that is not visible to clause/2 at runtime, that will cause an error.
It is, thank you for your patience, but since I have you on the line, here’s an error I’m getting that seems to be something to do with clp(BNR):
You are here:(m(segment,_94366,_94368):-m(_94366,_94380,_94382,_94384,_94386),m(_94368,_94380,_94382,_94384),m(_94368,_94380,_94384,_94386),m(_94368,_94380,_94386,_94382))=(_94260:-{sqrt((0-0.01)^2+(0-0.015)^2)=<_55084{real(0.02236067977499771,0.10000000000000002)}})
ERROR: Arguments are not sufficiently instantiated
ERROR: In:
ERROR: [135] arg(3,_27970,_27972)
ERROR: [134] clpBNR:linkNodeList_([_28010],[...|_28024]/_28018,_28006) at C:/Users/YeGoblynQueenne/AppData/Local/swi-prolog/pack/clpBNR/prolog/clpBNR.pl:1351
ERROR: [133] clpBNR:linkNodeList_([node(le,_28070,1,...),_28078],[...|_28092]/[...|_28098],_28058) at C:/Users/YeGoblynQueenne/AppData/Local/swi-prolog/pack/clpBNR/prolog/clpBNR.pl:1354
ERROR: [132] clpBNR:attr_unify_hook(interval(real,(0.09999999999999999,0.10000000000000002),[...|...],[]),_28130) at C:/Users/YeGoblynQueenne/AppData/Local/swi-prolog/pack/clpBNR/prolog/clpBNR.pl:860
ERROR: [131] uhook(clpBNR,interval(real,(0.09999999999999999,0.10000000000000002),[...|...],[]),_28180) at C:/Program Files/swipl/boot/attvar.pl:86
ERROR: [130] call_all_attr_uhooks(att(clpBNR,interval(real,...,...,[]),[]),_28228) at C:/Program Files/swipl/boot/attvar.pl:63
ERROR: [129] '$wakeup'(wakeup(att(clpBNR,...,[]),_28276,wakeup(...,_28292,...))) at C:/Program Files/swipl/boot/attvar.pl:58
ERROR: [128] trie_gen_compiled(<clause>(0x1f8e9b27010),ret(_28324,_28326,_28328,[_28362],_28332,_28334,_28336,_28338,_28340,_28342,_28344,_28346,_28348,_28350,_28352,_28354,'<skipped 111 of 128>',[...]))5-th clause of trie_gen_compiled/2 <no source>
ERROR: [127] done_leader(complete,fresh(2168579626272,2168582599536),ret(_28410,_28412,_28414,[_28448],_28418,_28420,_28422,_28424,_28426,_28428,_28430,_28432,_28434,_28436,_28438,_28440,'<skipped 111 of 128>',[...]),<clause>(0x1f8e9b27010)) at C:/Program Files/swipl/boot/tabling.pl:541
ERROR: [126] create_table(<trie>(0x1f8d9ef3f00),fresh(2168579626272,2168582599536),ret(_28498,_28500,_28502,[_28536],_28506,_28508,_28510,_28512,_28514,_28516,_28518,_28520,_28522,_28524,_28526,_28528,'<skipped 111 of 128>',[...]),vanilla:prove(...,2,...,...,...,...,...),'$tabling':call(...)) at C:/Program Files/swipl/boot/tabling.pl:397
ERROR: [125] catch('$tabling':create_table(<trie>(0x1f8d9ef3f00),...,...,...,...),deadlock,'$tabling':restart_tabling(<closure>(vanilla:prove/7),...,...)) at C:/Program Files/swipl/boot/init.pl:565
ERROR: [124] start_tabling_2(<closure>(vanilla:prove/7),vanilla:prove(...,2,...,...,...,...,...),'$tabling':call(...),<trie>(0x1f8d9ef3f00),fresh(2168579626272,2168582599536),ret(_28712,_28714,_28716,[_28750],_28720,_28722,_28724,_28726,_28728,_28730,_28732,_28734,_28736,_28738,_28740,_28742,'<skipped 111 of 128>',[...])) at C:/Program Files/swipl/boot/tabling.pl:378
ERROR: [123] start_tabling(<closure>(vanilla:prove/7),vanilla:prove(...,2,...,...,...,...,...),'$tabling':call(...)) at C:/Program Files/swipl/boot/tabling.pl:370
ERROR: [122] vanilla:'$wrap$prove'({... =< _28870},2,[(... :- ...)],[triangle],[bk,builtins|...],[m(segment,triangle,edge)],[m(segment,triangle,edge)])1-st clause of vanilla:'$wrap$prove'/7 <no source>
ERROR: [121] vanilla:prove(dist(p(0.01,0.015),p(0,0),_28978),2,[(... :- ...)],[triangle],[bk,builtins|...],[m(segment,triangle,edge)],[m(segment,triangle,edge)]) at C:/Users/YeGoblynQueenne/Documents/Prolog/vanilla/src/vanilla.pl:294
ERROR: [120] call('$tabling':<closure>(vanilla:prove/7)(...,2,...,...,...,...,...)) at C:/Program Files/swipl/boot/init.pl:502
ERROR: [119] reset('$tabling':call(...),_29120,_29122) at C:/Program Files/swipl/boot/init.pl:604
ERROR: [118] delim(ret(_29166,_29168,_29170,[_29204],_29174,_29176,_29178,_29180,_29182,_29184,_29186,_29188,_29190,_29192,_29194,_29196,'<skipped 111 of 128>',[...]),'$tabling':call(...),2168582597664,[]) at C:/Program Files/swipl/boot/tabling.pl:610
ERROR: [117] activate(ret(_29254,_29256,_29258,[_29292],_29262,_29264,_29266,_29268,_29270,_29272,_29274,_29276,_29278,_29280,_29282,_29284,'<skipped 111 of 128>',[...]),'$tabling':call(...),2168582597664) at C:/Program Files/swipl/boot/tabling.pl:590
ERROR: [116] run_leader(ret(_29346,_29348,_29350,[_29384],_29354,_29356,_29358,_29360,_29362,_29364,_29366,_29368,_29370,_29372,_29374,_29376,'<skipped 111 of 128>',[...]),'$tabling':call(...),fresh(2168579621664,2168582597664),_29340,_29342) at C:/Program Files/swipl/boot/tabling.pl:576
Exception: (135) arg(3, _122012, _27918) ? creep
Exception: (134) clpBNR:linkNodeList_([_27876], [node(le, _3628, 1, $(1, _11832{real(0.02236067977499771,0.10000000000000002)}, _3644{real(0.09999999999999999,0.10000000000000002)}))|_27908]/_27908, _27900) ? creep
Exception: (133) clpBNR:linkNodeList_([node(le, _3628, 0, $(1, _11832{real(0.02236067977499771,0.10000000000000002)}, _3644{real(0.09999999999999999,0.10000000000000002)})), _27876], _27896/_27896, _27900) ? creep
Exception: (132) clpBNR:attr_unify_hook(interval(real, (0.09999999999999999, 0.10000000000000002), [node(le, _3628, 0, $(1, _11832{real(0.02236067977499771,0.10000000000000002)}, _3644{real(0.09999999999999999,0.10000000000000002)}))|_3688], []), _3644{real(0.09999999999999999,0.10000000000000002)}) ? creep
Exception: (128) trie_gen_compiled(<clause>(0x1f8e9b27010), ret(_3578{real(0.02236067977499771,0.10000000000000002)}, _3628, _3644{real(0.09999999999999999,0.10000000000000002)}, _3688, _3716{real(0.018027756377319938,0.018027756377319952)}, _3766, _3780{real(0.0003249999999999998,0.00032500000000000015)}, _3830, _3846{real(9.999999999999995e-05,0.00010000000000000006)}, _3896, _3912{real(-0.010000000000000002,-0.009999999999999998)}, _3956, _3966{real(0.0002249999999999999,0.00022500000000000008)}, _4016, _4032{real(-0.015000000000000001,-0.014999999999999998)}, _4076, _4082, _3962, _4088, _4094, _4106, _4122{real(0.02236067977499771,0.022360679774998116)}, _4172, _4186{real(0.0004999999999999917,0.0005000000000000098)}, _4236, _4252{real(9.999999999999995e-05,0.00010000000000000006)}, _4302, _4318{real(0.009999999999999998,0.010000000000000002)}, _4362, _4372{real(0.0003999999999999918,0.00040000000000000967)}, _4422, _4438{real(-0.02000000000000024,-0.019999999999999796)}, _4488, _4504{real(-1.0200000000000002,-1.0199999999999998)}, _4548, _4554, _4560, _4368, _4566, _4572, _4600{real(0.0014142135623730167,0.0014142135623731746)}, _4650, _4664{real(1.999999999999779e-06,2.000000000000225e-06)}, _4714, _4730{real(9.999999999999993e-07,1.0000000000000006e-06)}, _4780, _4796{real(-0.0010000000000000002,-0.0009999999999999998)}, _4840, _4850{real(9.999999999997795e-07,1.000000000000224e-06)}, _4900, _4916{real(0.0009999999999998899,0.001000000000000112)}, _4966, _4982{real(-0.9990000000000001,-0.9989999999999999)}, _5026, _5032, _5038, _4846, _5044, _5050, _5078{real(0.020024984394500674,0.020024984394500924)}, _5128, _5142{real(0.00040099999999999554,0.00040100000000000546)}, _5192, _5208{real(9.999999999993355e-07,1.000000000000224e-06)}, _5258, _5274{real(-0.001000000000000112,-0.0009999999999996678)}, _5324, _5340{real(-1.0010000000000001,-1.0009999999999997)}, _5384, _5390, _5400{real(0.0003999999999999962,0.0004000000000000052)}, _5450, _5466{real(0.019999999999999907,0.02000000000000013)}, _5516, _5532{real(-0.9800000000000001,-0.9799999999999999)}, _5576, _5582, _5588, _5396, _5594, _5600, _5612, _5628{real(0.022360679774997613,0.02236067977499822)}, _5678, _5692{real(0.0004999999999999874,0.0005000000000000143)}, _5742, _5758{real(9.999999999999572e-05,0.00010000000000000463)}, _5808, _5824{real(-0.010000000000000231,-0.009999999999999787)}, _5874, _5890{real(-1.0100000000000002,-1.0099999999999998)}, _5934, _5940, _5950{real(0.0003999999999999918,0.00040000000000000967)}, _6000, _6016{real(-0.02000000000000024,-0.019999999999999796)}, _6066, _6082{real(-1.0200000000000002,-1.0199999999999998)}, _6126, _6132, _6138, _5946, _6144, _6150, _6178{real(0.020000249998437514,0.02000024999843753)}, _6228, _6242{real(0.0004000099999999998,0.0004000100000000003)}, _6292, _6308{real(0.0003999999999999998,0.00040000000000000024)}, _6358, _6374{real(-0.020000000000000004,-0.019999999999999997)}, _6418, _6428{real(9.999999999999997e-09,1.0000000000000005e-08)}, _6478, _6494{real(-0.00010000000000000002,-9.999999999999999e-05)}, _6538, _6544, _6424, _6550, _6556, _6156, _7198, _7200, _7212, _7214, _7216, _7218, _10982)) ? creep
^ Exception: (123) start_tabling(<closure>(vanilla:prove/7), vanilla:prove({sqrt((0-0.01)^2+(0-0.015)^2)=<_3578{real(0.02236067977499771,0.10000000000000002)}}, 2, [(m(segment, _7198, _7200):-m(_7198, _7212, _7214, _7216, _7218), m(_7200, _7212, _7214, _7216), m(_7200, _7212, _7216, _7218), m(_7200, _7212, _7218, _7214))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, edge)], _10982), '$tabling':call(<closure>(vanilla:prove/7)({sqrt((…- …)^2+(…- …)^2)=<_3578{real(0.02236067977499771,0.10000000000000002)}}, 2, [(m(segment, _7198, _7200):-m(_7198, _7212, _7214, _7216, _7218), m(_7200, _7212, _7214, _7216), m(…, …, …, …), m(…, …, …, …))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, edge)], _10982))) ? creep
Exception: (121) vanilla:prove(dist(p(0.01, 0.015), p(0, 0), _3578{real(0.02236067977499771,0.10000000000000002)}), 2, [(m(segment, _7198, _7200):-m(_7198, _7212, _7214, _7216, _7218), m(_7200, _7212, _7214, _7216), m(_7200, _7212, _7216, _7218), m(_7200, _7212, _7218, _7214))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, edge)], _10982) ? creep
^ Exception: (111) start_tabling(<closure>(vanilla:prove/7), vanilla:prove(dist(p(0.01, 0.015), p(0, 0), _3578{real(0.02236067977499771,0.10000000000000002)}), 2, [(m(segment, _7198, _7200):-m(_7198, _7212, _7214, _7216, _7218), m(_7200, _7212, _7214, _7216), m(_7200, _7212, _7216, _7218), m(_7200, _7212, _7218, _7214))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, edge)], _10982), '$tabling':call(<closure>(vanilla:prove/7)(dist(p(0.01, 0.015), p(0, 0), _3578{real(0.02236067977499771,0.10000000000000002)}), 2, [(m(segment, _7198, _7200):-m(_7198, _7212, _7214, _7216, _7218), m(_7200, _7212, _7214, _7216), m(…, …, …, …), m(…, …, …, …))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, edge)], _10982))) ? creep
Exception: (109) vanilla:prove((dist(p(0.01, 0.015), p(0, 0), _3578{real(0.02236067977499771,0.10000000000000002)}), dist(p(-0.01, 1.02), p(0, 1), _3578{real(0.02236067977499771,0.10000000000000002)})), 2, [(m(segment, _7198, _7200):-m(_7198, _7212, _7214, _7216, _7218), m(_7200, _7212, _7214, _7216), m(_7200, _7212, _7216, _7218), m(_7200, _7212, _7218, _7214))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, edge)], _8632) ? creep
^ Exception: (99) start_tabling(<closure>(vanilla:prove/7), vanilla:prove((dist(p(0.01, 0.015), p(0, 0), _3578{real(0.02236067977499771,0.10000000000000002)}), dist(p(-0.01, 1.02), p(0, 1), _3578{real(0.02236067977499771,0.10000000000000002)})), 2, [(m(segment, _7198, _7200):-m(_7198, _7212, _7214, _7216, _7218), m(_7200, _7212, _7214, _7216), m(_7200, _7212, _7216, _7218), m(_7200, _7212, _7218, _7214))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, edge)], _8632), '$tabling':call(<closure>(vanilla:prove/7)((dist(p(0.01, 0.015), p(0, 0), _3578{real(0.02236067977499771,0.10000000000000002)}), dist(p(-0.01, 1.02), p(0, 1), _3578{real(0.02236067977499771,0.10000000000000002)})), 2, [(m(segment, _7198, _7200):-m(_7198, _7212, _7214, _7216, _7218), m(_7200, _7212, _7214, _7216), m(…, …, …, …), m(…, …, …, …))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, edge)], _8632))) ? creep
Exception: (97) vanilla:prove((point(b, _9066), dist(p(0.01, 0.015), p(0, 0), _3576{real(0.02236067977499771,0.10000000000000002)}), dist(p(-0.01, 1.02), _9066, _3576{real(0.02236067977499771,0.10000000000000002)})), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(_7198, _7210, _7214, _7216), m(_7198, _7210, _7216, _7212))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, edge)], _8630) ? creep
^ Exception: (87) start_tabling(<closure>(vanilla:prove/7), vanilla:prove((point(b, _9066), dist(p(0.01, 0.015), p(0, 0), _3576{real(0.02236067977499771,0.10000000000000002)}), dist(p(-0.01, 1.02), _9066, _3576{real(0.02236067977499771,0.10000000000000002)})), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(_7198, _7210, _7214, _7216), m(_7198, _7210, _7216, _7212))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, edge)], _8630), '$tabling':call(<closure>(vanilla:prove/7)((point(b, _9066), dist(p(0.01, 0.015), p(0, 0), _3576{real(0.02236067977499771,0.10000000000000002)}), dist(p(-0.01, 1.02), _9066, _3576{real(0.02236067977499771,0.10000000000000002)})), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(…, …, …, …), m(…, …, …, …))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, edge)], _8630))) ? creep
Exception: (85) vanilla:prove((point(a, _9060), point(b, _9066), dist(p(0.01, 0.015), _9060, _3576{real(0.02236067977499771,0.10000000000000002)}), dist(p(-0.01, 1.02), _9066, _3576{real(0.02236067977499771,0.10000000000000002)})), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(_7198, _7210, _7214, _7216), m(_7198, _7210, _7216, _7212))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, edge)], _8630) ? creep
^ Exception: (75) start_tabling(<closure>(vanilla:prove/7), vanilla:prove((point(a, _9060), point(b, _9066), dist(p(0.01, 0.015), _9060, _3576{real(0.02236067977499771,0.10000000000000002)}), dist(p(-0.01, 1.02), _9066, _3576{real(0.02236067977499771,0.10000000000000002)})), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(_7198, _7210, _7214, _7216), m(_7198, _7210, _7216, _7212))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, edge)], _8630), '$tabling':call(<closure>(vanilla:prove/7)((point(a, _9060), point(b, _9066), dist(p(0.01, 0.015), _9060, _3576{real(0.02236067977499771,0.10000000000000002)}), dist(p(-0.01, 1.02), _9066, _3576{real(0.02236067977499771,0.10000000000000002)})), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(…, …, …, …), m(…, …, …, …))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, edge)], _8630))) ? creep
Exception: (73) vanilla:prove((seg(_9052, _9054), point(a, _9060), point(b, _9066), dist(_9052, _9060, _3576{real(0.02236067977499771,0.10000000000000002)}), dist(_9054, _9066, _3576{real(0.02236067977499771,0.10000000000000002)})), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(_7198, _7210, _7214, _7216), m(_7198, _7210, _7216, _7212))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, edge)], _8630) ? creep
^ Exception: (63) start_tabling(<closure>(vanilla:prove/7), vanilla:prove((seg(_9052, _9054), point(a, _9060), point(b, _9066), dist(_9052, _9060, _3576{real(0.02236067977499771,0.10000000000000002)}), dist(_9054, _9066, _3576{real(0.02236067977499771,0.10000000000000002)})), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(_7198, _7210, _7214, _7216), m(_7198, _7210, _7216, _7212))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, edge)], _8630), '$tabling':call(<closure>(vanilla:prove/7)((seg(_9052, _9054), point(a, _9060), point(b, _9066), dist(_9052, _9060, _3576{real(0.02236067977499771,0.10000000000000002)}), dist(_9054, _9066, _3576{real(0.02236067977499771,0.10000000000000002)})), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(…, …, …, …), m(…, …, …, …))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, edge)], _8630))) ? creep
Exception: (61) vanilla:prove(m(_8170, _3576{real(0.02236067977499771,0.10000000000000002)}, a, b), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(_7198, _7210, _7214, _7216), m(_7198, _7210, _7216, _7212))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, _8170)], _8630) ? creep
^ Exception: (51) start_tabling(<closure>(vanilla:prove/7), vanilla:prove(m(_8170, _3576{real(0.02236067977499771,0.10000000000000002)}, a, b), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(_7198, _7210, _7214, _7216), m(_7198, _7210, _7216, _7212))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, _8170)], _8630), '$tabling':call(<closure>(vanilla:prove/7)(m(_8170, _3576{real(0.02236067977499771,0.10000000000000002)}, a, b), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(…, …, …, …), m(…, …, …, …))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, _8170)], _8630))) ? creep
Exception: (49) vanilla:prove((m(_8170, _3576{real(0.02236067977499771,0.10000000000000002)}, a, b), m(_8170, _3576{real(0.02236067977499771,0.10000000000000002)}, b, c), m(_8170, _3576{real(0.02236067977499771,0.10000000000000002)}, c, a)), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(_7198, _7210, _7214, _7216), m(_7198, _7210, _7216, _7212))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, _8170)], _7722) ? creep
^ Exception: (39) start_tabling(<closure>(vanilla:prove/7), vanilla:prove((m(_8170, _3576{real(0.02236067977499771,0.10000000000000002)}, a, b), m(_8170, _3576{real(0.02236067977499771,0.10000000000000002)}, b, c), m(_8170, _3576{real(0.02236067977499771,0.10000000000000002)}, c, a)), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(_7198, _7210, _7214, _7216), m(_7198, _7210, _7216, _7212))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, _8170)], _7722), '$tabling':call(<closure>(vanilla:prove/7)((m(_8170, _3576{real(0.02236067977499771,0.10000000000000002)}, a, b), m(_8170, _3576{real(0.02236067977499771,0.10000000000000002)}, b, c), m(_8170, _3576{real(0.02236067977499771,0.10000000000000002)}, c, a)), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(…, …, …, …), m(…, …, …, …))], [triangle], [bk, builtins, hypothesis, metarules], [m(segment, triangle, _8170)], _7722))) ? creep
Exception: (37) vanilla:prove(m(triangle, _3576{real(0.02236067977499771,0.10000000000000002)}, a, b, c), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(_7198, _7210, _7214, _7216), m(_7198, _7210, _7216, _7212))], [triangle], [bk, builtins, hypothesis, metarules], [], _7722) ? creep
^ Exception: (27) start_tabling(<closure>(vanilla:prove/7), vanilla:prove(m(triangle, _3576{real(0.02236067977499771,0.10000000000000002)}, a, b, c), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(_7198, _7210, _7214, _7216), m(_7198, _7210, _7216, _7212))], [triangle], [bk, builtins, hypothesis, metarules], [], _7722), '$tabling':call(<closure>(vanilla:prove/7)(m(triangle, _3576{real(0.02236067977499771,0.10000000000000002)}, a, b, c), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(…, …, …, …), m(…, …, …, …))], [triangle], [bk, builtins, hypothesis, metarules], [], _7722))) ? creep
Exception: (25) vanilla:prove(m(triangle, _3576{real(0.02236067977499771,0.10000000000000002)}, a, b, c), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(_7198, _7210, _7214, _7216), m(_7198, _7210, _7216, _7212))], [triangle], [], _7722) ? creep
Exception: (24) louise:metasubstitutions(m(triangle, _3576{real(0.02236067977499771,0.10000000000000002)}, a, b, c), 2, [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(_7198, _7210, _7214, _7216), m(_7198, _7210, _7216, _7212))], _7600) ? creep
^ Exception: (19) findall(_7600, louise:(member(_7610, [m(triangle, _3576{real(0.02236067977499771,0.10000000000000002)}, a, b, c), m(triangle, _2424{real(0.02236067977499771,0.10000000000000002)}, b, c, a), m(triangle, _244{real(0.02236067977499771,0.10000000000000002)}, c, a, b)]), debug_clauses(examples, 'Positive example:', _7610), metasubstitutions(_7610, 2, [(m(segment, _7196, _7198):-m(…, …, …, …, …), …, …)], _7600), forall(member(_7664-_7666, _7600), constraints(_7664)), debug_clauses(generalise, 'Passed metasub constraints:', [_7600])), _7686) ? creep
Exception: (18) louise:generalise([m(triangle, _3576{real(0.02236067977499771,0.10000000000000002)}, a, b, c), m(triangle, _2424{real(0.02236067977499771,0.10000000000000002)}, b, c, a), m(triangle, _244{real(0.02236067977499771,0.10000000000000002)}, c, a, b)], [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(_7198, _7210, _7212, _7214), m(_7198, _7210, _7214, _7216), m(_7198, _7210, _7216, _7212))], _7384) ? creep
^ Exception: (16) setup_call_cleanup(louise:(write_problem(user, [[(m(edge, _7002, _7004, _7006):-seg(_7016, _7018), point(…, …), …, …), (dist(p(…, …), p(…, …), _6934):-{…=< …}), point(a, p(…, …)), point(…, …)|…]], [<clause>(0x1f8e9687f80), <clause>(0x1f8e96869c0), <clause>(0x1f8e9687200), <clause>(0x1f8e9687980), <clause>(0x1f8e96872c0)|…]), refresh_tables(untable), refresh_tables(table)), louise:(debug(top_program, 'Constructing Top program...', []), generalise([m(triangle, _3576{real(0.02236067977499771,0.10000000000000002)}, a, b, c), m(triangle, _2424{real(0.02236067977499771,0.10000000000000002)}, b, c, a), m(triangle, _244{real(0.02236067977499771,0.10000000000000002)}, c, a, b)], [(m(segment, _7196, _7198):-m(_7196, _7210, _7212, _7214, _7216), m(…, …, …, …), …, …)], _7384), debug_clauses(top_program, 'Generalised Top program', _7384), specialise(_7384, [(m(…, …, …):- …, …)], [], _7414), debug_clauses(top_program, 'Specialised Top program', _7384), respecialise(_7414, […|…], […], _7322), debug_clauses(top_program, 'Repecialised Top program', _7322)), louise:(erase_program_clauses([<clause>(0x1f8e9687f80), <clause>(0x1f8e96869c0), <clause>(0x1f8e9687200), <clause>(0x1f8e9687980), <clause>(0x1f8e96872c0)|…]), refresh_tables(untable))) ? creep
Do you have any ideas what’s going on there? As far as I can tell, the last call made before the error is to this term (in my “You are here” comment):
(m(segment,_94366,_94368):-m(_94366,_94380,_94382,_94384,_94386),m(_94368,_94380,_94382,_94384),m(_94368,_94380,_94384,_94386),m(_94368,_94380,_94386,_94382))=(_94260:-{sqrt((0-0.01)^2+(0-0.015)^2)=<_55084{real(0.02236067977499771,0.10000000000000002)}})
Is it possible the attempt to unify those two terms is eventually causing the error?
It appears there are three “things”: data, system and engine, and I don’t understand what each is, and what restrictions, if any you want to apply. So I should probably just stop guessing.
The unification should obviously fail since the LHS is a term of the form:
`m(segment,_94366,_94368) :- ,(...) % ',' list of 4 m/3 terms
and the RHS is a term of the form:
Var :- {sqrt((0-0.01)^2+(0-0.015)^2) =< _55084{real(0.02236067977499771,0.10000000000000002)}}
But it appears that tabling is involved and I’m not sure of the current state of tabling predicates with attributed variables. (I believe at one point it was not supported.) So if you can disable tabling, it would be good to know if it still happens.
Having said that, I’d like to understand how the error occurs. It shouldn’t cause an error, but I’m having tough time reproducing the problem on something I can test. A non-tabled unification just fails:
?- Z::real(0.02236067977499771,0.10000000000000002),
(m(segment,_94366,_94368):-m(_94366,_94380,_94382,_94384,_94386),m(_94368,_94380,_94382,_94384),m(_94368,_94380,_94384,_94386),m(_94368,_94380,_94386,_94382))
=
(_94260:-{sqrt((0-0.01)^2+(0-0.015)^2)=<_Z}).
false.
It appears that somewhere in attr_unify_hook, which handles unifying two intervals, a malformed list gets produced. (Tabling triggers the hook in your test.) But I can’t reproduce it by calling the hook directly using the arguments in the error report, i.e., when I try that, the indefinite list is well formed (with a terminating list).
I need to keep digging but if you come across a simple test, please forward it. And thanks for reporting.
Thanks for the hint. I can indeed disable tabling so I tried it and it seems I’m getting the same error but it looks like it may now be happening in a different place. I’ll get some logs and see if I can figure it out.
I noticed that too, although I got this error first because I didn’t try to bind Z:
54 ?- (m(segment,_94366,_94368):-m(_94366,_94380,_94382,_94384,_94386),m(_94368,_94380,_94382,_94384),m(_94368,_94380,_94384,_94386),m(_94368,_94380,_94386,_94382))=(_94260:-{sqrt((0-0.01)^2+(0-0.015)^2)=<_55084{real(0.02236067977499771,0.10000000000000002)}}).
ERROR: Syntax error: colon_expected
ERROR: (m(segment,_94366,_94368):-m(_94366,_94380,_94382,_94384,_94386),m(_94368,_94380,_94382,_94384),m(_94368,_94380,_94384,_94386),m(_94368,_94380,_94386,_94382))=(_94260:-{sqrt((0-0.01)^2+(0-0.015)^2)=<_55084{rea
ERROR: ** here **
ERROR: l(0.02236067977499771,0.10000000000000002)}}) .
I guess this doesn’t happen during execution of my program because Z is bound at that point, but not when I copy/paste the query to the command line. Is that right?
Sorry there’s a lot of ground to cover to explain the design. The choice to have a stand-alone engine and a bunch of separate systems doesn’t make it easier to explain and if you haven’t worked with ILP before (as most normal people) then it’s all even more weird.
Here’s a link to the repository for the engine, Vanilla:
Just in case you feel like rummaging around at the code and pointing out what stupid hacks I’ve been using that can be much improved. Always appreciated.
The documentation for Vanilla is pretty poor though. Apologies, I really need to update it.
Yes, _55084{real(0.02236067977499771,0.10000000000000002)} is not legal Prolog syntax, it’s how the system renders clpBNR intervals in error messages. In a query, you need to replace it with a variable which has been “pre-constrained” to be an interval as I did. The bounds really don’t matter in this case; I was just trying to approximate the error condition.
I can see what is happening. I just can’t figure out why, i.e., the root cause. The fact that it’s happening with tabling disabled should make it easier to find. I’m guessing it has something to do with unification in the meta-interpreter (prove) if that helps. I also think it’s a clpBNR issue but I’m hoping you can help find a reproducible test case that allows me to troubleshoot.
Many apologies but I had tabling still on and that’s why I saw the same error (which I thought might look different). I set tabling on/off with two configuration options but I hadn’t set them as I thought (i.e. one was off but the other was on).
Now that I have made sure that tabling is off I can confirm that the error is not reproducible with tabling turned on.
I would like to get to the bottom of this and I’m happy to create a reproducible case. If it’s a simple matter of meta-interpretation modulo tabling mangling some clp(BNR) hook then it should be possible to show that with an ordinary Prolog meta-interpreter and an ordinary Prolog program with some clp(BNR) call in it. The entire machinery of my ILP engine should not be needed I mean. I hope.
Just to mention that a particularity of my engine is that it’s a meta-interpreter that is itself tabled. i.e. what is tabled is the meta-interpreter predicate itself. It’s the simplest way to have a Prolog meta-interpreter that can switch between SLD- and SLG- Resolution seamlessly. I guess. Anyway this may go some way to point to the path where the problem lies.
Unfortunately this doesn’t seem to break in the same way as my code although I’m not sure why it’s failing:
:-module(meta_clp,[prove/1
]).
:- table(prove/1).
prove(true).
prove((L,Ls)):-
prove(L)
,prove(Ls).
prove(L):-
L \== (_,_)
,L \== true
,clause(L,Bs)
,prove(Bs).
105 ?- use_module(library(clpBNR)).
% *** clpBNR v0.13.1 ***.
% Arithmetic global flags will be set to prefer rationals and IEEE continuation values.
true.
106 ?- G = (X::real, [A,B]::real(0,1)), meta_clp:prove(G).
false.
Removing tabling seems to make prove/1 hang:
107 ?- make.
% Reconsult: removed tabling for meta_clp:prove/1
% C:/Users/me/Documents/note_files/project_notes/OMR/meta_clp compiled into meta_clp 0.00 sec, -4 clauses
true.
108 ?- G = (X::real, [A,B]::real(0,1)), time( meta_clp:prove(G) ).
Action (h for help) ? Unknown option (h for help)
Action (h for help) ? abort
% 1,603,087,533 inferences, 150.922 CPU in 157.669 seconds (96% CPU, 10621969 Lips)
% Execution Aborted
More spit-balling:
The current strategy for making this distinction is to use predicate_property and decline meta-interpretation if the predicate has property builtin or autoload(_). That seems to do the job for pretty much anything SWI-Prolog provides, i.e., system or library predicates.
But a problem arises when you want other code to participate in the meta-interpretation, i.e., external predicates are imported, some of which you want meta-interpreted and some not. And the current test is inadequate for this purpose. Further you don’t want to impose any restrictions, e.g., module encapsulation on this code.
Can I assume that any code not subject to meta-interpretation will be imported from somewhere, i.e., it will be constructed as a well formed module and not just random predicates dumped into the user context? If so, any predicates will have an imported_from(Module) property including those that are builtin or autoloaded.
Now the question becomes how are these modules that are not to be subject to meta-interpretation identified. You don’t want to distinguish based on module name because that somehow couples the “engine” to the system/data being meta-interpreted. So consider the module property class(C). Builtin and autoloaded will have class system and library respectively so I think that covers the current test. However all third party modules have class user so that doesn’t help address the problem we’ve been discussing.
Dead end? Well, maybe not quite. For any given module, e.g., clpBNR, the module class can be modified using set_module/1, so it can be arranged that clpBNR can have class library and so look like any SWIP library module. Unfortunately, the circumstances under which this can be done is restrictive, i.e., called from inside the module at module initialization time based on experiments I’ve done.
So the test might look something like:
builtin_or_library_predicate(H) :-
predicate_property(H,imported_from(M)), % imported from a ..
module_property(class(C)),
(C = system ; C = library). % system or library module
So it would appear that a one line patch to clpBNR could solve the immediate problem. But maybe a more general purpose solution could be done, e.g., adding a module_property option to load_files/2 (analogous to option module(M) for renaming modules).
Thanks for posting the link to your GitHub repo. Obviously I’m not going to understand it all anytime soon, but it’s a helpful goto resource for specific detailed questions.
When you do an abort, could you print the goal stack (option ‘g’)?
Here you go but I also remembered to turn on debug and that caused stack overflow exception that I didn’t get before:
103 ?- G = (X::real, [A,B]::real(0,1)), time( meta_clp:prove(G) ).
Action (h for help) ? goals
[8,174,739] meta_clp:prove(call((list(_88), !, intervals_(_88, real))))
[19] meta_clp:prove('<garbage_collected>')
[18] meta_clp:prove((_88::real, [_90, _92]::real(0, 1)))
[1] epilog:thread_run_interactor(<pce>(0x17ba2278650,prolog_terminal), main, '<garbage_collected>', '<garbage_collected>', prolog, '<garbage_collected>', load)
Action (h for help) ? abort
% 24,525,480 inferences, 1.969 CPU in 5.802 seconds (34% CPU, 12457387 Lips)
% Execution Aborted
104 ?- debug.
true.
[debug] 105 ?- G = (X::real, [A,B]::real(0,1)), time( meta_clp:prove(G) ).
% 15,249,870 inferences, 2.375 CPU in 2.681 seconds (89% CPU, 6420998 Lips)
ERROR: Stack limit (1.0Gb) exceeded
ERROR: Stack sizes: local: 0.4Gb, global: 0.2Gb, trail: 93.1Mb
ERROR: Stack depth: 3,049,991, last-call: 0%, Choice points: 3,049,990
ERROR: In:
ERROR: [3,049,991] system:clause(<compound call/1>, _1110)
ERROR: [3,049,990] meta_clp:prove(<compound call/1>)
ERROR: [3,049,989] meta_clp:prove(<compound call/1>)
ERROR: [3,049,988] meta_clp:prove(<compound call/1>)
ERROR: [3,049,987] meta_clp:prove(<compound call/1>)
ERROR:
ERROR: Use the --stack_limit=size[KMG] command line option or
ERROR: ?- set_prolog_flag(stack_limit, 2_147_483_648). to double the limit.
Exception: (19) throw(error(resource_error(stack), stack_overflow{choicepoints:3049990, depth:3049991, environments:3049989, globalused:262139, localused:452734, stack:[frame(3049991, system:clause(call/1, _402), []), frame(3049990, meta_clp:prove(call/1), []), frame(3049989, meta_clp:prove(…/ …), []), frame(3049988, meta_clp:prove(…), []), frame(3049987, …: …, [])], stack_limit:1048576, trailused:95312})) ? creep
Call: (18) _302=true ? creep
Exit: (18) true=true ? creep
^ Exception: (15) prolog_statistics:time(meta_clp:prove((_88::real, [_90, _92]::real(0, 1)))) ? creep
The new exception may just be because debugging makes everything more memory-intensive. After increasing the stack to 8GB I no longer get the exception. So probably a red herring but reporting it for the sake of completeness.
EDIT: oops, no, hang on. Yeah I still get the stack overflow with 8GB stack limit. I still think it’s a red herring.