Wrapping predicates

Request For Comments
1

I have an early prototype for something I wanted to achieve for a long time: be able to add wrappers around arbitrary predicates. The driving use-case is tabling. As is, tabled predicates can only be defined by loading a source file with a :- table ... directive. The directive creates a wrapper predicate and the real predicate is renamed. This has some unpleasant consequences:

  • As said, we cannot (in a nice and supported way) create tabled predicates on the fly.
  • All the usual predicate inspection and modification affects the wrapper rather than the thing itself.
  • We cannot switch between tabled and non-tabled execution for a predicate.

Some way to execute Prolog code before/after any predicate was already part of Quintus AFAIK. It provides a huge opportunity for debugging tools, ranging from logging, conditional spy points, checking the execution by verifying pre- and post conditions, determinism, etc.

What I now have is a predicate like this:

%!  wrap_predicate(:Head, +Name, -Closure, +Body) is det.
%
%   Wrap the predicate referenced by Head   using Body. Subsequent calls
%   to Head call the  given  Body  term.   Body  may  call  the original
%   definition using a term
%
%       call(Closure, A1, ...)
%
%   Name names the wrapper for inspection using predicate_property/2 or
%   deletion using unwrap_predicate/2.

This allows us to write e.g.,

det(M:Name/Arity) :-
    functor(Head, Name, Arity),
    Head =.. [_|Args],
    Wrapped =.. [call, Closure|Args],
    wrap_predicate(M:Head, det, Closure,
		   (   prolog_current_choice(Ch0),
                       Wrapped,
                       prolog_current_choice(Ch1)
		   ->  (   Ch0 == Ch1
		       ->  true
		       ;   throw(error(det_error(nondet, M:Head), _))
		       )
		   ;   throw(throw(error(det_error(failed, M:Head), _)))
		   )).

Which validates that a predicate is indeed deterministic (succeeds and leaves no choice point behind). We can use this at any time to start checking a predicate, even before it is defined.

Current properties:

  • Wrappers may be added before or after defining the predicate
  • Wrappers are named.
  • A predicate can have multiple wrappers. The execution is simply nested, starting with the last added wrapper.
  • Wrappers may be deleted by name or variable to delete the outermost (last added) wrapper
  • Adding/deleting wrappers is thread-safe, which allows for e.g., temporary wrapping a predicate to track activity of another thread.
  • The wrapper may, but need not to call the real implementation. That is hard to avoid with this design and can be considered a feature or a bug.
  • All predicates can be wrapped, even built-ins. Only built-ins that are implemented as virtual machine instructions (true, fail, unification, comparison, (optimized) arithmetic), etc.) cannot be wrapped.

Questions

  • One of the things I’m struggling with is whether we should allow for multiple wrappers with the same name on a predicate. One of the disadvantages is that when using the above det/1 as a directive in a source file, every re-load will add another layer of wrappers :frowning: If we deny that, we have some options, such as replacing the existing wrapper while maintaining order, rejecting or delete the old, adding the new as outermost.
  • Should wrapping be the leading term to refer to this functionality?
  • Any prior art from which to borrow terminology and functionality?
  • Any use cases that need a different API?

Implementation

For those who want to know how it works, Creating a wrapper creates a predicate __wrap$<name> with the same arity as the predicate being wrapped. The Body is used to create a clause for this predicate. Any SWI-Prolog predicate is executed by starting its supervisor: a (short) sequence of VM instructions that depends on the implementation (C, dynamic, thread-local, etc) and indexing of the predicate. Wrapping creates a closure blob that allows us executing a predicate with a specific supervisor instead of the default one. That is passed as Closure above. Wrapping replaces the supervisor of the wrapped predicate to call the wrapper clause in the __wrap$<name> predicate. The overhead is pretty low :slight_smile:

4 Likes
2

This sounds like/reminds me of Aspect-oriented programming. :woozy_face:

1 Like
3

You are right. Thanks for reminding. Aspect-oriented programming through program transformation is not uncommon in the Prolog world. Tabling is also a form of Aspect-oriented programming where you disconnect the program from the resolution strategy.

1 Like
4

In det/1

I wonder whether we should lift that to wrap_predicate/4, i.e., passing a callable term that calls the unwrapped predicate. That has the advantage of simplifying usage. It makes it more awkward if you want call the original predicate with different arguments though, forcing the user to modify the callable term. Surely someone will sooner or later wish to do so :slight_smile: I do not see much use for changing the call except for quick and dirty work arounds for some bug. This is still possible, but a bit harder.

5

Indeed :slight_smile: But there is prior art to AOP. E.g. before, around, and after methods from OO extensions to LISP. Logtalk implementation of event-driven programming allows the programmer to define before and after event handlers that can also be seen as predicate wrappers. The semantics of these handlers (see Event-driven programming — The Logtalk Handbook v3.72.0 documentation) touch some of the questions that Jan are asking (both explicitly and implicitly). But my quick reading of Jan proposal (and notably his example) is that the wrappers will work as around methods. That’s something that I always wanted to have and I will be looking closely to his proposal… tomorrow (too tired from MTB today :stuck_out_tongue:).

6

I was aware of before and after methods. For Prolog we have to watch 4 ports (5 if we add exceptions) though, making this rather complicated. Around methods are arguable the most generic. I can see some disadvantages

  • The around method may not call the thing wrapped.
  • The wrapper can make arbitrary changes to the semantics.

I consider this in general misuse in the same family as directly calling private predicates inside modules, i.e., you should typically not do that but it is sometimes useful that you can :slight_smile:

1 Like
7

That depends on the composition semantics being the same. By supporting both before/after methods and around methods, we can provide and benefit from the availability of different composition semantics.

8

IIRC starting with the last wrapper added is a problem because if the new wrapper changes something then all the down stream wrappers become affected thus putting the responsibility on the creator of each wrapper to adjust to the changes made by the last added wrapper. On the other hand if the wrappers are executed in the order they are added then then once a wrapper is in place and working, the person creating the wrapper may have to dance around the mistakes of an earlier wrapper but the code doesn’t cause other developers to go back and fix a working begin/end wrapper.

Then there there could also be problems with paired before/after wrappers such as transactions. Should all the logging be done within the transaction, outside the transaction or have one set of logging before the transaction starts and another after the logging starts.

I think while a default of first added first called is reasonable, there should be room left open in case a more complex means of ordering and paring of the wrappers is needed.

I have to admit that the only development related to this I can currently recall is

  1. Begin/end with unit testing
  2. Low level calling conventions to C and assembly code, (Think Endianness) and stack setup/cleanup.
  3. Event models of browsers (That was during the Netscape - Microsoft wars of the late 1990s).

Usually when I get near problems like this at a higher level I try to switch to a functional language like F# with functional composition or in other paradigms switch to events and trampolining . :slightly_smiling_face:

9

A key design decision is indeed if the wrappers are allowed to change the goal being wrapped. In Logtalk, the answer is no:

The execution of actions, defined in a monitor, associated to each event, never affects the term that denotes the message involved.

The relevant code in the runtime is:

% '$lgt_guarded_method_call'(+object_identifier, +callable, +object_identifier, +callable)
%
% wraps the method call with the before and after event handler calls; the "before" event handler
% may prevent a method from being executed by failing and an "after" event handler may prevent a
% method from succeeding by failing; however, event handlers cannot modify the method call

'$lgt_guarded_method_call'(Obj, Msg, Sender, Method) :-
	% call before event handlers
	\+ ('$lgt_before_event_'(Obj, Msg, Sender, _, Before), \+ Before),
	% call method
	call(Method),
	% call after event handlers
	\+ ('$lgt_after_event_'(Obj, Msg, Sender, _, After), \+ After).
10

That seems to support my idea to make a term available that calls the things being wrapped unmodified. Your before and after handlers may fail, causing the method to fail. What is the reasoning behind that?

Not sure I get that. Seems with an around method one can always do the same as with any set of before and after handlers. Yes, what you do before calling and after calling can bind goal variables. That is not a good idea, but allows for things such as reporting the CPU time used by a predicate.

11

SWI-Prolog has quite a rich pallet of primitives to run things out of order: attributed variables, delimited continuations and engines :slight_smile:

You are raising some interesting points though. I never considered much more than tabling and debugging. Just wrapping arbitrary goals with the idea to make them do something more or even different than what they were designed for feels way worse then goto. I always want to reason about concrete scenarios. Your transaction/logging seems one.

I guess the most meaningful composition is to use logging as outer wrapper so we can see when an action failed due to a transaction conflict. Where and how do we want to use this knowledge?

Technically is is quite easy to add a wrapper that is not the outermost one. So, if there is feeling this is useful we should add that to the interface.

12

One other piece of prior art is Python, where

@wrapper
def foo(x): ...

is an abbreviation for(*)

def foo(x): ...
foo = wrapper()(foo)

The wrapper (called “decorator” in Python) can’t look inside the function,(**) so the wrapper functionality is mainly used to add functionality (e.g., caching or “read only”). In general, it’s possible to compose decorators, although this seems to be rare.

Surprisingly, this simple capability seems to have covered most of the use cases for macros or preprocessors.(***) Of course, Prolog is used for more complex data structures than the typical Python program, but it’s instructive to see how much can be done with a relatively simple and safe feature.

(*) It’s a bit more complicated than this (note that wrapper() returns a function), but I don’t want to get into the details. Also, the “more complicated” part is one reason why this tends to be a “for experts only” feature.

(**) In Python, it is possible to look at the byte codes, but this is generally frowned for obvious reasons.

(***) There are some packages for manipulating Python ASTs, but they tend to be used for whole program transformations.

13

Notably, it allows event handlers to work as message (goal) guards.

A major aspect of the composition semantics is the order of applying/calling before/around/after handlers when multiple ones are defined. One option is to tell the programmer that he/she cannot rely on order (analogous to the usual advice for multifile predicates). That’s the design choice in Logtalk, btw (see the link I provided earlier). But you can also, in alternative, use handler registration order and choose either last registered handler applies first or last. Thus, you can use one of these choices for before/after handlers and a different choice for around handlers.

Not in the Logtalk case. Glad that you agree that is not a good idea.

Hum? That doesn’t not require the handlers to be able to modify (i.e. further instantiate) the wrapped message/goal term. Am I missing something in this particular usage example?

14

Interesting. I should learn Python one of these days :stuck_out_tongue: There’s one significant usage case for wrappers that’s worth emphasizing: it allows writing applications that react to what happens at runtime without breaking the encapsulation of the observed/monitored entities and without requiring cooperation of those entities (as e.g. in the Smalltalk’s dependents mechanism). This, btw, strongly suggests that the best design decision is to not allow wrappers to modify the wrapped goals.

15

Simply things such as

statistics(cputime, T0),
Wrapped,
stattistics(cputime, T1),
T is T1 - T0,
...

This requires T0 to be preserved.

I fully agree this is in general a bad idea. Using around wrappers it is hard to avoid that wrappers can affect the semantics of the wrapped goal. For tabling this is actually what we want :slight_smile: Still @EricGT hints that even without modifying the wrapped goal we may want a particular order.

16

But T0 is not a variable in Wrapped; it’s added by the wrapper itself. What I have been emphasizing is that a wrapper should have access to Wrapped bindings but not be able to further instantiate Wrapped. That recommendation doesn’t prevent your example from working.

Regarding relying on a particular order, that would be as bad as relying on the order of multifile predicate clauses. Eric, do you have where a specific case where a wrapper calling fixed order would be convenient or even required? Ideally, there will be a better way to handle such cases.

17

The mixing of the transactions and logging were just something to think about. At the start the basic use-cases you purpose would be a worthy accomplishment. Once those have had time to settle and real usage feedback arrives then expanding out to these ideas might be desired.

I do agree at the start with with @pmoura A key design decision is indeed if the wrappers are allowed to change the goal being wrapped. In Logtalk, the answer is no: I take this to mean that the API does not change and the structure of the values are not altered.

However if/when the scenarios get more complex in the future a feature of F# widely used and liked is
F# computation expressions. The part of it I want to bring over to this idea is that with computation expression they encapsulate the details of the concept into the syntax of the language thus hiding the details from the user. It is very simple and expressive. In the same light that DCG is syntactic sugar this could be implemented as syntactic sugar. See: The “Computation Expressions” series

18

But, T0 must remain bound. This implies we cannot call the before part in a (double) negation to avoid binding. As you (correctly) state that the wrapper requires access to the bindings as well, I do not really see how you would like to prohibit it from binding these arguments. We could split the before part into two pieces, but this too doesn’t help. Suppose we want to write a wrapper that verifies that the arguments are not further instantiated by some goal. We should write this using term_variables/2 to get the variables and pass these to after part.

Thus far I cannot see a sensible way for the system to prevent misbehaving wrappers using around methods. As I said, my primary use case is tabling which is a misbehaving wrapper :slight_smile:

19

I never found a way to implement around methods using before and after methods. I don’t say that it is not possible but I doubt it. As your example illustrates, the semantics are not compatible. The snippet of code that I posted from the Logtalk runtime uses two inlined forall/2 goals to run all handlers without ever binding variables in the wrapped goal. That snippet was not meant to illustrate a suggested implementation for around methods, however, just to show how the semantics for before and after methods are implemented. We ended up discussing different aspects.

I think we’re in agreement that (1) your example doesn’t require modifying the wrapped goal but (2) does require the around method before and after steps (which should not to be confused with before and after methods) to be part of the same conjunction of goals (is there a better way to phrase it?), which (3) precludes using a negation based solution to run all defined handlers.

20

Off the top of my head security might be of importance because another layer might inadvertently disclose something, e.g. a catch returns a message that gives the internal address of a server or something worse like an unencrypted password gets logged.