The library(apply_macros) provides goal_expansion/2 rules that cause various meta predicates to be compiled rather than using relatively costly meta-calling. Typically it doesn’t matter too much whether this transformation is done or not. The transformation does cause some of the development tools to produce harder-to-interpret results and, as we have seen, can cause some esoteric differences when binding variables in goal positions to a cut (!). It can also cause library(yall) expressions to be compiled or meta-called. Compiling yall Lambda expressions vs meta-calling them has a huge performance impact as well as semantic differences wrt. variable sharing with the enclosing clause.
As is, any call to one of the affected predicates in code loaded after this library is loaded causes the goal to be re-written. This is rather dubious. I’ve pushed a proposal to improve on that. It introduces a flag apply_macros that is defined in the library. It has 4 values:
false: never rewrite
true: always rewrite. This is the old behavior.
optimise: rewrite if the Prolog flag optimise is true (by setting the flag or using swipl -O). This is the default in the current GIT version
imported: rewrite in the current module if this module explicitly imported the library.
Using swipl -O now loads this library at startup time.
I was actually thinking recently about the more general concern of how to manage libraries that make global changes when loaded. The motivating incident for me was having one module in an application I was building import library(http/http_json) in order to use reply_json/2, which caused breakage in another module that was making HTTP requests and parsing the response, as said response was now no longer a string, but a JSON term.
I’m not sure what a good solution is. Having libraries never actually assert hooks directly, but export some install_blah_hook/0 predicate gives the user more control, but is obviously wildly unergonomic and isn’t backwards compatible. I was considering making something like use_module/3 that would take a third argument, which would control whether or not importing the library would activate hooks (my planned implementation would goal_expand away any definitions of multifile predicates in the library being loaded if hooks were disabled).
The tripartite goals of backwards compatibility, control over effects, and ergonomic typical usages seem pretty hard to satisfy all at once though…
Good point. I think the story about compile-time rewrites is slightly different from hooks like the HTTP input and output parsing/serialization. They are also related though. I don’t know how many other cases are hidden in the libraries.
For the HTTP input, we add hooks for processing certain document types, processing all other types as a simple string. That is not really a problem as http_read_data/3 allows you to specify you want a string, regardless of hooks. The library also allows you to overwrite the content type you get from the server, so you can activate the desired plugin regardless of what the server claims to be the format of the content (some servers are misconfigured).
I think that is enough to deal properly with the HTTP case, but I agree to the general idea that adding hooks can break things. I don’t know a good solution. One option is to use the module system and call the hooks in the context module rather than it the module being hooked. That would mean you need to import the hooks you want into the module making (in this case) the http_post/4 call. One problem with that is that we cannot import clauses for a predicate from multiple modules. I.e., if module A defines p/1 and module B defines p/1 would like to import A and B and get the disjunction of both definitions. Of course, we could make that possible. One issue is how to deal with cuts? Would they cut only alternative clauses in the definition module or cut over the entire predicate in the target module?
Are there Prolog systems that can compose predicates this way? If so, what is this used for and does it work in practice?