I’m using: SWI-Prolog version 9.2.5
I have a test case that initially fails, but if I reconsult the file and run again (without any changes) it passes.
I don’t understand why simply reconsulting and running again changes the outcome. There should be no state accumulated between the runs (no dynamic predicates being asserted).
The code is at muggin/state.pl at 0815e8bbca59aef24d16cb3fdea69a0ca0f96d4c · tatut/muggin · GitHub
Do you happen to write to files in your tests? Any other side-effects?
After downloading the file, the test failing is the one that fails for me with
ERROR: /home/jan/Downloads/state.pl:225:
ERROR: test state:patch1: received error: source_sink `"test/01-patch.json"' does not exist
Didn’t check the code much, but maybe it is changing directory? The only quick suspicious things I see is the :- use_module(state). inside the test block. That causes recursive loading. The system handles loops in use_module/1 fine, but I’m not sure about loading the module you are defining inside the same module 
Tests units create a sub module that has full access to the predicates in the rest of the file (but definitions inside the block hide the ones from outside).
No writing, just reading the file test/01-patch.json.
But that test that reads a file is the only one that fails.
Good to know, I removed the use_module from the test… no effect.
The test requires the external file that is in the repo. Working directory doesn’t seem to be the problem, I tried with an absolute path to the file, no effect.
Found the problematic line in example(File) goal:
maplist([Codes,Str]>>string_codes(Str, Codes), Examples, Strs),
If I change the name Codes to Codes1, it works… I don’t know why that lambda expression is evaluated differently on the 2nd consultation,
library(yall) has different semantics on whether the code is compiled or interpreted wrt. variables shared with the remainder of the clause. If all is right, the execution should print a warning when autoloading >>/2.
As a rule of thumb, when using library(yall), make sure you load it explicitly 
Thank you for the explanation, important thing to know.
I don’t remember seeing any warning. I thought that the {Free}/List form would share, the comment about compilation wasn’t clear to me.
Library yall installs a rule for goal_expansion/2 that translates calls to lambda expressions into normal Prolog code, possibly adding a helper predicate. That is, if the library is loaded. If you call a lambda expression without library(yall) loaded, it is not translated at compile time. Instead. the call to (in this case) >>/4 triggers auto loading of library(yall) and uses the runtime version of >>/4. Ideally, goal expansion may improve performance, but should never change the semantics. Unfortunately that is not true for library(yall).
Maybe library(yall) shouldn’t be autoloaded but require an explicit use_module(library(yall))?
(I’ve run into weird error messages due to a typos with one the operators that library(yall) uses (/ or >>), which unexpectedly triggered an autoload of library(yall) when I had no intention of using it.)
Since some time, autoloading should give a warning, e.g.
?- a/b.
Warning: Auto-loading (/)/2 from library(yall) into module user is deprecated due to term- or goal-expansion
ERROR: Unknown procedure: b/0 (DWIM could not correct goal)
I think that should be enough. Oh, I see this (still) depends on the experimental flag
:- set_prolog_flag(warn_autoload, true).
Possibly that should be enabled by default. Unfortunately it is not always right in the sense that some autoloaded calls are just fine. That notably applies to library(debug), which is indeed better loaded explicitly when called in code, but calling e.g., debug/1 interactively is just fine. I dislike a list of exceptions, but possibly that is the best option …
If I understand correctly the code for warn_autoload/2 (in boot/Autoload.pl), it checks for an expansion hook in the auto-loaded file – library(debug) is interesting in that its goal expansions turn off some built-in goal expansion.
Yes. The only thing that matters is that if there is an expansion hook, using the goal may behave differently when called without the hook than with the hook. The main issue is that the hook typically applies to only a subset of the exported predicates and we do not know which goals are affected. Also, the general intend is that goal expansion should not affect semantics, only performance. For many, that is true. For library(yall), it is not.
A possible alternative might be some declaration that tells the system whether the expansion affects semantics or only performance. Given that, we could offer more precise feedback. Another issue is that some goal expansion acts globally (default) and other only affect files that explicitly load the library with the expansion.
Expansion is a hard topic. I’ve spend some time on a new design quite a while ago. Some systems have a better designed expansion mechanism, but none really satisfactory IMO and many are seriously incompatible and we cannot afford that.
Could library(debug) refine the message hook so that no message is done if warn_autoload is on? That would allow changing the default to on (which would have helped me a few times).
(I don’t think I’ve ever seen a macro system (m4, cpp, lisp macros, LaTex, etc.) that didn’t have some significant problem – macros seem to be a “deep problem” in computer science that so far doesn’t have good solutions.
Probably. Either that or some more general declaration about what the macro expansion does. I’m more tempted to look into the latter.
Probably true Unfortunately the alternatives tend to come with their problems and limitations as well
DSLs come to mind, esp. if homegrown or proprietary those can be a real blocker in some contexts