Errors considered harmful

Interesting discussion on Picat extensions and cuts and I’d like to add my own personal pet peeve with “standard” Prolog.

I don’t think there would be much opposition to the topic title. After all (unhandled) errors rip control away from the programmer and terminate the search for solutions. There might be perfectly valid solutions produced by backtracking but they never get to see the light of day. So one of the objectives of debugging a program, in addition to ensuring that successful execution satisfies the program specification, is to remove the possibility of “unexpected” errors, since premature termination is usually not specified as a requirement.

So how does the programmer accomplish this? By adding more code. This usually takes the form of additional guard filters that cause a failure before the call to the predicate that potentially generates the error. The other alternative is deal with errors after the fact by using catch/3. Either way, the programmer has to bear the burden when predicates are called which result in something other the allowed true (continue) or false (fail and backtrack) allowed in logic programs.

Now there are a few cases that are legitimate errors, but they should be confined to circumstances where continued execution isn’t possible or recommended. Such circumstances include running out of memory or possibly some detection of a non-termination condition. But most of the so-called errors generated by builtin predicates don’t fit into this category.

I would argue that if a predicate is called that cannot produce a successful binding for the logic variables in the head should just fail. This includes type errors, domain errors, existence errors, permission errors, and the like. Failure indicates there are no possible bindings of the head variables that would result in success (result = true). And failure allows alternative choices to be tried without necessitating additional guard predicates.

Instantiation errors are a different animal. Failure (result=false) isn’t appropriate because there might be some binding of the head variables that would succeed. But neither is success (result=true) because at some future point a binding could occur which would have caused the predicate to fail. So the result should be conditionally_true which is really just a constraint. One way of implementing this would be a global hook, like user:exception, which would determine the fate of instantiation errors: either continue with possible constraint applied or fail (the default).

I’m anticipating some opposition to this idea from those who use errors to provide diagnostic information to the debugging process, since failure does not provide such information. In this context, I’ve also seen the term “unexpected failure” described as a problem. I really don’t understand how failure can be unexpected in a logic programming langauge, where success and failure are the only two results allowed in the foundational theory. IMO the real problem is unexpected errors. If, for any given predicate, failure is not an option, then there should be an explicit last clause that handles “everything else” and succeeds.

To compensate for the loss of diagnostic information, I would propose that when in “debug” mode any predicate that currently generates an error with diagnostic info would fail and output identical information using the standard message mechanism. But there should be a way of disabling this, since such failures may now be considered normal behaviour.

Going down this path results in something that definitely isn’t “standard” Prolog, whatever that means these days. Perhaps it’s a dialect or perhaps it’s a different programming language like Picat. But I think it results in cleaner, more obvious code. And it should be compatible with any Prolog code that uses (now redundant) guards to eliminate unexpected errors. It won’t necessarily be compatible with code that uses catch/3 (expected failure?) unless the handler in the catch is fail. Other use cases will have to implement handlers as alternative clauses.

1 Like

This isn’t a bad solution for program level code, but the biggest culprits IMO are the numerous builtin predicates that throw errors at the drop of a hat. No matter how you structure it, the only escape for the programmer is adding more code.

For application software “errors at the drop of a hat” are great. You avoid writing code. I might be missing something of course. But the idea is that you don’t have compile-time type checks, and you don’t want to religiously check the arguments of the predicates you write, so instead you use built-ins and you hope that if you don’t see errors, you didn’t use the predicate in a way that you didn’t mean to.

But maybe some examples would make everything easier to argue one way or the other.

2 Likes

This sounds all kinds of wrong to me. But who am I? I am a poor soul who somehow ended up making a living by reading, maintaining, and sometimes writing code, the kind that has been touched by an unknown number of people and will probably be running somewhere, completely forgotten, until it breaks and someone has to fix it. People like me love their errors and exceptions. They are, in practice, always (yes, always) caused by programmer errors. So they are, you know, errors.

Disclaimer: I don’t use Prolog for the code that I share with others. For this, I have to use boring, verbose, well-understood procedural languages like Java and Python, and more rarely, whatever was in vogue when it was written. I sometimes use Prolog when I need results instead of code. So I write it in a hurry, and again, “errors at the drop of the hat” are exactly what I want. (Are they what I need though? I wish I had time to ask myself those kinds of questions… :wink: )

2 Likes

No that’s not the idea at all. Check the hell out the code at compile time; errors at that time are perfectly acceptable. But Prolog is a dynamic and typeless language based on a formal logic that pays little little attention to the meaning of a term. So you only discover the actual value of a term at runtime. What I’m proposing is that the program have the option of dealing with expected errors with simpler code.

Lets not confuse errors and exceptions with bugs. A bug is something that causes a program to not behave as specified. The kind of errors I’m talking about are those sanctioned by the system to be considered normal semantics. Errors (as currently defined by SWIP builtins) may or may not be bugs; that’s for the program to decide. The problem is it never gets the chance unless it adds additional “defensive” code every time it calls one of those predicates.

Are you sure? I think what you really want (need?) is diagnostic information that enables you to fix bugs. I don’t think I’m proposing anything that precludes that possibility.

I am not confusing errors and exceptions with bugs. I am just saying that as a software practitioner, in my limited experience, exceptions are invariably caused by human mistakes. I don’t like calling them bugs, they didn’t creep into my code when I was looking the other way :slight_smile: those are mistakes I made, or someone else made, but I need to fix. They are not always programming mistakes, often the mistake happened earlier, at the specification stage, or is hiding between components of the system. But still, mistakes.

Again, difficult to talk about it without concrete examples, but in my approach to writing Prolog, I avoid a lot of code by knowing that there is a high chance I get an error if I mess up.

Not sure about checking Prolog code at compile time, how do I do it?

As for errors, they are by definition unexpected?

I suspect we are talking about different things altogether. For example, if an error is expected, why not just not cause the error to start with? So I guess I am totally missing your point.

Not sure, but a distinct possibility on any Internet discussion.

And how exactly is that accomplished? For example:

?- N=fred, number_chars(N,Cs).
ERROR: Type error: `number' expected, found `fred' (an atom)
ERROR: In:
ERROR:   [11] number_chars(fred,_3042)
ERROR:    [9] toplevel_call(user:user:(...)) at /Applications/SWI-Prolog8.3.15.app/

So to avoid this error I typically have to add guard code to cause it to fail:

?- N=fred, number(N), number_chars(N,Cs).
false.

So I’m arguing that number_chars should just outright fail given a first argument which is not a number rather than “blowing up”, and save the programmer the trouble of having to “not cause the error to start with”.

So a trivial example with an equally trivial solution, but the suite of builtin predicates is littered with them:

?- sort(fred,X1).
ERROR: Type error: `list' expected, found `fred' (an atom)
ERROR: In:
ERROR:   [10] sort(fred,_5474)
ERROR:    [9] toplevel_call(user:user:(...)) at /Applications/SWI-Prolog8.3.15.app

?- N = fred, X2 is N+1.
ERROR: Arithmetic: `fred/0' is not a function
ERROR: In:
ERROR:   [11] _8680 is fred+1
ERROR:    [9] toplevel_call(user:user:(...)) at /Applications/SWI-Prolog8.3.15.app/

?- findall(X3,42,Bag).
ERROR: Type error: `callable' expected, found `42' (an integer)
ERROR: In:
ERROR:   [14] findall_loop(_3284,user:42,_3288,[])
ERROR:   [13] setup_call_catcher_cleanup('$bags':'$new_findall_bag','$bags':findall_loop(_3342,...,_3346,

For each of these there are perfectly acceptable logical outcomes, i.e., failure. There are no X1’s that satisfy the relationship sort(fred,X1), or X2’s that result from evaluating the arithmetic expression fred+1, etc. And yes, these are all silly things to try to do, but when you’re building libraries you have no control over the questions users will ask.

As I said in the original post, this is a pet peeve, not a showstopper. But it’s still a real issue IMO.

I think this is another way of implementing guard code, so it still requires the addition of defensive code to turn errors into what I think should be the default, i.e., failure. My main issue with assertions is when is it ever safe to turn them off, however they are implemented. But that’s another topic.

Sorry, I missed this earlier post in the flurry.

I’m not proposing to do, or not do, anything that isn’t currently done. The compiler/loader catches a ton of errors from simple syntax to missing predicate definitions. It also warns you about singleton variables in clauses, etc. which may or may not be bugs. All good stuff.

Not really, the SWIP manual includes lots of cases where if using builtin functions you should expect errors. And just to be clear, I’m talking about errors as formally defined by SWIP, the ones that cause termination of your program, not errors in design or other programming mistakes.

I guess that we are talking about the same thing, but looking at it from very different angles. From where I stand, what I see is that the errors that SWI-Prolog is throwing are meant to do exactly that: blow up your program, so that you can catch your design mistakes and your programming mistakes.

Looking at it from my angle, there is nothing worse than silent failure when you infact gave fred to number_chars/2.

But of course I do not develop libraries, I write code in a hurry without thinking about it too deeply. The proverbial code monkey. :monkey:

1 Like

True. And perhaps even better, I can use goal expansion to rewrite “number_chars(N,Cs)” as “number(N), number_chars(N,Cs)” so the additional overhead is minimal. This is definitely doable but it niggles me a bit to repeat the same checks that the called predicate is doing anyway, just to fail a branch of the the search path so I can try an alternate; just doesn’t feel like logic programming anymore.

So the next problem in taking this approach if finding all the builtin cases that generate exceptions. Unfortunately there is no annotation for exceptions like there are for argument modes, determinism, and ISO compliance. One would think errors are equally, if not more, important. Java, and perhaps others, insists they be in the method declaration; so much stronger than an annotation.

Big Aside: In general I’m looking for ways to restore the ability to read a Prolog program declaratively, i.e., as a piece of logic. There are innumerable reasons (side-effects, red-cuts, argument modes, etc.) why this is hard, if not impossible, but one issue is that the filters (var/1, atom/1, etc. which do not do any unification) do not commute with unification in clause bodies (conjunction). For this reason perhaps, good logic programming style dictates that filters should be placed before any unification, as in the pre-unification portion of a Picat goal (if I’m understanding it correctly). The filters I’m talking about here should really be in the head of the called predicate, but to avoid errors they must be placed before the call. Rewriting hides all this grotty detail, but I really wish it didn’t have to be that way.

For those who might question the desire for a declarative reading, from the preface of “The Art of Prolog”:

A good Prolog programming style develops from thinking declaratively about the logic of a situation.

It’s taken me about 30 years to appreciate the wisdom of this statement.

To each his own, I guess.

Just remember that eliminating “ERRORS” is no guarantee of success, so called silent failures are much more common, in fact normal procedure for logic programming, i.e., every time you have multiple clauses in a predicate, or use an “if-then-else” construct.

And here’s one that caught me by surprise:

?- atom_chars(42,Cs).
Cs = ['4', '2'].

?- atom_chars(f(x),Cs).
ERROR: Type error: `atom' expected, found `f(x)' (a compound)
ERROR: In:
ERROR:   [10] atom_chars(f(x),_775260)

In neither case is the first argument an atom, yet one succeeds and the other “blows up”. I would argue both should fail, but at least if both generated “ERRORS”, that would be consistent.

How?

The rdet package exists to throw an error when a deterministic predicate fails. By using it, I’ve saved a lot of time in debugging.

(*) The package has some side-effects – e.g., it also does an implicit cut. I’ve written some code to allow more nuanced checks (e.g., succeeds at least once or succeeds without any choice points), but haven’t yet published it.

Looks like a useful tool to have in the toolbox. The standard debuggers help but can be a little intimidating, particularly when you’re learning. This package is more limited but focuses on what I suspect is a common issue.

Yes. I think this stuff needs to find a place in the system. I also have a little library that allows annotating individual goals. See https://github.com/JanWielemaker/my-prolog-lib/blob/188108f67b0c9edc1a0274a4bf0e39d54cd62cdc/det.pl

Ideally I’d like VM support for det and possibly semidet validation that comes at (practically) zero overhead.

You actually discover the type of a term during runtime – if you added meta-data to check it.

Is it Mercury that you are looking for?

Edit:

Btw, I am a big believer in Ada which has very strong type checking – e.g. even a range of integer can be its own type – but, I guess, Prolog as a “general purpose” Logic programming language, if you want types you have to define them.

I am curious – how much type information can be added for compile time checks, without loosing the generality of the language?

Dan

Back to the original topic, a possible middle ground: the main argument against treating errors as logical failure seems to be the loss of diagnostic information, particularly during debugging. Suppose errors only generate exceptions when in debug mode (TBD).

From the discussion in the manual on exceptions, I think this was similar to how errors were originally treated contingent on the debug_on_error flag. This flag now appears to only affect foreign code execution, but perhaps it could be extended to switch between silent failure and exception for other kinds of errors. An ideal implementation would be to allow this flag to be enabled on a per error type basis: true/false for all, or false([ErrorTypes]) to selectively disable exceptions for those error types, i.e., silent failure.

This would permit me to treat “ERRORS” as normal logical failures which cause backtracking to alternatives. Those who prefer that their code “blow up” to get the diagnostic information can leave the flag set to true (the default). And you can switch between the two modes as you move from development mode to production mode - your choice.

I don’t understand this at all. First order logic has no third truth value.

There are many so called errors, that are certainly not a third value. If there are no possible bindings of logic variables than can succeed, that’s failure. arg(-1,f(x),A) is a failure, not an exception. There is no value of A that will ever satisfy this relation. And, by extension, \+ arg(-1,f(x),A) is true. No extended truth table required.

Now in the process of evaluating a goal, there is a third possible case: the predicate cannot determine whether to succeed or fail, based on the current value of its arguments. These are normally treated as instantiation errors. But another interpretation would be conditionally true, i.e., defer the decision until more information is available and continue with the next goal in the conjunction. It will eventually resolve itself as part of subsequent unifications of shared variables, or appear as part of the “conditional” answer. This is what CLP languages do; conditions remaining are expressed as residual goals.

Bottom line: IMO exceptions are not logical, they’re an implementation artifact. They have their uses as in many other programming langauges, but they’re not a fundamental part of a logic programming language.

I can agree with that. On the other hand, built-ins are rarely used in pure logical code. They tend to appear in the more “procedural” parts of applications.