Bug hunting toolbox

Note: This is a work in progress. When it is complete these notes will be removed.

Note: Do not reply to this topic, questions, concerns, comments, etc. are to handled in
Wiki Discussion: Bug hunting toolbox

Note: This is just to get the topic started and hopefully others to jump in and make this useful. Even if you are brand new to Prolog, this is such a basic and fundamental concept that you can and should join in to help improve the value of this Wiki. Join the discussion at Wiki Discussion: Bug hunting toolbox

Note: This post is a wiki page and if you have Trust level: Basic you can edit this by clicking on the pencil icon in the lower right. Capture
These topics also have history so they can be rolled-back if needed.

Note: For now this is just a random collection of items as a list. They still need to be properly formatted and have a more detailed explanation with examples, but for now having them collected in one place is better than not having them at all. As noted, anyone can edit this and even add more details and example code to this. :smiley:


Prolog Unit Tests

Individual test can be run from the command line, e.g.

:- begin_tests(test_group).

test(01) :- 
   format('done.~w',[]).

:- end_tests(test_group).
?- run_tests(test_group:1).

This can also be run using gtrace/0, e.g.

?- gtrace.
true.

[trace]  ?- run_tests(test_group:1).

Debugging and Tracing Programs

catch/3

catch_with_backtrace/3

call_with_time_limit/2

call_with_depth_limit/3

call_with_inference_limit/3

Control Predicates

Failure slice

protocol/1

print_term/2

write_canonical/1

write_term(X,[dotlists(true)]).

List the program, predicates or clauses

Verify Type of a Term

format/2

library(debug) - Manage debug messages and check assertions. Example of using debug/1

Obtaining Runtime Statistics

Execution profiling


error.pl – Error generating support

jan
Just for the record, error:has_type/2 should not throw exceptions. That is the task of the more general must_be/2 that uses has_type/2.


  • is_of_type(+Type, @Term) is semidet
    True if Term satisfies Type.

Runtime determinacy checker (rdet) - can be used to trap predicates that should never fail.

library(error): Error generating support

  • must_be (+Type, @Term)
    True if Term satisfies the type constraints for Type. Defined types are atom , atomic , between , boolean , callable , chars , codes , text , compound , constant , float , integer , nonneg , positive_integer , negative_integer , nonvar , number , oneof , list , list_or_partial_list , symbol , var , rational , encoding , dict and string .

library(check): Consistency checking - This library provides some consistency checks for the loaded Prolog program.

Cross-referencer - A cross-referencer is a tool that examines the caller-callee relation between predicates, and, using this information to explicate dependency relations between source files, finds calls to non-existing predicates and predicates for which no callers can be found.

  • gxref - Run cross-referencer on all currently loaded files and present a graphical overview of the result.

Exception handling

Draw diagrams - Graph drawing software (Wikipedia) or Pen and paper.
GraphViz is a popular console application which has several domain specific languages for describing graphs on which DOT seems to be the most popular. Example here.

Generate diagrams for your Prolog module applications

Consider Cognitive dimensions of notations

If the data has a sequential pattern to it check Online Encyclopedia of Integer Sequences(OEIS) for references and ideas.Catalan numbers and Hamilton Paths are common

When writing formatted text to a string:
format/3 with with_output_to/2

Environment Control (Prolog flags)

  • generate_debug_info(bool, changeable)
    If true (default) generate code that can be debugged using trace/0, spy/1, etc. Can be set to false using the -nodebug . This flag is scoped within a source file. Many of the libraries have :- set_prolog_flag(generate_debug_info, false) to hide their details from a normal trace.
    See: Append/3 isn’t deterministic if first arg is var?

listing/2 The option source(true) is quite helpful to find the actual definition of multifile predicates and where the clauses come from.

Add this to your code at the place where you want a conditional break and run make/0 .

    (some_condition -> gtrace ; true),

Using the Logtalk linter to check Prolog modules code: See: Linter

Descriptions of the icons/glyphs in the graphical debugger?

For a Finite State Automata of the states using gtrace/0 see: Gtrace - Example finite-state automaton
The FSA is a PDF which needs to be downloaded: SWI-Prolog Debugger FSA.pdf (682.7 KB)

prolog_clause.pl – Get detailed source-information about a clause. This module started life as part of the GUI tracer. As it is generally useful for debugging purposes it has moved to the general Prolog library.

Examining the program - Noted in topic: Line number (and file name) of fact read, specifically this post.

Edit Exceptions in the prolog editor. Who knew that existed and had error defined to trace “if not caught”? - Trace_on_error

SWI-Prolog web-enabled monitor (GitHub) (post)
Usage: ?- prolog_ide(debug_monitor).

Related: https://stackoverflow.com/a/59491635/1243762

?- debug(concurrent). See: Some notes on writing a concurrent programing Howto. Comments and corrections welcome

Note: The coverage analysis tool as of 11/28/2019 is just a proof of concept. See: PLunit How-To / teaching example

Related to debugging (trying to understand working code).
When the code is working, there are lots of good test cases and you don’t know what a predicate or line is doing then comment it out, run the test and see what breaks.

Sometimes when using gtrace/0 and single stepping, the code will appear to skip ahead. To see the predicates that were skipped over look in the Call Stack dialog and click on the previous predicate call in the stack. You can continue clicking on the predicate calls in the stack all the way back to the start of the thread.

Note: When using any form of assert with any form of retract, and you are getting unrepeatable results, e.g. unit tests are now failing that use to pass and no change was made to the test or predicates used by the test, then shut down the top level with halt/0 and start again as the Prolog database is probably corrupted.

Advise: If you are not sure how a predicate works before using it, take the time to use it by itself to understand how it works so that you are not compounding your problem by introducing more unknowns. See: Assignment question about lists using findall and between

Advise: For brain storming ideas read " How to solve it : a new aspect of mathematical method" by George Pólya (WorldCat) (partial pdf)

Concolic Testing in Logic Programming

Interacting with a server

It is often useful to be able to interact with a running Prolog process, notably if it concerns an embedded Prolog engine or server process. This allows for inspecting the database, threads, etc., activate debug/3 statements or reload some code using make/0 without service interrupt. This can be achieved using library(prolog_server), providing a telnet connection to the server. Instead of telnet, one can also use nc (net cat). Be aware of the seurity implications! One day, we should provide this functionality on top of libssh and using a pseudo tty such that we can deal with command line editing and ^C.


This idea has one foot in debugging and one foot out so it is included.

Part of debugging is trying to understand which of different strategies may be faster. To make this easier to determine is first_solution/3.

See: Goal Evaluation with Timeouts


Use fuzzing.

5 Likes