Once code gets sizable test cases are often added.
During the early stages of development, it might be beneficial to put the test cases after the code using :- begin_tests(<Name>).
and :- end_tests(<Name>).
When preparing the code for an initial release moving the test cases into a separate file is common.
Currently I know of two common ways to do this with Prolog.
- Use a
plt
file type. (ref) - Wrap the test cases with a module and add a predicate to call run_tests/1. This is prevelent in the SWI-Prolog Tests directory on GitHub
Example code (ref)
:- module(test_list,
[ test_list/0
]).
:- use_module(library(plunit)).
:- use_module(library(terms)).
:- use_module(library(apply)).
test_list :-
run_tests([ memberchk
]).
:- begin_tests(memberchk, []).
test(memberchk, X == y) :- % Verify unbinding after failing PL_unify()
memberchk(f(X,a), [f(x,b), f(y,a)]).
:- end_tests(memberchk).
I suspect that wrapping the test cases with a module has a side benefit for automation and continuous integration; I am not sure of the details of how the test cases are activated during the CI check.
The way it is done with SWI-Prolog on GitHub is the way I plan to go unless there is some serious problem with that pattern.
Just looking for ideas and pros and cons of the various ways for publishing code with test cases before I am deeply down a path that I don’t want to have to spend time redoing if I choose wrong.
EDIT
After trying both ways and reading Controlling the test suite in the documentation, am considering that using plt
files is the better way to go, but as Jan W. mentions
Possibly we should have a library that implements this running all tests from some directory.
which should be relatively easy to implement.
Granted it will quite some time before I have so much code to publish for one project that it needs multiple directories.
Two specific reasons for using plt
files
The test files can be easily loaded if
:- load_test_files([]).
is added to the source code file for the predicates.
Then when the source code file for the predicates is consulted, the test are automatically loaded and then run_tests/0 works the same as if the test were in the same file as the the predicates, e.g.
Welcome to SWI-Prolog (threaded, 64 bits, version 8.3.4-19-gefb65916d)
?- working_directory(_,'C:/Users/Groot/Documents').
true.
?- [rfc5234].
true.
?- run_tests.
% PL-Unit: rfc5234 ...... done
% All 6 tests passed
true.
While I have not tried this, this should apply
set_test_options(+Options)
Option: load(+Load)
Finally, when using the default value
normal
, tests are loaded if the code is not compiled with optimisation turned on.