Coverage info mysterioulsy disapear

Help!
1

I’m using: SWI-Prolog version 8.5.16 for x86_64-linux. I want the code to show the Test Coverage Report of a file containing predicates called by the test suite for any test that calls predicates defined in the “predicates under tests” file.

What I’m getting is the Test Coverage Report but for only when a specific test is present in the test file. If the first test is removed the other simiilar tests are not able to make the file visible in the report.

The Seven Steps to Reproduce:

  1. Copy the Predicates under test into a file called example.pl
  2. Copy the Test code into a file called example.plt
  3. Execute: swipl -g "load_test_files([]),show_coverage(run_tests,[dir(cov)]),halt" examples.pl
  4. Observe the examples.pl present in the Coverage by File report
  5. Comment the first test in the test file
  6. Execute: swipl -g “show_coverage(run_tests),halt” examples.pl
  7. Observe the examples.pl missing in the Coverage by File report

Predicates under test:

:- module(examples,[hello_coverage_1/2,hello_coverage_2/2,hello_coverage_3/2]).
:- use_module(library(http/html_write)).
:- [library(dcg/basics)].

html_write:expand(_) --> !.
increment(E, N):-N is E + 1.
a_number(N) --> integer(N).

hello_coverage_1(X, Y) :- phrase(html([X]),Y,[]).
hello_coverage_2(X, Y) :- phrase(examples:a_number(Y), X).
hello_coverage_3(X, Y) :- increment(X, Y).

Test code:

:- module(test_examples, [test_examples/0]).
:- use_module(examples).
test_examples :- run_tests.

:- begin_tests(examples).

test(hello_coverage_1) :- hello_coverage_1(true, []). % This is the only test that makes the coverage data visible
test(hello_coverage_2) :- hello_coverage_2(`4`, 4).
test(hello_coverage_3) :- hello_coverage_3(2, 3).

:- end_tests(examples).

when I run:

swipl -g "load_test_files([]),show_coverage(run_tests,[dir(cov)]),halt" examples.pl

I can see coverage information:

==============================================================================
                               Coverage by File                               
==============================================================================
File                                                     Clauses    %Cov %Fail
==============================================================================
/usr/lib/swi-prolog/library/ansi_term.pl                       8    37.5  12.5
...uez/Code/Learn/abbreviated_dates/prolog/examples.pl         1   100.0   0.0
/usr/lib/swi-prolog/library/plunit.pl                          3    33.3   0.0
==============================================================================

If I comment the test hello_coverage_1 then the coverage info for the example file disapear:

==============================================================================
                               Coverage by File                               
==============================================================================
File                                                     Clauses    %Cov %Fail
==============================================================================
/usr/lib/swi-prolog/library/ansi_term.pl                       8    37.5  12.5
/usr/lib/swi-prolog/library/plunit.pl                          3    33.3   0.0
==============================================================================

Looks like there is something special with the first predicate that is not present in the others.

3

I think this is caused because the show_coverage/1 only reports on files that have some coverage. Files that are not touched at all are not mentioned.

1 Like
4

Thanks so much for your timely response! How may I learn more about the mechanism of touching files in the Swi-prolog pl-unit system?

Also: What makes the first test capable of making coverage in the file? What happens with the last two test that they cannot do the same?

5

I might be wrong. With touch I mean some code in the file is executed. That also seems to be the case for the other tests. It also claims there is only one clause. That could be a clue. Using a dir(cov) option to show_coverage/2 also only annotates the html_write:expand(_) --> !. clause. Could be a bug :frowning:

1 Like
6

Thanks so much for your response Jan!

I have updated the experiment description with a test that is closer to the one that actually works (Using DCG). Unfortunatelly the problem is still there.

Where can I find working examples of the Code Coverage functionality that I can use to better understand the problem?

7

As I feared, it was a bug in source_file/2 that is used to find the predicates defined in some file. Pushed a fix. If you compiled from source, just pull and rebuild.

1 Like
8

Awesome! Thanks a lot Jan!

9

As you know finding useful real world examples of code coverage is hard.

Care to share your working example(s) for others that follow?

1 Like
10

Hi Eric!

Thanks for your comment! As you said: finding useful real world examples of code coverage is hard. But why? If we “think of the test suite like a parachute” then finding useful real world examples of code coverage would be as easy as finding parachutes without holes. :wink:

I’ve just got the example described above :point_up: in this thread “working” after compiling from source as adviced by Jan. Here is how it looks like after removing a few lines for the sake of clarity:

swipl -g "load_test_files([]),show_coverage(run_tests,[dir(cov)]),halt" prolog/examples.pl
% PL-Unit: examples .. done
% All 2 tests passed

==============================================================================
                               Coverage by File                               
==============================================================================
File                                                     Clauses    %Cov %Fail
==============================================================================
uez/Code/Learn/abbreviated_dates/prolog/examples.pl         6    66.7   0.0
ez/Code/Learn/abbreviated_dates/prolog/examples.plt         5    80.0   0.0
==============================================================================
1 Like
11

This directory now contains annotated versions of the source files that indicate exactly what has been called and how often. The annotated files use ANSI color sequences to make the annotations more readable. Normally, less should show these.

1 Like
12

Fantastic! When would be possible to have this feature available in the SWI-Prolog stable PPA channel?

13

Current plan is to start a new stable cycle no later than end of October.

1 Like
14

Super! :star_struck: Thanks a lot Jan!