Unclear use of timeout in unit tests

Help!
1

Hi

first of all, thank you for adding timeouts to the PLUnit test library.

I’m using: SWI-Prolog version 9.1.xxx

It is unclear to me how to use the set_test_options/1 predicate in conjunctions with timeouts.

My code looks like this:


:- use_module(library(plunit)).

:- begin_tests(timeout_tests).

test(fail_waking) :-
    sleep(30).

:- end_tests(timeout_tests).

:- set_test_options([timeout(10)]).

However, the test succeeds after 30 seconds. If instead I do not use the set_test_options/1 predicate, but instead I add the [timeout(10)] to the begin_tests(...), I get the desired result.

I am missing something. (Also, the timeout option is not listed among those that can be specified for begin_tests or test).

All the best

Marco

2

Works fine for me. I can imagine some issues with development environments that use threads. ?- set_test_options([timeout(2)]). in the end is the same as ?- set_prolog_flag(plunit_timeout, 2). If you execute this in a thread, for example by loading the file in a thread, it has no effect in the main thread. This issue affects several development environments when loading a file through the editor. The native PceEmacs as well as the Emacs sweep mode, Eclipse PDT plugin all use threads.

Prolog flags use copy semantics wrt. threads, i.e., setting a flag sets it only for the calling thread and new threads copy (actually implemented using copy-on-write) the flags from their creator.

3

Hi Jan

that is exactly how I was using it.

I hit the “Compile buffer” menu and it did not work.

This behavior wrt threading should be documented, as it is surprising as it is.

In any case it does not work if you “Consult” the file from the menu and then you issue “run_tests” at the prompt, especially if one of your tests is raising some other exception.

all the best

Marco

4

Better fixed, although I’m unsure whether that is possible. It probably is for most individual cases.

That also does its work from a background thread. Should work when using

?- [myfile].

from the prompt. It should also work if you open the app using the file, i.e., with the proper association by double clicking the .pl file in the finder/explorer/…

5

Hi.

it is now 2025 and I must report that the situation has not changed.

I believe this is important. Timeouts is testing is very important (at least to me).

All the best

Marco

6

If timeouts would not work, it is worrying. As is, it is merely a matter of setting them in a place that does work. Setting them inside a test file is dubious anyway as it will affect the timeout in all tests. Note that it should work to run

?- run_tests(all, [timeout(10)]).
1 Like
7

Hi

sorry, I am back after some months. Meanwhile, I tried to use the Linux command timeout, but the granularity is to large. I am experimenting with on_signal but the logic fails as run_test does not seem to have a way to save the summary report. And to restart.

Using run_tests(all, [summary(Report), timeout(10)]) does not work in my setting, for no clear reason. No timeout is properly issued. I run swipl -g ‘run_tests_stuff.pl’ -t halt.

All the best

Marco

8

Well, it works here. If you want this resolved, please create a complete example with instructions for reproducing.

1 Like
9

Hi @jan

after much soul searching, I got it to work.

The issue is that I was issuing something like

swipl -g ‘consult(“file.plt”), consult(“running_file.pl”) -t

with
:- run_my_tests.

In the running_file.pl that invoked run_tests.

This somehow messed up something: I presume something with threading.

It did take me a while (as you see the threads, some years) to figure out that

swipl -g run_my_tests -t file.plt running_file.pl

was the way to fix everything.

Sorry

Marco

10

You should not use directives to run your code like this. These are executes when encountered while loading the file and loading the file is protected against signals to avoid ending up with a half loaded file.

Indeed, use -g goal or

:- initialization(Goal, main).

This (single) Goal is executed after all code is loaded. Note that ISO defines :- initialization(Goal). that is executed after the file is loaded. I think that should work as well, but only for the toplevel file. If this appears in a file loaded from some other file the directive is executed as part of loading the containing file.

1 Like
11

Hi

yes. That explains it. Our replies crossed.

Now (after a few years), I know better.

Thank you for your patience.

All the best

Marco