# Does PLDoc have a way to incorporate unit tests akin to Python's doctest?

**URL:** <https://swi-prolog.discourse.group/t/does-pldoc-have-a-way-to-incorporate-unit-tests-akin-to-pythons-doctest/999>\
**Category:** Help!\
**Created:** [July 27, 2019, 8:25am UTC](https://swi-prolog.discourse.group/t/does-pldoc-have-a-way-to-incorporate-unit-tests-akin-to-pythons-doctest/999 "2019-07-27T08:25:46Z")\
**Posts on this page:** 9\
**Page:** 2

<div class="post-metadata">

**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\
**Post date:** [July 31, 2019, 11:55am UTC](https://swi-prolog.discourse.group/t/does-pldoc-have-a-way-to-incorporate-unit-tests-akin-to-pythons-doctest/999/22 "2019-07-31T11:55:52Z")

</div>

Just some remarks. The PlDoc system allows extracting the comments from loaded code using [doc\_comment/4](https://www.swi-prolog.org/pldoc/doc_for?object=pldoc_process%3Adoc_comment/4) or as found through the cross referencer using [xref\_comment](https://www.swi-prolog.org/pldoc/doc_for?object=prolog_xref%3Axref_comment/4) for not-loaded code. The [doc\_wiki](https://www.swi-prolog.org/pldoc/doc/_SWI_/library/pldoc/doc_wiki.pl) library translates the documentation to a Prolog term, from which you can extract the right subterm.

The code for the LPN (Learn Prolog Now) [proxy](https://github.com/LearnPrologNow/lpn-swish-proxy) contains a lot of useful stuff to analyse code fragments.

---

<div class="post-metadata">

**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\
**Post date:** [July 31, 2019, 12:46pm UTC](https://swi-prolog.discourse.group/t/does-pldoc-have-a-way-to-incorporate-unit-tests-akin-to-pythons-doctest/999/23 "2019-07-31T12:46:21Z")

</div>

In looking at the LPN proxy code and thinking about one of the goals of the original question,

> [@joeblog](#):
>
> it leaves helpful examples of how to use it in its wake.

it is just a small intuitive leap to realize that the same concept of extracting source code of LPN to generate working SWISH consoles could be applied to the SWI-Prolog documentation. So instead of just adding example source code to the SWI-Prolog documentation which would be read as static code, instead it could have the gear icon added and then a user could click on the gear to launch a SWISH console and play with the predicate. That would be nice.

* * *

Details for those not up on this concept.

Originally there was a [paper](http://www.dbnet.ece.ntua.gr/~adamo/csbooksonline/prolog-notes.pdf) for learning Prolog called `Learn Prolog Now`.

That evolved into a poplar web site: [LPN](http://www.learnprolognow.org/)

Then an enhanced [version](https://lpn.swi-prolog.org/lpnpage.php?pageid=online) was created ([Notice](https://www.swi-prolog.org/news/9a17c93e-6050-11e4-b1ed-00144fee017c)) that scans the LPN HTML pages for Prolog source code and converts the Prolog source code into an active component with a link, shown as a gear

![image](https://global.discourse-cdn.com/free1/uploads/swiprolog/original/1X/d3f1856f39cc2c9d90653a8c5712c30fa954dc5c.png)

that converts the source code into a SWISH console.

The code that scans the HTML is in [lpn-swish-proxy](https://github.com/LearnPrologNow/lpn-swish-proxy)

* * *

Example from enhanced LPN, [5.3 Arithmetic and Lists](http://lpn.swi-prolog.org./lpnpage.php?pagetype=html&pageid=lpn-htmlse20)

So this Prolog code in the original LPN site

```prolog
len([],0).
len([_|T],N) :- len(T,X), N is X+1.

```

becomes this in the enhanced web site

![image](https://global.discourse-cdn.com/free1/uploads/swiprolog/original/1X/936998f815d1671a46eac9abddfeb2150889aaa4.png)

and when the gear is clicked

 ![image](https://global.discourse-cdn.com/free1/uploads/swiprolog/original/1X/0227a448683aab748f289ea4d563eb5b5018e160.png)

the Prolog source code is loaded into a SWISH console ready to be run.

 ![image](https://global.discourse-cdn.com/free1/uploads/swiprolog/original/1X/7526f943b27a1249e16dd275fbccef0ab7fa1d5c.png)

---

<div class="post-metadata">

**Author:** ![peter.ludemann](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/peter.ludemann/32/48_2.png) [@peter.ludemann](https://swi-prolog.discourse.group/u/peter.ludemann)\
**Post date:** [July 31, 2019, 4:12pm UTC](https://swi-prolog.discourse.group/t/does-pldoc-have-a-way-to-incorporate-unit-tests-akin-to-pythons-doctest/999/24 "2019-07-31T16:12:22Z")

</div>

> [@joeblog](#):
>
> Python is unfortunately a strong argument against my idea: it has supported doctest since way back but hardly anyone uses it because test driven development has to be drummed into programers from an early age.

I worked 10 years at Google, which heavily emphasizes testing (from unit tests through system tests), and doctest is not used – but an extended form of Python unittest _is_ heavily used. I think that a major reason for not using doctest is that often some test setup/teardown is needed, and doctest is not very good for that. (At Google, the same test framework also supports C++ and Java, and for those, doctest is even less useful.)

It would be nice to have a tool that validates documentation examples. The extraction part should be quite easy; but integrating with plunit seems non-trivial (I recall a moderate amount of effort was put into such a tool for documenting IBM Prolog).

---

<div class="post-metadata">

**Author:** ![PaulBrownMagic](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/paulbrownmagic/32/211_2.png) [@PaulBrownMagic](https://swi-prolog.discourse.group/u/PaulBrownMagic)\
**Post date:** [July 31, 2019, 4:31pm UTC](https://swi-prolog.discourse.group/t/does-pldoc-have-a-way-to-incorporate-unit-tests-akin-to-pythons-doctest/999/25 "2019-07-31T16:31:42Z")

</div>

I’m just looking at [`sub_string/5`](https://www.swi-prolog.org/pldoc/doc_for?object=sub_string/5) in the docs, which as an example could become:

```
name_value(String, Name, Value) :-
        sub_string(String, Before, _, After, "="), !,
        sub_string(String, 0, Before, _, NameString),
        atom_string(Name, NameString),
        sub_string(String, _, After, 0, Value).

?- name_value("foo=bar", foo, "bar").

```

More challenging to parse and very useful for examples, which leads me to the conclusion that Eric is onto a better idea with adapting the LPN SWISH thing to the docs.

---

<div class="post-metadata">

**Author:** ![joeblog](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/joeblog/32/122_2.png) [@joeblog](https://swi-prolog.discourse.group/u/joeblog)\
**Post date:** [August 1, 2019, 7:53am UTC](https://swi-prolog.discourse.group/t/does-pldoc-have-a-way-to-incorporate-unit-tests-akin-to-pythons-doctest/999/27 "2019-08-01T07:53:31Z")

</div>

doc\_comment/4 looks interesting. If only it had some illustrative examples of how to use it 🙂

One of the paradoxes I’ve noticed is the worst documented part of most programing languages is how to do documentation.

This is why I’m a fan of MIT’s How to design Programs philosophy: When people learn a new programing language, instead of getting taught “Hello World” they should start with a documentation template to get clear in their head what the purpose of their predicate/function they are writing is, what it will consume, what it will produce, with some examples which initially act as stubs but in due course (possibly after many auxilary functions have been written), do the job.

---

<div class="post-metadata">

**Author:** ![pmoura](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/pmoura/32/11_2.png) [@pmoura](https://swi-prolog.discourse.group/u/pmoura)\
**Post date:** [August 1, 2019, 8:27am UTC](https://swi-prolog.discourse.group/t/does-pldoc-have-a-way-to-incorporate-unit-tests-akin-to-pythons-doctest/999/28 "2019-08-01T08:27:59Z")

</div>

> [@peter.ludemann](#):
>
> When I worked on IBM Prolog, we had a script that took the system documentation, extracted all the examples, and ran them to verify that they all worked. I don’t recall anything special in the markup language. But this wasn’t for literate programming(\*) - it was for verifying that the documentation was correct.
> 
> I also found this:  
> [http://mirrors.ibiblio.org/CTAN/macros/latex/contrib/gene/pl/pl.pdf](http://mirrors.ibiblio.org/CTAN/macros/latex/contrib/gene/pl/pl.pdf)  
> [CTAN: /tex-archive/macros/latex/contrib/gene/pl](https://ctan.org/tex-archive/macros/latex/contrib/gene/pl)  
> and @pmoura has a bit to say here: [https://dtai.cs.kuleuven.be/projects/ALP/newsletter/archive\_93\_96/net/techniques/lit.html](https://dtai.cs.kuleuven.be/projects/ALP/newsletter/archive_93_96/net/techniques/lit.html)

That’s a really old reference (by Internet standards) 🙂 But the idea remains the same: in Logtalk, all data relevant for documentation (and other tools) is saved at compile time (assuming the `source_data` flag is on) and made available using the reflection API. This includes the predicate examples if any (no parsing required). It’s thus quite easy to programmatically access the examples and construct test queries. But this is a bit tricky to _automate_ as there might be a need to e.g. setup a query context or clean up after it (I think that was already mention elsewhere in this thread). The examples may also be included in e.g. a protocol or category documentation instead of an object documentation.

P.S. Are the IBM Prolog manuals and other documentation publicly available? My understanding is that it’s a discontinued system.

---

<div class="post-metadata">

**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\
**Post date:** [August 1, 2019, 2:30pm UTC](https://swi-prolog.discourse.group/t/does-pldoc-have-a-way-to-incorporate-unit-tests-akin-to-pythons-doctest/999/29 "2019-08-01T14:30:14Z")

</div>

It is all not that different from SWI-Prolog’s infrastructure. Documentation is comment, so if you do nothing it is lost. If you use `:- doc_collect(true).` it collects the structured comment while loading the sources. That parses the mode declaration and makes these available through `library(pldoc/doc_modes)`. It associates the remainder of the documentation as a string with the documented predicates, making this raw material available through `doc_comment/4`. Next, `library(pldoc/doc_wiki)` can be used to turn this into a structured Prolog term. In the normal documentation setup, `library(pldoc/doc_html)` turns this into HTML and `library(pldoc/doc_latex)` into a LaTeX document.

The whole things is pretty transparent. Most functionality is only documented using reference style documentation as PlDoc in the sources. So, you need a little look around in the source to find the relevant APIs.

---

<div class="post-metadata">

**Author:** ![swi](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@swi](https://swi-prolog.discourse.group/u/swi)\
**Post date:** [August 1, 2019, 4:39pm UTC](https://swi-prolog.discourse.group/t/does-pldoc-have-a-way-to-incorporate-unit-tests-akin-to-pythons-doctest/999/30 "2019-08-01T16:39:08Z")

</div>

Best way to get examples on how to use a predicate is to do something like this:

```prolog
$ cd <swi-prolog source dir>
$ grep -r -A 3 -B 3 'doc_comment' .                 

```

---

<div class="post-metadata">

**Author:** ![peter.ludemann](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/peter.ludemann/32/48_2.png) [@peter.ludemann](https://swi-prolog.discourse.group/u/peter.ludemann)\
**Post date:** [August 9, 2019, 3:51am UTC](https://swi-prolog.discourse.group/t/does-pldoc-have-a-way-to-incorporate-unit-tests-akin-to-pythons-doctest/999/31 "2019-08-09T03:51:20Z")

</div>

> [@pmoura](#):
>
> P.S. Are the IBM Prolog manuals and other documentation publicly available? My understanding is that it’s a discontinued system.

I tried finding the manuals but they seem to have disappeared. It is a long-discontinued system. (A long sad story … amongst other things, IBM Marketing pushed some inferior products; there were a number of bad product decisions due to IBM SAA AD/Cycle and PS/2; etc., etc.)

[Previous page](https://swi-prolog.discourse.group/t/does-pldoc-have-a-way-to-incorporate-unit-tests-akin-to-pythons-doctest/999.md?page=1)
