Does PLDoc have something akin to Python’s doctest, ie a way to include simple examples when describing what a function does, which double as unit tests? In prolog, this might look something like this %% sort(+List, -Sorted) % True if Sorted can be unified ... % % ?- sort([4,3,1,2], Sorted). % Sorted = [1, 2, 3, 4]. % .... The example in the comment would be ignored unless an option (in Python’s case -v) is used when the script is run. I think having a feature like this is nifty for at least three reasons: It encourages test driven development, leading to better work methods when creating brand new code. Once the code is done and hopefully used by other people, it leaves helpful examples of how to use it in its wake. (I’m sure I’m not the only one who finds the lack of examples in SWI Prolog’s documentation frustrating). When the code inevitably has to get refactored, the documentation acts as a test framework to ensure old stuff hasn’t been broken while introducing new stuff.

[image] joeblog: improve SWI Prolog’s “educability” for novice programmers. A few years ago when I was helping others learn Python I used Runestone Academy which hosts interactive text books for learning Python among other things. The site had many more types of widgets than I have seen in any other type of online textbook. Again I am not asking that these be added to SWISH, but just showing them as an example of what can be done by extending the usefulness of an online notebook with HTML and CSS.

Hi Eric Sadly, only converted Prolog enthusiasts like us are going to make the effort to play with Swish, find examples in git, and search the discussion forums. Most people are going to turn to the official documentation as their first port of call – and it will be their last if it doesn’t answer their question. While you can include examples in the documentation using markdown’s example(Arg1,...) :- .... the age old problem remains that coders don’t enjoy writing documentation and put minimal effort into it. A trend I’ve noted is the more a language puts an effort into automating its documentation and making it useful, the longer it lives and prospers. Relying on people to keep cutting and pasting examples from separate unit test files inevitably results in either no illustrative examples in the documentation, or out-of-date examples.

[image] joeblog: Sadly, only converted Prolog enthusiasts like us are going to make the effort to play with Swish, find examples in git, and search the discussion forums. Most people are going to turn to the official documentation as their first port of call – and it will be their last if it doesn’t answer their question I partly agree with that and partly do not agree with that. :slightly_smiling_face: A person who has to accomplish a task with Prolog and is just learning, e.g. student, as you note, will likely turn to the documentation first, but then many of them will turn to Google at which point they may/will find many similar answers at StackOverflow, and then may or may not ask a question there. I do agree that the documentation should be the first place with the best reference style examples, but that How To examples for more complex task should not be in the documentation next to a predicate but somewhere else. [image] joeblog: coders don’t enjoy writing documentation I think some do, but like me, it is not writing the documentation that is the problem, the problem is keeping it in sync with the code, especially if it has a nice graphic which I like to do for the more complex concepts. The other problem with documentation for code is, when do you do the documentation? I often make so many changes at the start that if I took the time to write the documentation as I write the code it would take me 3x or more before completing the code because the I would also be updating the documentation. [image] joeblog: A trend I’ve noted is the more a language puts an effort into automating its documentation and making it useful, the longer it lives and prospers. That one I will disagree with because there are many languages that have had long life and that don’t/didn’t have automated documentation. When I started code it was on key punch machines and languages like C, Fortran and Basic existed and are still used commonly today, but other languages also existed and seem to have fallen out of favor. And yet there are some modern languages that have had automated documentation and are dying, e.g. Objective-C There are many factors that influence if a programming language will live and prosper, and IMHO the backing of a major company like Google, Microsoft, or Oracle has more to do with it than the documentation. One trick I have learned over the years for some of the more basic Prolog predicates to learn them is to just run the most general query. Just a few days ago I needed to understand sub_string/5 and while the docs kind of made sense, the part that had me confused was the type of values for each parameter, which ones were string and which ones were integer. So using this query ?- sub_string("abc",Before,Length,After,Substring). Before = Length, Length = 0, After = 3, Substring = "" ; Before = 0, Length = 1, After = 2, Substring = "a" ; Before = 0, Length = 2, After = 1, Substring = "ab" ; Before = After, After = 0, Length = 3, Substring = "abc" ; Before = 1, Length = 0, After = 2, Substring = "" ; Before = Length, Length = After, After = 1, Substring = "b" ; Before = 1, Length = 2, After = 0, Substring = "bc" ; Before = 2, Length = 0, After = 1, Substring = "" ; Before = 2, Length = 1, After = 0, Substring = "c" ; Before = 3, Length = After, After = 0, Substring = "". and then a quick reformat of the output with an added note gave me Before Length After Substring 0 0 3 "" 0 1 2 "a" 0 2 1 "ab" 0 3 0 "abc" 1 0 2 "" 1 1 1 "b" 1 2 0 "bc" 2 0 1 "" 2 1 0 "c" 3 0 0 "" Before + Length + After = length(String) and then in made perfect sense. Maybe one of the first things that new people, (students) need to learn when learning Prolog is how to run the most general query and then make sense of what the result(s) means to help them understand the predicate. I personally know that if I take time to write a note down rather than just reading it, or as many young people do today of just taking a picture with their phone, I remember it much more, and recall it faster and in more detail. Maybe the problem is not to teach people, but how to better teach virtual assistants (Alexa, Siri, Google Assistant, Cortona), as that seems to be where the newer generation goes for answers. :slightly_smiling_face: Not that I agree with that.

That’s a great idea. Plus, the ```test … ``` notation would likely do it without touching PLDoc, for example: %% sort(+List, -Sorted) % True if Sorted can be unified ... % % ```test % ?- sort([4,3,1,2], Sorted), % Sorted = [1, 2, 3, 4]. % ``` sort(In, Out) :- magic(In, Out). If we run PLDoc on that it should just see the test case as a code block and print it in the docs. PLDoc has nothing to run, it’s not executable, it’s just a printed example. What we’d want then is something that could parse these test cases out of the doc-string, consult the file in question, and then call the test cases. It could be quite distinct from PLUnit and even SWI-Prolog. But parsing is my biggest blind-spot in CS…

When I worked on IBM Prolog, we had a script that took the system documentation, extracted all the examples, and ran them to verify that they all worked. I don’t recall anything special in the markup language. But this wasn’t for literate programming(*) - it was for verifying that the documentation was correct. I also found this: http://mirrors.ibiblio.org/CTAN/macros/latex/contrib/gene/pl/pl.pdf https://ctan.org/tex-archive/macros/latex/contrib/gene/pl and @pmoura has a bit to say here: https://dtai.cs.kuleuven.be/projects/ALP/newsletter/archive_93_96/net/techniques/lit.html (*) I once had a 100-line script (written by someone at Quintus - perhaps Tim Lindholm) that generated nice LaTex from code that used special comments. But it didn’t do anything special with test cases – they were just like any other code. [Literate Programming for Prolog is a lot easier than other languages because Prolog allows predicates to be defined in almost any order (directives can be a bit of a problem).

A key thing Jan said in an earlier post: The most promising part I see is that it is now not uncommon for examples in the documentation to be incorrect, often even simply containing syntax errors (because the IDE doesn’t check syntax inside comments). If PlDoc would know an example is supposed to be valid and complete it could evaluate it and complain. I think there is a lot of value in checking that examples in documentation don’t have typos and actually run. Combining illustrative examples and documentation would lead to good habits in unit tests, improving development besides the quality of documentation. But am I going to be the person to actually implement this? Uhm, I’m just a hobbyist programer, journalist by trade…

OK, so I’ve started playing around with a program that could possibly do this. First I tried to do it in SWI-Prolog, but my attempts at reading a file as plain text and parsing out code blocks in comments were just miserable. If someone can provide a DCG or something for this then I’m happy to continue it in SWI-Prolog. Again, I’m not good at parsing! In the meantime, I’m building a prototype in Python3. My approach is this: Read in the file Parse out the code blocks in comments marked ```test using Regex Write out a .plt file Call run_tests and let SWI-Prolog do the testing Delete .plt file by default It’s not perfect, it’s not robust, it’s not in SWI-Prolog (so adds a dependency). But it’s enough to raise a question: PLUnit uses options in the tests (e.g. test(foo, [nondet]) :- bar.), these should be included in the code comment block, so how do we like: % ```test([Options]) and the program names the test with the line in the file Or % ```test(foo, [Options]) and we just put that into the test file as the head I really like the second one, it would mirror PLUnit so closely, we’d just have to shift things round a bit. But % ```test renders fine in PLDoc, as soon as we add a space or bracket into that line we’ll need to change PLDoc to handle it. An alternative proposal, we don’t provide the full power of PLUnit assuming this usecase is for examples only that will succeed and unify all their test arguments in the code-block so the user can see everything clearly. In this case we can parse: %% sort(+List, -Sorted) is nondet. % True if Sorted can be unified, either % by magic or science % % ```example % ?- sort([4,3,1,2], Sorted), % Sorted = [1, 2, 3, 4] ; % false. % ``` sort(In, Out) :- magic(In, Out) ; science(In, Out). Grabbing a nondet option from the first line and assuming no options otherwise. Nothing to change in PLDoc and closer to the queries typically used in the docs.

I don’t know why it doesn’t show in the docs, but there is xref_comment/[3,4] which parses the comment for you. If you have the source code like this: %% sort(+List, -Sorted) is nondet. % True if Sorted can be unified, either % by magic or science % % ```example % ?- sort([4,3,1,2], Sorted), % Sorted = [1, 2, 3, 4] ; % false. % ``` sort(In, Out) :- magic(In, Out) ; science(In, Out). in a file called /tmp/s.pl, you can run the following query: 1 ?- S='/tmp/s.pl', xref_source(S), xref_comment(S, Head, Summary, Comment). S = '/tmp/s.pl', Head = user:sort(_13402, _13404), Summary = "True if Sorted can be unified, either by magic or science.", Comment = "%% sort(+List, -Sorted) is nondet.\n% True if Sorted can be unified , either\n% by magic or science\n%\n% ```example\n% ?- sort([4,3,1,2], Sorted) ,\n% Sorted = [1, 2, 3, 4] ;\n% false.\n% ```" . So the comment is returned as a string which you can parse with DCGs or with regular expressions ( library(pcre) ).

Python is unfortunately a strong argument against my idea: it has supported doctest since way back but hardly anyone uses it because test driven development has to be drummed into programers from an early age. Ideally getting working illustrative examples into documentation has to be part of the basic development cycle. And it can easily be done now by cutting and pasting relevant tests from the *.plt files, but the existing documentation shows this involves too much admin for many developers. Unfortunately, creating a Python script to do this will be no easier than cutting and pasting, so I’m a bit of a “doomster and gloomster” (to quote Boris Johnson) about it getting used much.

Sweet! Thank you swi! I reckon I can handle that in SWI-Prolog!

Does PLDoc have a way to incorporate unit tests akin to Python's doctest?

Help!

Topic		Replies	Views
Add working examples to PLDoc Request For Comments	4	1319	July 31, 2019
Towards an infrastructure for examples Request For Comments	20	1851	December 17, 2020
Slight documentation bug on SWI Prolog website SWI-Prolog web site and services bug	2	516	March 11, 2020
Some notes on using the SWI-Prolog "ecosystem" I've written General	1	411	September 11, 2019
A page for beginners to add to SWI-Prolog internet site Request For Comments	34	2052	June 19, 2021

Does PLDoc have a way to incorporate unit tests akin to Python's doctest?

Related topics