Using markdown with pldoc to generate manual content

I’ve gone through the process of writing a replacement for the debugger overview using “regular” markdown as understood by the StackEdit editor and trying to integrate it into the build. Mostly it went pretty smoothly. I wanted to write it up here to save others the time and also to point out a bug and missing feature that I hit.

  • Prolog Keywords: I originally surrounded keywords like trace/0 with backticks(``). Unfortunately, this meant pldoc didn’t recognize them as prolog keywords and link them, so I had to remove all of the backticks. However, I did need to leave them on non-built-in predicates so they would look right. Due to the default formatting of the links in html, this meant that built-in predicates and example predicates look different (notice that trace/0 and noun(X, rock) are formatted differently in the image):

    • suggestion: allow `` around a built-in predicate to get formatted like back ticks but also link
  • I had to add section IDs to each section using {#} so they could be linked to. Like this:
    ## The Byrd Box Model And Ports {#byrd-box-model}

  • Some latex commands (like \cite) are missing from markdown, so I just had to manually write in the URL, which is not good since it means the local docs now link to the website, etc.

    • suggestion: have a way to embed latex commands in markdown. The only one I needed was \cite so maybe there aren’t a lot?
  • Links: To make sure the build system linked everything up properly, I needed to link to sections using the [](#section) syntax. However, instead of creating a link, it actually puts the section name in () like this: [the debugger section](#debugger) generates debugger section (section 4.39) instead of just hot linking the text. Maybe that is by design?

  • Putting capitalized text in backticks generated italics in HTML and non capitalized did not. Seemed odd.

  • Creating a markdown table like this:
    | | | |
    |-----------|-|------------------------|
    | **Abort** |a| Abort Prolog execution.|
    generated a table like this in HTML (note the missing line on the top):
    image

  • For some reason set_breakpoint/4 and spy/2 were not found in the docs and automatically linked to. Maybe a problem with my local build environment?

The only two issues that seem really problematic are the odd table border and the lack of a way to do \cite “symbolically”. Overall, it worked pretty well for this doc. And afterward the only thing that was odd if I round tripped it back to the StackEdit editor was the section IDs showing up in {}.

2 Likes

Thanks for writing this up.

Sounds good, also to improve reusing the markdown.

You don’t really have to but you get a warning. The section IDs are used to generate the HTML file names and the node ideas for representing the entire manual hierarchically. Guess we could generate them automatically. Doing it by hand creates nice concise labels though.

I’m less enthusiastic about this. I do not want to tie the markdown to a particular backend. Note that currently there are two backends: HTML and LaTeX. The HTML is used if PlDoc is used directly on Prolog code. The LaTeX is used for the manual (which is then subsequently translated into HTML). A little search only raised R markdown. Doesn’t look too bad though. Possibly for now only add [@cite1;@cite2;…] to map to \cite{cite1,cite2,…}?

Guess that if there is a text we could omit adding the section number. Probably we would need an abstraction over LaTeX references to allow adding the text for the link?

As is, `....` considers the content Prolog code and tries to format it nicely. HTML is a Prolog variable :slight_smile: Double backtick (how do I get that literally in here?) considers the content code without any assumptions on the language.

The first two lines are not needed. Something in the transformation apparently gets upset from the inserted empty cells of the first line. Turns out PlDoc doesn’t like a |---|--- line to start with. Maybe that should be fixed? That would also allow adding column alignment.

I’ll wait for your pull request.

OK, pull request submitted!

Yeah, I think requiring them to be added manually was fine. The warnings notified me that I should do it.

That seems like a good localized fix. Note that pandoc has a more extensive spec for their version of this extension, and their support probably gives it some legs towards getting adopted more widely. Your short version seems like a simple subset of what they have.

I don’t know enough about latex and the build infrastructure to have an opinion on this. If you look at my pull request, the current approach isn’t really that bad, just different than I expected from “normal” markdown.

Good to know. That approach also does the right thing in my one test using StackEdit as well.

This one and the citation feature are the only two I think are important fixes. Supporting the (as far as I can tell pretty universally supported) markdown table format that allows a header and alignment row is pretty important. In fact, it is required by the three online markdown editors I’ve used and won’t render the table without them: github, dillinger, and stackedit. So, anyone writing markdown coming from those sources, at least, will hit this issue.

Here’s the spec for the GitHub table format, which is at least a subset of what the others use too

Pushed some stuff that should make your life easier:

  • [@cite1;@cite2;…] is supported. This is a limited version of the R Studio Markdown extension and is only provided for the LaTeX backend. No white space is allowed in the citation.
  • `name/arity` is handled the same as name/arity
  • ``name/arity`` translates to \nopredref{name}{arity}, i.e., a predicate indicator that is not a link to the manual. This should be used to refer to predicates in examples.
  • … flag `name` … creates a link to a Prolog flag if name is the name of an existing Prolog flag.
  • Enhanced table handling
    • Deal with a |---|... column specification, including alignment
    • An | | | ... line above the column specification (all empty cells) is removed.
    • A row above the column specification is translated into a header (HTML <th>, LaTeX \textbf(...} content)
    • The LaTeX route uses \tabularx to achieve flexible wrapping for columns holding long content.
1 Like

There seems to be a bug in table layout that leaves gaps in the vertical border rules if the last column is blank; you can see an example here: SWI-Prolog -- Manual

Is it possible to comment out a section of text? The usual “<!-- … ->” didn’t work (I tried a few variants, such as using , and they didn’t work either).

Is there any way to get fixed-font layout for source? It seems that &nbsp; doesn’t work; and the “==” layout uses a variable font.

Is there an easy way to get the CSS that http://swi-prolog.org uses? When I generate html using cmake, there’s an inline “” that contains the CSS; when I look at the page source at http://swi-prolog.org, I see something quite different.

Works fine in Firefox. Reproduces in Chrome. Now, who is right and is there a simple fix?

No. SWI-Prolog’s markdown does not allow for HTML in general. This is because we deliberately want to support other backends than just HTML.

? Code blocks using ``` or == guards should both work (prefer the first as that is the current de-facto standard). There are enough code blocks in the manual that show it works, including on the protobufs pages. I think there is no way to get consecutive spaces in inline code (using single or double backquote).

Not that easy I fear. You’ll need to install the website locally (clone GitHub - SWI-Prolog/plweb: SWI-Prolog web server sources for www.swi-prolog.org) and a bit fiddling. Eventually it might be a good idea to move some of the stuff from the website to PlDoc. As is, the website parses the HTML pages to extract predicates and (sub-)sections and repacks this HTML in the final page, applying its own style.

Warning: off-topic unsubstantiated rant. Chrome is cancer. It is also literally hitler. It is playing the “catch me if you can” game that Microsoft played while it was on top. By now it is clear (to me) that they purposefully break stuff just to make sure that the few people who still use Firefox give it up eventually.

Seems the real problem was a bug in the LaTeX to HTML translation that failed to generate a <td> for the last empty LaTeX cell. Note that docs are processed by converting everything to LaTeX (so we can make the PDF docs) and then from LaTeX to HTML for the online docs as well as help/1. Pushed a fix.

In Firefox, code blocks use a fixed font. In Chrome, it’s a variable-width font. :frowning:
(At least, on my machine)

It appears that the generated HTML is <pre class="code">, which seems to inherit from whatever the browser default is, and that’s different on Firefox and Chrome (and Safari and whatever-Microsoft-is-calling-IE-these-days, which both use the same rendering engine as Chrome, I think).

I was able to fix this to by adding font-family: "Source Sans Pro", Lucida Console, Courier, monospace, but then Firefox used an ugly fixed font (perhaps the swi server fixes the Firefox font?)
BTW, I noticed that a number of the .css files have font-family: Source Sans Pro, sans; – the “sans” should probably be replaced by “monospace”, even though that doesn’t seem to work as expected with Chrome.

PS: entering protobufs into the swi-prolog search bar finds the appendix for protobufs and doesn’t show the main entry.

Code blocks on e.g., SWI-Prolog -- Manual look fine in Chrome (well, Chromium as I only run open source on my development machine). What exactly are you looking at?

The main entry is called “Google’s Protocol Buffers Library”, so that does not match. Still, the result page looks good enough for people to find it.

Attached are two screen shots – you can see that the Chrome version (the first one) doesn’t line things up because it’s not using a fixed-width font.
Screenshot from 2021-05-05 12-16-52 Chrome
Screenshot from 2021-05-05 12-17-30 Firefox

I’m running whatever Chrome Ubuntu 20.0.4 installs. - chrome://version shows 90.0.4430.72 (Official Build) (64-bit). According to dpkg, the package and executable are google-chrome-stable: /opt/google/chrome/google-chrome. I’m not aware of any special customization.

But on Windows (using the executable from https://google.com/chrome/, a fixed-width font is used). Hmmm …

Only because the main section for protobufs happens to be near the end of the documentation, so it also shows in the table of contents when the appendix entry shows. :wink:

And so does Chromium on Ubuntu 20.04. I think a browser that renders <pre> is anything but a monospace font is broken. Of course unless one deliberately overrules the font for the <pre> in CSS.

Could be inheritance from (plweb.css)

body {
    margin: 0px;
    width: 100%;
    background-color: #ecebe3;
    font-family:  Source Sans Pro, sans;
}

We can of course add an explicit fixed font for <pre>. If anyone knows something that looks good in all browsers, please make a PR (our just suggests some CSS here).

I’ve had success with adding this to my HTML:

<link href="https://fonts.googleapis.com/css?family=Source Code Pro" rel="stylesheet"/>

and using it in .css:

    font-family: "Source Code Pro", mono;

Others that look OK (also available from googleapis.com) are “Roboto Mono” and “Inconsolata”.
Some mono fonts have weird ideas about font-size compared to the variable-width fonts used in the text, so figuring out the best font will require playing around with all combinations of windows/ubuntu/mac and chrome/firefox/safari/edge … which is one reason I’ve avoided being a front-end engineer.

I think the issue us with the body defining the font the overrules also for <pre> Now changed to:

@import url(https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,700|Fauna+One);
@import url('https://fonts.googleapis.com/css2?family=Roboto+Mono:wght@400&display=swap');

body {
    margin: 0px;
    width: 100%;
    background-color: #ecebe3;
    font-family:  Source Sans Pro, sans;
}

pre {
    font-family: 'Roboto Mono', monospace;
}

Which should give a browser independent experience. Should get online with the next release.

1 Like

The examples are now in fixed font, but the font seems to be a bit big. (Chrome Ubuntu 20.0.4; Firefox is OK, and so is Chrome on Windows).

PS: The broken border shows at SWI-Prolog -- Manual but which is fixed at SWI-Prolog -- Manual