Pldoc bug? losing docs on blank line

There seems to be a bug in pldoc that stops parsing when it finds a blank line, this causes the documentation for http_open/3 (for example), to be truncated:

On the web the http_open/3 documentation ends with the option:

raw_encoding(+Encoding)
    Do not install a decoding filter for Encoding. For example, using raw_encoding('applocation/gzip') the system will not decompress the stream if it is compressed using gzip.

but if you look at the http_open/3 source code you can see many other options which are not found on the website:


%     * raw_headers(-Lines)
%     Unify Lines with a list of strings that represents the complete
%     reply header returned by the server.  See also headers(-List).
%
%     * redirect(+Boolean)
%     If `false` (default `true`), do _not_ automatically redirect
%     if a 3XX code is received.  Must be combined with
%     status_code(Code) and one of the header options to read the
%     redirect reply. In particular, without status_code(Code) a
%     redirect is mapped to an exception.
%
%     * status_code(-Code)
%     If this option is  present  and   Code  unifies  with the HTTP
%     status code, do *not* translate errors (4xx, 5xx) into an
%     exception. Instead, http_open/3 behaves as if 2xx (success) is
%     returned, providing the application to read the error document
%     from the returned stream.
%
%     * output(-Out)
%     Unify the output stream with Out and do not close it. This can
%     be used to upgrade a connection.
%
%     * timeout(+Timeout)
%     If provided, set a timeout on   the stream using set_stream/2.
%     With this option if no new data arrives within Timeout seconds
%     the stream raises an exception.  Default   is  to wait forever
%     (=infinite=).
%
%     * post(+Data)
%     Issue a =POST= request on the HTTP server.  Data is
%     handed to http_post_data/3.
%
%     * proxy(+Host:Port)
%     Use an HTTP proxy to connect to the outside world.  See also
%     socket:proxy_for_url/3.  This option overrules the proxy
%     specification defined by socket:proxy_for_url/3.
%
%     * proxy(+Host, +Port)
%     Synonym for proxy(+Host:Port).  Deprecated.
%
%     * proxy_authorization(+Authorization)
%     Send authorization to the proxy.  Otherwise   the  same as the
%     =authorization= option.
%
%     * bypass_proxy(+Boolean)
%     If =true=, bypass proxy hooks.  Default is =false=.
%
%     * request_header(Name = Value)
%     Additional  name-value  parts  are  added   in  the  order  of
%     appearance to the HTTP request   header.  No interpretation is
%     done.
%
%     * max_redirect(+Max)
%     Sets the maximum length of a redirection chain.  This is needed
%     for some IRIs that redirect indefinitely to other IRIs without
%     looping (e.g., redirecting to IRIs with a random element in them).
%     Max must be either a non-negative integer or the atom `infinite`.
%     The default value is `10`.
%
%     * user_agent(+Agent)
%     Defines the value of the  =|User-Agent|=   field  of  the HTTP
%     header. Default is =SWI-Prolog=.
%
%   The hook http:open_options/2 can  be   used  to  provide default
%   options   based   on   the   broken-down     URL.   The   option
%   status_code(-Code)  is  particularly  useful   to  query  *REST*
%   interfaces that commonly return status   codes  other than `200`
%   that need to be be processed by the client code.
%
%   @param URL is either an atom or string (url) or a list of _parts_.
%
%               When provided, this list may contain the fields
%               =scheme=, =user=, =password=, =host=, =port=, =path=
%               and either =query_string= (whose argument is an atom)
%               or =search= (whose argument is a list of
%               =|Name(Value)|= or =|Name=Value|= compound terms).
%               Only =host= is mandatory.  The example below opens the
%               URL =|http://www.example.com/my/path?q=Hello%20World&lang=en|=.
%               Note that values must *not* be quoted because the
%               library inserts the required quotes.
%
%               ```
%               http_open([ host('www.example.com'),
%                           path('/my/path'),
%                           search([ q='Hello world',
%                                    lang=en
%                                  ])
%                         ])
%               ```
%
%   @throws error(existence_error(url, Id),Context) is raised if the
%   HTTP result code is not in the range 200..299. Context has the
%   shape context(Message, status(Code, TextCode)), where `Code` is the
%   numeric HTTP code and `TextCode` is the textual description thereof
%   provided by the server. `Message` may provide additional details or
%   may be unbound.
%
%   @see ssl_context/3 for SSL related options if
%   library(http/http_ssl_plugin) is loaded.

I spent about 45 minutes looking around the website to find the way to get the http status code back without throwing an exception, but it wasn’t there. I thought to myself, Jan doesn’t code this way :smiley: “I am sure he provided an option to get the http status code back” :sweat_smile: So I went to look at the source code and sure enough, I found it (aptly named status_code(-Code). This is when I saw the blank line above the raw_headers(-Lines) option, surmising that Pldoc had stopped parsing after the blank line.

I think pldoc should be less strict and continue parsing after blank lines, otherwise users of the documentation may never now they are missing a good chunk of the documentation, and it may even take months for developers to see error, so all in all I think it is better to make pldoc more flexible. I happened to find it because I know the kind of developer Jan is, but I don’t think the average prolog user would be able to predict it. They may just spend time coding it when it was there all along.

Thanks for noting! Fixed the source. Not sure what do do with it. It isn’t really easy to figure out there is an empty line separating this. I sometimes deliberately add a blank line to continue with additional information for developers. In this case it is of course an obvious mistake resulting from
editing the docs :frowning: