Ann: SWI-Prolog 9.1.0

jan · November 24, 2022, 2:31pm

Dear SWI-Prolog user,

SWI-Prolog 9.1.0 is ready for download. This version starts a new
development cycle and is practically identical to 9.0.0. Highlights:

Many new features and fixes to the new Emacs sweeprolog mode by
@oskardrums
Almost final version of the new C++ interface SWI-cpp2,h by
@peter.ludemann. There are still a few discussions going on
about part of the API. Anyone working in C++ is kindly
requested to review the current state.
Pretty stable GMP replacement using LibBF is included. Should
work fine for most applications, but can be slow, notably for
handling larger rational numbers (> 64 bits).
Lots of small stuff, notably about configuration, etc.

Enjoy — Jan

SWI-Prolog Changelog since version 8.5.20

SNAP: CMAKE_INSTALL_LIBDIR seems undefined inside snapcraft, causing
the build to fail.
SNAP: Latest snapcraft sets LD_LIBRARY_PATH to provide access to the
“parts”. We should not consider this a broken environment.
DOC: Fixed LaTeX issues blocking the generation of the PDF manual.
TEST: Avoid name conflict between two tests to make test_installation/0
work again.
ENHANCED: library(main) to delay looking for tspy/1 or gspy/1 until
the option is activated.
FIXED: Make X11 event dispatching work in single threaded version.
TEST: Avoid testing stuff that requires threads in the single threaded
version.
CLEANUP: Do not compile code required for multi-threaded loading in
the single threaded version.
DOC: format/1-3: document behavior of ~0f.
CONFIG: Do not install thread libraries for single threaded version.
MODIFIED: When ISO mode is enabled, unknown options are result in a
domain error. This now also holds if the option argument is a dict.
MODIFIED: Ssprintf() and Ssnprintf() functions now use UTF-8 encoding
(used to be ISO Latin 1), so they can handle Unicode.
ENHANCED: Use xref_source/2 for fetching PlDoc comments from loaded
sources. Patch by @swi.
PORT: Provide backtrace with line numbers on MacOS using atos
as replacement for addr2line.
ADDED: prolog_interrupt/0 to make a thread behave as if SIGINT is
received using thread_signal/2.
FIXED: Do not reload system files to get their documentation.
PORT: More integer passing problems over vararg functions.
FIXED: Prolog flag pid on some 64 bit platforms.
PORT: Use CMake to test for <threads.h> support.
MAINTENANCE: Provide a Prolog flag asan when the system is compiled
with AddressSanitizer.
INSTALL: When using -DSWIPL_INSTALL_IN_LIB, make sure that the
CMake export files can be relocated.
WASM: Avoid undefined HTMLCollection.toList(). Emacs whitespace update
MODIFIED: pack_install/1 and friends to pass all environment variables
to the tools that may be executed to build the pack.
PORT: Issue#1070: pack_install/1: Pass PKG_CONFIG_PATH to build
tools
FIXED: format/2,3: manage proper arithmetic environment for
expressions.
PORT: Make LibBF code compile for Clang
FIXED: fetch/restore of Mersenne Twister random state. Also reduced
the size of the state.
MODIFIED: **/2 and ^/2: If the power can be done using rational
arithmetic but the numbers are too big, make the behaviour depend on
max_rational_size_action. When float, try to continue after float
conversion, else raise a resource error.
FIXED: **/2 and ^/2 overflow checking when computing the power of
two rational numbers.
ENHANCED: Prefer mpz_sgn() over mpz_cmp_ui() to 0.
ENHANCED: Key generation for bignum and rational clause indexing
ADDED: BF_FTOA_PL_QUIRKS for correct float formatting.
MODIFIED: CMake target_link_swipl() now removes the lib prefix.
PORT: Updated MacOS dependencies
FIXED: swipl.cmake outside pack_rebuild/2 context gets linking against
libswipl wrong.
FIXED: Issue#163: -s and -l files were loaded before the argv Prolog
flag is cleaned.
ADDED: prolog_walk_code/1: option walk_meta_predicates(+Boolean) to
prevent the code walker from considering meta-predicates. There are
few cases where this is interesting, but it was needed for constructing
a s(CASP) dependency graph.
DOC: Document (part of) the PL_cvt_i_() functions and improve
documentation for several of the PL_get_() and PL_get_*_ex()
functions.
ADDED: PL_get_bool() now also accepts the integers (0,1)
PORT: set RPATH correctly on MacOS for native libs in packages
ADDED: PL_cvt_i_llong() PL_cvt_i_ulong() and PL_cvt_i_bool().
This completes the types supported by these functions.
ADDED: listing/1,2 to also accept a clause reference as entity to list.

Package clib

TEST: Disable tests that require threads on the single threaded
version.

Package cpp

TEST: Fixed C interface PL_scan_options() tests.
TEST: Renamed foreign function test module to test_ffi. This avoids
a conflict (and confusion) with the ffi pack to run the tests
interactively.
TEST: Added PL_scan_options example

Package http

CONFIG: Reorganise installation and docs to distinguish core part we
always want, the HTTP client and HTTP server libraries.

Package odbc

PORT: Allow compiling in single threaded mode.

Package pldoc

CONFIG: Make most of PlDoc work in the single threaded version.
CONFIG: Make PlDoc work if the HTTP server components are not part
of the configuration.
ENHANCED: Source button to prefer loaded code over xref data.

Package semweb

FIXED: rdfa.pl: Avoid dependency oin development tools

Package ssl

TEST: Drop tests when single threaded.

Package sweep

ADDED: PlUnit tests block skeleton and command for inserting it *
sweeprolog.el (sweeprolog-format-string-as-atom): new function.
(sweeprolog-module-header-skeleton): use define-skeleton, making it a
command instead of a defconst. (sweeprolog-plunit-testset-skeleton):
new skeleton and command. * sweeprolog-tests: test it. * README.org
(“Writing Tests”): new section. (“Using templates for creating new
modules”): rename to “Creating New Modules”.
FIXED: instantiation error in color term normalization
FIXED: sweeprolog-beginning-of-next-top-term: other possible loop *
sweeprolog.el (sweeprolog-beginning-of-next-top-term): don’t loop
when called in the last term of the buffer * sweeprolog-tests.el
(beginning-of-next-top-term-at-last-clause): new test case
FIXED: possible infinite loop looking for next term * sweeprolog.el
(sweeprolog-beginning-of-next-top-term): handle comments more reliably
FIXED: exporting predicates in presence of exported
operators * sweep.pl (sweep_exportable_predicates/2): new
predicate. * sweeprolog.el (sweeprolog–module-term)
(sweeprolog–exportable-predicates)
(sweeprolog-analyze-start-exportable)
(sweeprolog-analyze-fragment-exportable): no longer
used, deleted. (sweeprolog-exportable-predicates): new
function. (sweeprolog-read-exportable-predicate): use it.
(sweeprolog-export-predicate): handle exported operators.
ENHANCED: allow sweeprolog-forward-hole to wrap around * sweeprolog.el
(sweeprolog–next-hole, sweeprolog–previous-hole): new functions.
(sweeprolog–forward-hole, sweeprolog–backward-hole): add WRAP arg.
ENHANCED: conditional syntax error highlighting
DOC: mention sweeprolog-swipl-path in the manual
ADDED: minor mode for automatic whitespace insertion * sweep.pl
(sweep_context_callable/2): only consider the context to be
callable if a neck is present up the tree, i.e. not in the head.
- sweeprolog.el (sweeprolog-context-callable-p): new function,
  extracted from (sweeprolog-predicate-completion-at-point)
  (sweeprolog-electric-layout-post-self-insert-function): new function.
  (sweeprolog-electric-layout-mode): new minor mode.
ENHANCED: revise predicate completion at point Completion at point now
detects whether predicate completion is appropriate, by parsing Prolog
code back from point to determine the context of point. If predicate
completion is not appropriate, fallback to atom completion.
ADDED: user option specifying where to define new predicates
- sweeprolog.el (sweeprolog-default-new-predicate-location)
  (sweeprolog-new-predicate-location-above-current): new functions.
  (sweeprolog-new-predicate-location-function): new user option.
  (sweeprolog-maybe-define-predicate): use it.
FIXED: possible error in sweeprolog-describe-predicate
DOC: document sweeprolog-xref-project-source-files in the manual
ADDED: new command sweeprolog-xref-project-source-files
FIXED: find user predicate definitions more reliably
TEST: add test for updating syntax error faces
FIXED: clear syntax error face after fix more aggressively
FIXED: properly update query highlighting in the toplevel
FIXED: simplify & make variable highlighting more reliable
FIXED: only consider backslash as an escape inside strings
FIXED: restrict variable underlining to current clause
FIXED: variable name completion on Emacs 27 * sweeprolog.el
(sweeprolog-variable-completion-at-point): don’t rely on
char-uppercase-p * sweeprolog-tests.el (complete-variable): new test.
ADDED: sweeprolog-forward/backward/mark-predicate
REFACTOR: Simplify and deduplicate code all around * sweeprolog.el
(sweeprolog-colourise-): rename to sweeprolog-analyze-
FIXED: correctly recognize “public” head terms
TEST: add two test cases
FIXED: sweeprolog-identifier-at-point could return unbound module
FIXED: write_sweep_module_location/0 for Windows.
DOC: added section “Code Completion” to the manual
ADDED: completion-at-point for variable names *
sweeprolog.el (sweeprolog-local-variables-collection,
sweeprolog-variable-completion-at-point):
new functions. (sweeprolog-mode): add to
completion-at-point-functions. (sweeprolog-mode-syntax-table): update
according to char_type(Char, prolog_symbol).
ADDED: context-based command for inserting new clauses or terms *
sweeprolog.el (sweeprolog–forward-hole, sweeprolog–backward-hole):
new functions. (sweeprolog-forward-hole): new command.
(sweeprolog-end-of-predicate-definition,sweeprolog-insert-clause,
sweeprolog-maybe-insert-next-clause,
sweeprolog-maybe-define-predicate): new functions.
(sweeprolog-insert-term-functions): new hook.
(sweeprolog-insert-term-dwim): new command. (sweeprolog-mode-map):
bind new commands. * sweeprolog-tests.el (dwim-define-predicate): new
test case. * README.org (Context-Based Term Insertion): new section.
ENHANCED: render refs to Prolog library in help buffers * sweeprolog.el
(sweeprolog-render-html-a): recognize and render references to Prolog
library files.
FIXED: handling of operators as predicate completion candidates
FIXED: don’t include irrelevant candidates in apropos results *
sweep.pl (sweep_predicate_apropos/2): filter candidates by minimum
match quality.
ENHANCED: add predicate properties list in help buffers * sweep.pl
(sweep_predicate_properties/2): new predicate. * sweeprolog.el
(sweeprolog–render-predicate-properties): new function.
(sweeprolog–describe-predicate): use it.
ENHANCED: load and initialize sweep-module just-in-time Instead
of loading sweep-module and initializing Prolog as part of loading
sweeprolog.el (subject to the user option sweeprolog-init-on-load),
this commit introduces just-in-time loading and initialization
of Prolog. Suggested by Stefan Monnier.
- sweeprolog.el: (sweeprolog–open-query): new function, wraps
  sweeprolog-open-query and initializes Prolog on the first call. Used
  in place of sweeprolog-open-query throughout.
  (sweeprolog-init-on-load): make variable obsolete.
ENHANCED: teach sweep to jump to C code for built-in predicates
- sweeprolog.el: - sweeprolog-swipl-sources: new user option.
- sweeprolog-native-predicate-location: new function. -
  sweeprolog-predicate-location: use it. - README.org: Built-in Native
  Predicates: new section.

Package swipl-win

FIXED: Check return codes of many operations on lists.
PORT: Changes for SWI-cpp.h API Version 2

Package windows

PORT: add libname to install() routine
PORT: add libname to install() routine From this discussion:
Static executable (-DSTATIC_EXTENSIONS) - #2 by jan

“the install function of a library must be called install_
and the Prolog side must use use_foreign_library/1”

Package xpce

CONFIG: Avoid dependency on threads.
FIXED: replace internal expand_path/2 with absolute_file_name/3
old code could not handle aliases like swi(library/clp) only
swi(‘library/clp’)

Package zlib

TEST: Disable tests that requires threads

mgondan1 · November 26, 2022, 2:02pm

I encountered problems while running the test suite under MSYS2. Two of the failures are already known (semweb:turtle and ntriples), I think this would require support for Unicode code points > 0xFFFF which isn’t yet there. But the other two failures are new, cpp and mqi, and I think these may be fixed. Consider cpp, the error is at c:/msys64/home/c7201178/swipl-devel/packages/cpp/test_cpp.pl:84

test(as_string, S == "ä¸\u0096ç\u0095\u008Cå\u009B\u009B") :-
    error:current_encoding(Enc),
    writeln(Enc),
    atom_to_string(世界四, S).

I added the two lines with the encoding, they write “octet” on the screen. [On Linux, it says the same, though].

The error says

Test project C:/msys64/home/c7201178/swipl-devel/build
    Start 43: cpp:cpp
1/4 Test #43: cpp:cpp ..........................***Failed    3.51 sec
% PL-Unit: cpp ...........octet
ERROR: c:/msys64/home/c7201178/swipl-devel/packages/cpp/test_cpp.pl:84:
        test as_string: received error: atom_to_string/2: Cannot represent due to `encoding' (Cannot represent char U4e16 using current locale encoding)

Still, I am wondering if the error just results from the test suite not loading test_cpp.pl properly. If I reduce the file to the minimum, like this:

$ cat test_cpp2.pl
% -*- mode: Prolog; coding: utf-8 -*-

test :-
    atom_string(世界四, S),
    writeln(S).

and then invoke swipl, I need to explicitly state the encoding of the file:

c7201178@PC105-C720 MINGW64 ~/swipl-devel/packages/cpp
$ ~/swipl/bin/swipl
Welcome to SWI-Prolog (threaded, 64 bits, version 9.1.0-7-g039a75912-DIRTY)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
Please run ?- license. for legal details.

For online help and background, visit https://www.swi-prolog.org
For built-in help, use ?- help(Topic). or ?- apropos(Word).

1 ?- [test_cpp2].
ERROR: c:/msys64/home/c7201178/swipl-devel/packages/cpp/test_cpp2.pl:4:17: Syntax error: Operator expected
true.

2 ?- load_files(test_cpp2).
ERROR: c:/msys64/home/c7201178/swipl-devel/packages/cpp/test_cpp2.pl:4:17: Syntax error: Operator expected
true.

3 ?- load_files(test_cpp2, [encoding(utf8)]).
true.

4 ?- test.
世界四
true.

5 ?- halt.

Any idea how I could isolate this better?

jan · November 26, 2022, 4:02pm

Well, since some time the core supports the full Unicode range on Windows. These external libraries consider wide strings UCS-2 encoded (on Windows), while they should use UTF-16.

What is the problem here? Could be that the JSON libraries cannot handle the full Unicode range for the same reason as the semweb libraries.

The test is invalid. The code extracts the raw data from a (wide) atom and translates this into a byte string. The raw data format is different in Windows (UTF-16) than on non-Windows (UCS-4). For now I disabled these two tests for Windows. I think as_string() should better raise an exception when applied to wide-character arrays as the result is basically undefined.

mgondan1 · November 26, 2022, 5:48pm

The failures in cpp and mqi have disappeared after updating.

Ah, I wasn’t sure about this. Great!

mgondan1 · November 26, 2022, 8:35pm

Indeed, addBuf in ntriples.c “cuts” the upper bits away,

static inline int
addBuf(string_buffer *b, int c)
{
  if ( b->in < b->end )
  { *b->in++ = c; // here, \\U00012345 becomes \\u2345
    return TRUE;
  }

  return growBuffer(b, c);
}

It’s a bit disturbing that gcc does not even issue a warning. (EDIT: -Wconversion would do so)

jan · November 27, 2022, 9:10am

For the core we have some support functions and macros in src/os/pl-utf-8.[c,h] (name of the file is a bit misleading now as it support both UTF-8 and UTF-16). I don’t really like the idea of providing support for processing encoding to SWI-Prolog.h as that has nothing to do with Prolog.

Possibly we should have a reusable header/library to support the packages with stuff that is not related to Prolog directly, but still required for multiple packages?

peter.ludemann · November 27, 2022, 6:51pm

I use these flags – they’ve caught a few problems (and also some in included libraries):
-Wall -Wextra -Wconversion -Wsign-conversion -Wfloat-conversion -Wno-unused-parameter
and -Warith-conversion (which doesn’t work with Clang)

I’d use -Werror as well, but there were a few “bogus” warnings unfortunately.

mgondan1 · December 7, 2022, 6:44pm

A quick fix would thus be

static inline int
addBuf_old(string_buffer *b, int c)
{ if ( b->in < b->end )
  { *b->in++ = c;
    return TRUE;
  }

  return growBuffer(b, c);
}

static inline int
addBuf(string_buffer *b, int c)
{ if ( c <= 0xffff )
    return addBuf_old(b, c) ;

  int l, t ;
  utf16_encode(c, &l, &t);
  return addBuf_old(b, l) && addBuf_old(b, t) ;
}

Does this make sense?

Topic		Replies	Views
Ann: SWI-Prolog 9.0.0 Releases stable	1	1256	November 27, 2022
Ann: SWI-Prolog 9.1.1 Releases	0	330	December 6, 2022
Ann: SWI-Prolog 9.2.0 (stable) and 9.3.0 (new devel series) Releases	41	1072	March 11, 2024
Ann: SWI-Prolog 9.2.2/9.3.2 Releases	1	216	March 11, 2024
Ann: SWI-Prolog 9.0.4 (stable) Releases	2	555	May 3, 2023