Ann: SWI-Prolog 9.3.32

Releases
1

Dear SWI-Prolog user,

SWI-Prolog 9.3.32 is ready for download. There are many changes in
this release. Highlights:

  • Performance. A small change to the VM interpreter as well as
    enabling gcc -O3 improved performance on i86_64 by over
    20%. The VM change and better tuning more than doubles the
    performance of the WASM version.

  • Preliminary changes to dict handling for synchronization with
    the PIP working group:

    • Use of _{...} (unbound key) is going to be deprecated.
      #{...} is the new syntax for anonymous dicts.
    • For the dict built-ins, # now matches any tag.
    • The Var{key: Value, ...} syntax is optionally used for
      attributed variables as well as writing terms that are not
      trees, but arbitrary directed graphs (including rational
      trees/cyclic terms). This version implements the current
      state of the proposal. Details may change.

    See the Prolog flag var_tag for controlling compatibility.

  • library(uri) has now support or urn URIs.

  • Several enhancements to the GUI:

    • Bugfix in xpce thread handling caused undefined methods
      if multiple threads try to lazily bind a method.
    • Proper copy from Epilog consoles
    • Avoid illegal UTF-8 sequences when handling fast streaming
      output in Epilog consoles.
    • Fixed backspace handling in incremental search
    • Make keyboard selection management work properly in PceEmacs.
    • Avoid errors when making the thread monitor too small.

  • Several portability issues by @mgondan.

    Enjoy ā€” Jan

SWI-Prolog Changelog since version 9.3.31

  • ADDED: Implement read_term/1: implement reading attributed and
    labeled terms. This patch supports Var{Attr: Value, ā€¦} if the
    Prolog flag var_tag is set to attvar. This allows reading terms
    with attributes. It also supports labeled subterms that allows to
    read terms that are directed graphs rather than trees. This includes
    cyclic terms. For this we reserve the attributed =. For example

    ?- A = X{= : f(X)}.

    Binds A to a cyclic term equivalent to executing X = f(X). This is
    part of ongoing discussions in the PIP working group. It is currently
    implemented by ECLiPSe and SWI-Prolog.

  • DOC: Updates documentation for using of # anonymous dicts.

  • TEST: Update tests for use of # anonymous dicts

  • MODIFIED: Dict.put(Path, Value): create new dicts as #{...}.

  • MODIFIED: select_dict/3 handling of # tag The # tag matches any tag
    at the other end and the dict holding the remainder has the # tag.

  • MODIFIED: write_term/2 using attributes(write) This now writes
    Var{Att1: Value1, Att2: Value2, ...}, which is valid syntax for
    read_term/2 if the Prolog flag var_tag is attvar.

  • ADDED: read_term/2 and friends to respect the new var_tag flag.

  • ADDED: Prolog flag var_tag Defines how Var{ā€¦} terms are
    interpreted. This commit only implements setting and getting the flag.

  • MODIFIED: :</2 and >:</2: match # tag The tag # matches any
    tag on the other side without instantiating the tag.

  • FIXED: instantiation errors on prolog_frame_attribute/3 on exceptions.

  • FIXED: Exception details from '$tmp_file_stream'/4

  • ERROR: Domain error: encoding' expected, found txtā€™

    The txt does not make sense * CLEANUP: Fixed more compilation warnings
    when using -O3

  • BUILD: Fix PGO building for AppleClang

  • BUILD: Fix PGO build for Clang. This now does optimize.

  • BUILD: Changed default optimization for gcc to -O3 Earlier tests showed
    no noticeable difference, but using gcc-15 and setjmp()/longjmp out
    of the way we get about 15% improvement.

  • SANDBOX: Declare string quasi quotation syntax as safe.

  • TEST: Fix format_time/3 tests by setting correct timezone.

  • WASM: Set defaults for best performant build.

  • ENHANCED: Move setjmp() out of the VM main function Using setjmp()
    harms register allocation, which slows down the VM. Some data points:
    Clang: 6% (Clang-17 on Apple M1 as well as Clang-20 on AMD3950X),
    GCC: 13% (GCC-15 on AMD3950X), WASM: 35% (Emscripten 4.0.15 on Node.js
    22.19 on AMD3950X). MinGW-14: 18% (Windows binary running on AMD3950X
    under Wine).

  • TEST: Fixed ration number writing test to work regardless of flags.
    Test failed when run with rational_syntax set to natural.

  • ENHANCED: Get rid of setjmp/longjmp() in PL_next_solutions() This
    improves performance by about 12%. It does make a couple of scenarios
    for stack overflow handling impossible though. This patch merely
    introduces O_THROW as C macro to enable/disable this.

  • WASM: Make setting CFLAGS and LDFLAGS in BuildType.cmake work The
    if/elseif/ā€¦ selection triggered on Clang.

  • WASM: Compile VM using -O2 This produces a smaller binary with
    slightly better performance.

  • WASM: When using a monolithic PL_next_solution(), use -O1 in
    Debug mode Otherwise local variables with non-overlapping scope are
    not merge, causing an Emscripten compiler error

  • WASM: Do not build swipl-win Also cleanup handling platform-dependent
    cmake options

  • FIXED: Provide random_property/1 when compiled with LibBF.

  • FIXED: When a REPL loop receives the halt/0,1 exception, make
    it return. Before, if non-threaded or the REPL loop runs in the
    main thread, terminate the process. Otherwise halt/0 was ignored.
    The behaviour for the main thread is still the same, but prolog/0
    succeeds in other threads if the thread calls halt/0.

    Possibly we should add a flag to control this behaviour, i.e. whether
    or
    not halt/0 inside prolog/0 terminates the process or merely the
    REPL loop.

Package archive

  • CLEANUP: Avoid undefined warnings for gcc -O3

Package bdb

  • CLEANUP: Avoid undefined warnings for gcc -O3

Package clib

  • FIXED: crypt/2 on Windows using bsd-crypt.c: possible memory
    corruption.

  • FIXED: urn-schema URIs can have a query and fragment field.

  • MODIFIED: library(uri) to raise more exceptions and support URNs.

  • FIXED: directory_member/3: respect file_type(regular) option This
    allows ā€˜directory_member/3ā€™ to generate regular (non-directory)
    files with option ā€˜file_type(regular)ā€™, in accordance with how
    ā€˜absolute_file_name/3ā€™ treats this option.

  • CLEANUP: Avoid undefined warnings for gcc -O3

Package http

  • FIXED: Possible uninitialized variable (in error case)

  • SANDBOX: Declare Quasi Quotations for json, html and javascript
    as safe.

  • ADDED: json quasiquotation syntax This allows for embedding JSON
    documents represented as a Prolog dict using e.g.

    ?- X = 10, D = {|json(X)||{"x": X}|}
D = _{x:10}.

Package libedit

  • CLEANUP: Avoid undefined warnings for gcc -O3

Package mqi

  • PORT: Search for python3 executable to try python3 and then python.
    Modern systems are beginning to install only python, which is
    Python3.

    There isnā€™t any support for python2 anymore, since 2020, and the
    windows installer does not install a python3.exe.

Package odbc

  • CLEANUP: Avoid undefined warnings for gcc -O3 Also replaces local
    error functions with calls to PL_*_error()

Package semweb

  • CLEANUP: Avoid undefined warnings for gcc -O3

Package sgml

  • CLEANUP: Avoid undefined warnings for gcc -O3

Package ssl

  • CLEANUP: Avoid undefined warnings for gcc -O3

Package table

  • CLEANUP: Avoid undefined warnings for gcc -O3

Package tipc

  • CLEANUP: Avoid undefined warnings for gcc -O3

Package xpce

  • FIXED: Lazy binding of Prolog methods was not thread-safe.

  • FIXED: Make double and triple click in Epilog copy the selection
    properly. This set the selection twice from an event, which apparently
    causes SDL3 to reset the selection.

  • FIXED: Incremental search backspace handling

  • CLEANUP: Avoid undefined warnings for gcc -O3

  • FIXED: Emacs mark+move cursor to maintain an active selection.

  • FIXED: Correctly handle UTF-8 console output if the buffer ends with
    an incomplete UTF-8 character.

  • FIXED: PceEmacs: make shift-caret movement extend the selection.
    Selection is now deactivated without shift.

  • FIXED: Resizing the thread monitor too small leads to a type error.
    Reported by @mike.elston

3 Likes
2

dict_create/3 missed?

Welcome to SWI-Prolog (threaded, 64 bits, version 9.3.32)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
Please run ?- license. for legal details.

For online help and background, visit https://www.swi-prolog.org
For built-in help, use ?- help(Topic). or ?- apropos(Word).

true.

?- set_prolog_flag(var_tag,attvar), dict_create(D,_,[]), is_dict(D).
D = _{}.
3

In this step, little has changed. It merely allows using #{...} for dicts more comfortably and Dict.put(a/b, val) will create a dict with # rather than _ if it needs to create the embedded dict.

In subsequent steps we probably change more. I donā€™t feel comfortable forcing all this on the next stable release though. As we get more experience replacing _ with # for anonymous dicts weā€™ll decide on the next steps. Possibly dict_create/3 with an unbound tag will than act as writing _{...}.

4

I saw a ~10% slowdown with -O3 for one of my programs, which has a lot of delayed goals and backtracking, running unit tests in parallel. (I donā€™t use @janā€™s build script, so there might be a mistake in how I set the optimisation ā€¦ if you want me to look into this further, Iā€™ll convert to using @janā€™s scripts and test again.)

5

Yes, that is worthwhile. There are some things that may affect what is going on

  • First patch is the change in using setjmp/longjmp(). The performance gain there is consistent over platforms and C compilers.
  • Change to -O3. That helps for gcc 15.2 on x86_64. It does not help for Clang-20 on x86_64, neither for Apple-Clang 17 on M1. I donā€™t recall the details, but I think it caused a small slowdown. Possibly it also doesnā€™t help on older GCC versions?
  • Some attributed variable functions used to be private in pl-attvar.c, but are now public to support the Var{Att: Value, ...} syntax. Possibly that leads to poorer optimization?
  • Possibly there are not enough attributed variable programs in the benchmarks? That could cause PGO to make things worse. Possibly you can extract a new benchmark from your code? Ideally, benchmarks are widely portable, so we can compare them with other systems.
6

It will take me a bit of time to sort things out ā€“ I have to first apply the advice in xkcd: Git and then finish off some ā€œwork in progressā€ so that I can properly sync with GitHub.

The code is in GitHub - kamahen/nerdle: Nerdle game in Prolog ā€¦ Iā€™m not sure how to extract a ā€œbenchmarkā€ from it, but Iā€™ll ponder the problem.

Looking at your list of things, the one that stands out to me is attributed variables ā€“ my code relies heavily on dif/2. There is fairly heavy use of meta-predicates such as maplist/3 and foldl/5 ā€¦ I use library(apply_macros) but that doesnā€™t handle foldl/5. Dicts are also used, but theyā€™re small (no more than 10 items in a dict). There are a also some big sets of facts, and itā€™s possible theyā€™re not being indexed correctly ā€¦ Iā€™ll do some profiling and use JITI to inspect.

1 Like
7

Some more issues with test_installation([packages(false)]).

If I invoke it twice in a row, 4 tests fail (tabling, xsb). Sorry for not being more helpful/specific.

8

Didnā€™t look at it precisely, but AFAIK that is to be expected as part of the tests rely on static code being subject to clause/2, etc. while at some point this access is denied to test denied access. This flag (protect_static_code) is there to make it harder to reverse engineer code from a binary and cannot be restored. The only way out would be to start a sub process for running these tests. I donā€™t see the point.

Bottom line: you can run test_installation/0,1 once if your working directory is writeable and allows executing programs you write there. Probably there are even more restrictions. I donā€™t see much point in ā€œfixingā€ this. Documenting might be worthwhile.

9

All right. On a more abstract level, test_installation/0,1 has side effects. For example, fib/2 (and others) is defined after itā€™s been called.

So, if I want it to run the tests without side effects, I need to

current_prolog_flag(executable, Exe), process_create(Exe, ['-t', test_installation], []).
10

Which fib/2 are you referring to? There are a lot of fib/2 definitions in the test cases. Fibonacci seems popular :slight_smile:

Anyway, this could (and should) be avoided using proper cleanup. Most test files can be executed multiple times without side effects. Ideally, this should apply to all, except there is this flag protect_static_code, which can be set to true, but not back to false for security reasons.

That is why there is a test dir unprotected that contains tests that can only be executed if
protect_static_code is false.

We could make these tests conditional on this flag. That should (if other fixable issues are fixed) allow running (most of) the tests multiple times. Still, the system has the side effect of protect_static_code being set after running the tests, so I do not know how useful all this is. I still fail to see the point in trying hard to make it possible to run test_installation/0 multiple times ā€¦

The only reason I can imagine is stress testing. For that you now need a loop that fires a new Prolog for each run. That is also how I do stress testing, creating the loop in the gdb debugger. Then I typically create multiple of these to run concurrently and run them with varying number of assigned CPUs to stress concurrency issues. I would not run that with a production version and test_installation/0 though.

11

I basically agree.

If you donā€™t mind, Iā€™ll add a little test for protect_static_code in the beginning of test_installation/0,1. If itā€™s true, the test fails with an informative error message.

12

Iā€™d be more in favour of making the affected tests conditional. Pushed fe94b7813eef1fb35447b8ff388beaa6df572f37, which should prevent test/2 from running these tests. Not tested.

13

You need setup and teardown to do that properly: ideally, the tests must clean up after themselvesā€¦