Ann: SWI-Prolog 9.2.3 (stable)

Releases
1

Dear SWI-Prolog user,

SWI-Prolog 9.2.3 (stable) is ready for download. This is the last
almost complete copy of changes from the development series for 9.2.x
as the code base for the development series changes considerably,
starting with 9.3.4. Highlights:

  • Portability updates by @mgondan1
  • C++ interface and testing updates by @peter.ludemann
  • Avoid occasional HTTP test failures by @brebs
  • Several updates to the HTTP package, notably enhancements
    to session management, in particular when used with the
    Redis plugin. Required for the (now) distributed SWISH
    service.
  • Added library(http/http_server_health) to add a /health
    path to your server with basic statistics.
  • PlDoc HTML backend changes to handle most diacrytics and
    use proper Unicode quotes. In part by @peter.ludemann.
    Improve help/1 support for user predicates and ordering
    of predicates in the docs for included files by
    @ridgeworks.
  • PlUnit to report test setup failures properly.
  • Janus adds py_import/2 and fixes error handling for
    query iterators if the data conversion fails.
  • Fix race conditions when using stream_property(S, fileno(_)),
    enumerating Prolog streams connected to OS handles. This is
    a rather involved set of changes.
  • Avoid memory fragmentation and poor reuse on recursive findall/3
    calls.
  • Avoid memory fragmentation on bignum arithmetic. Avoid the use
    of malloc() for evaluating arithmetic expressions that require
    not much memory.

A good deal of the improvements result from migrating SWISH from
a single big server to three small servers with limited memory.

This version is released through all the channels: source, binaries
(including Intel MacOS), Macports, PPA, snap and Docker.

Enjoy --- Jan

SWI-Prolog Changelog since version 9.2.2

  • TEST: Disable max_integer_size test for emscripten Doesn’t handle
    longjmp() correctly in release mode. Works in debug mode, but that
    is not useful.

  • FIXED: write clauses holding > 0xffff Unicode chars in strings to
    QLF on Windows.

  • FIXED: forAtomsInClause() handling of B_*_?C instructions These called
    PL_unregister_atom() on the atom, rather than the argument function.

  • FIXED: Temp buffer handling when stack is close to top of address range

  • DEBUG: Fixed chk_secure report on invalid term due to
    PushPtr()/PopPtr() This construct to turn raw pointers into term
    references to preserve them during GC can create referenc pointers
    to the local stack. Another possible fix is to turn such references
    into direct term references. That avoids local stack references,
    but as this is only used around GC and GC itself uses local stack
    reference pointers, there is little to gain.

  • FIXED: unifiable/3 internal consistency Unifiable did not correctly
    handle stack resizing. This is detected when running using chk_secure.
    Not sure whether it can also cause real issues, but the new code is
    simpler and safe.

  • FIXED: powm/3 function: wrong type check on 3rd argument.

  • FIXED: Declaration of term stack for evaluating expressions

  • FIXED: Write bignums when inside arithmetic. Should normally not be
    needed, but debugging is much easier if we can.

  • PORT: Avoid spurious warnings from spoiling pthread_setname_np()
    test This used to use -Werror to ensure a compiler failure if the
    number of arguments do not match the prototype, but this is already
    enforced in todays compilers.

  • FIXED: use_module/2 with except and reexport/2 with except to
    print warning These predicates failed if the source did not export
    the excluded predicate. Now it prints a warning and ignores the
    non-existent predicate.

  • FIXED: Another fileno issue.

  • ADDED: library(sandbox): allow for Dict.get(key,Default)

  • FIXED: Stream initialization for swipl-win.exe Was broken since
    recent changes keeping track of fileno. Reported by Jan Burse.

  • ADDED: PL_system_error() This patch also adds documetation for
    PL_fatal_error().

  • ADDED: Prolog flag max_integer_size This flag limits the allocations
    on behalve of bignum and rational number processing, DoS attacks
    by exhausting memory using huge numbers as well as poor interrupt
    handling due to lack of signal handling inside the bignum libraries.

  • ENHANCED: Use temprary malloc API for bignum arithemetic

  • ENHANCED: Keep small bignums on the stack

  • CLEANUP: Simplify arithmetic context This is no longer nested and can
    (thus) be simplified.

  • FIXED: format/3: Ensure balanced AR_BEGIN()/AR_END() On errors it was
    possible that AR_END() was not called. This may lead to memory leaking
    and errornous operation when linked in a process that also uses gmp.

  • ENHANCED: Improve memory reuse after writing huge integers.

  • FIXED: #1254 crash in PL_get_mpq()

  • FIXED: tell/1: properly manage standard stream references.

  • FIXED: Stream reference counting and reset when closing a standard
    stream.

  • FIXED: Memory leak in string_bytes/3.

  • FIXED: Properly close temporary redirect to a string
    tellString()/toldString() could fail to reclaim the temporary stream
    object.

  • ENHANCED: Use a segmented stack for dealing with findall/3.
    This avoids memory fragmentatation when findall/3 is deeply nested.
    Although that is a rare occasion, deeply nested findall/3 leads to
    poor memory reuse. As activating and deactivating is stacked anyway,
    a segstack avoids the fragmentation.

  • ADDED: help(Name/Arity): handle user predicates.

  • FIXED: xref_source/2: keep processing PlDoc commants after including
    a file.

  • ADDED: help/1: give help on a specific user predicate using Module:PI.

  • ENHANCED: help/1: filter user predicates on the module class.
    This use to filter on whether they are documented in the manual,
    making it impossible to show documentation for loaded predicates that
    conflict with something defined in the manual.

  • FIXED: Thread I/O streams may be invalid If a thread is created
    using thread_create/3 with the inherit_from(Thread) option or using
    PL_thread_attach_engine() (which inherits from main), we may copy
    current_input or current_output from a temporary redirect.

    The reference counting when copying the streams was broken, possibly
    causing memory corruption. This is improved, but not 100% safe.
    However,
    we now set the current_input and current_output to user_input and
    user_output, which typically never change.

  • FIXED: help/1 to deal with help on public predicates in included files.
    help/1 finds the predicate and, if necessary, reloads the file in
    which it is defined to get the documentation. If the predicate is
    defined in an include file it should reload the main file to get the
    module context right.

  • ENHANCED: trace/1,2 and friends. - Code cleanup - Support conditional
    trace using trace(Head, Port(Condition))

  • FIXED: Ensure correct setting file handle for std streams. This was
    broken after adding the fileno field to the IOSTREAM struct.

  • MODIFIED: Discontinue thread_exit/1 on Windows The implementation was
    broken for some time. The reason is not entirely clear, but possibly
    related to issues in pthread-win32. As the predicate is unsafe and
    deprecated anyway, it has now been removed from the Windows version.

  • FIXED: pack metadata checking for prolog:Feature requirements.

  • FIXED: make stream_property(S, file_no(N)) safe. This used to be
    subject to race conditions between the enumerating thread and threads
    that close the underlying stream. We now copy the file handle as it
    was found when creating the stream to the main stream structure.

  • CLARIFIED: PL_free_blob() can be called multiple times. This patch
    changes subsequent calls to a no-op that returns FALSE and document
    this is valid. After comment by Peter Ludemann.

  • CLEANUP: Do not call blob release() on GC or cleanup after
    PL_free_blob()

  • ENHANCED: Answer/query expansion. Toplevel variable ($Var) handing
    is now always handled, regardless of the user:expand_query/4 or
    user:expand_answer/2 hooks. The user:expand_answer/2 hook is
    superseded by prolog:expand_answer/3.

  • DOC: Deprecate format/3 with a non-list 3th argument.

  • FIXED: Setting the C-stack limit to values not a multiple of the page
    size Causes problems on some platforms.

  • DOC: Use ASCII LaTeX ``word’’

  • FIXED: Preserve atributes set on foreign functions for saved states.
    This bug causes http_stream:stream_range_open/3 not be to transparent
    when reloaded from a saved state. This in turn raises an error with
    http_open/3 on HTTPS and chunked streams.

  • ENHANCED: coverage/2: analyze and annotate included files.
    The coverage analyzer now deals with files that are included. Files
    that are included multiple times in different modules get their
    numbers aggregated.

  • FIXED: Crash when using ‘$cov_data’/3 if there is no coverage data.

Package clib

  • PORT: Avoid warning on __RCSID() in bsd-crypt.c Suppresses a warning
    on CRAN win builder

Package cpp

  • TEST: exceptions in blob callback

  • DEPRECATED: PlStream::tell(), PlStream::seek() Also fixes initilization
    errors. Uses PL_system_error() to signal misuse rather than assert()

  • FIXED: Possibly uninitialized variable This is a temporary patch.
    blob compare() and acquire() cannot raise a Prolog exception.
    This should print a message and abort the process.

  • FIXED: handle nil blobs in callbacks

  • DOC: improve non-det example (and tests)

  • DOC: Use ASCII LaTeX ``word’'.

Package http

  • TEST: Don’t reuse original test port for HTTP proxy Instead, use port
    assigned by OS.

    Intention is to prevent intermittent build test failure on Linux
    armv7, and Mac M1.

    The HTTP, HTTPS, SOCKS and “unused” ports are still reused, though.

    Original build failure report:
    Swi-prolog 9.2.2 fails http:proxy test on armv7

  • FIXED: Make http session GC thread inherit from main This thread is
    started lazily from an HTTP worker and would by default interit the
    temporary CGI stream as current_output. By interiting from main we
    can safely print debug messages and avoid stream existence exceptions.

  • FIXED: HTTP Sessions using Redis to properly handle concurrent delete.
    Multiple participants may decide to delete a session. By checking
    the removal from the timeout queue we avoid that two nodes delete
    the same session.

  • ADDED: HTTP session management granularity This setting controls
    how precisely we manage idle time updated and session timeouts.
    This is notably important when using an external (Redis) database
    for sharing session information to reduce traffic.

  • ENHANCED: http_upgrade_to_websocket/3: discard a websocket on failure.
    When using the guarded(false) option, the created web socket is now
    only left open if the passed Goal succeeds.

  • DOC: http_server/2: Explictly mention the possible use of AF_UNIX
    sockets

  • ENHANCED: Implementation of open_files in server stats.

  • ADDED: Extended /health with CPU time and thread satistics.
    Also allows asking for specific keys only in the HTTP request, e.g.

    curl http://my.org/health?fields=workers,open_files

  • ADDED: http_workers/2: mode (?,-) (was (+,-)) Also point at
    library(http/http_dyn_workers) in docs and improve the implementation
    by using aggregate_all/3 rather than findall/3 and length/2.

  • ADDED: library(http_server_health)

Package ltx2htm

  • FIXED: handling of final single quote

  • ENHANCED: Warn on non-ASCII characters. We can’t handle these.

Package pldoc

  • ENHANCED: Deal with a large selection of diacritics in the LaTeX
    backend. When converting PlDoc markdown to LaTex, we now generate
    proper LaTeX sequences for ten common diacritics.

  • ENHANCED: Better ordering of predicates comming from other files.
    pldoc for “re-exported” predicates are now ordered by position, i.e.,
    file name and line number.

Package plunit

  • FIXED: Report tests where the setup goal fail as failed and print
    output.

Package semweb

  • FIXED: #112 C11 compatibility (no declaration after label)

  • FIXED: Message propagation when rdf_load_db/1 fails.

  • FIXED: Type error

  • FIXED: #110 Allow blank nodes as graph in TRiG format.

Package ssl

  • PORT: Fix test for X509_get0_signature() on Windows. This is still
    problematic as -Werror -c assumes gcc/clang/… We could use
    try_compile as in cmake/CheckFloatingPointFormat.cmake of the
    main source?

Package swipl-win

  • FIXED: Reset fileno for stdio streams

Package swipy

  • FIXED: iterator janus.query(): properly handle data conversion
    exceptions

  • ADDED: py_import/2: Allows to import a module with a different name.

2 Likes
2

MSYS2: The swipy tests fail on ctest, I can fix it locally by setting PYTHONHOME:

cmake etc.
ninja
ctest fails, saying:
Test project C:/msys64/home/matth/MINGW-packages/mingw-w64-swi-prolog/src/build-UCRT64
    Start 81: swipy:janus
1/2 Test #81: swipy:janus ......................***Exception: SegFault  0.48 sec
Could not find platform dependent libraries <exec_prefix>

This works:

PYTHONHOME=/ucrt64 ctest

I wonder why this is needed. The path /ucrt64 is not special, it corresponds to /usr under a “normal” unix.

Edit: Another workaround is to create a venv first,

python3 -m venv .
ctest

Edit: I understand that if I use an embedded python like in janus, I typically want to control its configuration etc. My question is mainly related to ctest.

3

Nothing changes AFAIK. This look like a pretty environment specific issue where it somehow pick up incompatible parts from some other Python installation? That is probably best resolved in the build scripts for the platform.

4

I have put PYTHONHOME=${MINGW_PREFIX} before ctest, works fine, now we have a swipl-9.2.3 on MSYS2 without any patches :muscle:

1 Like