Ann: SWI-Prolog 9.1.19

jan · November 16, 2023, 4:35pm

Dear SWI-Prolog user,

SWI-Prolog 9.1.19 is ready fro download. The ChangeLog is quite long.
Highlights:

More work on Janus resulting from ongoing discussions. Introduced
py_setattr/3 to set attributes on objects (or modules). Added
py_module/2 to define Python modules from Prolog. Synchronised
naming and semantics of Python functions with XSB. Mostly affects
infrequently used functions.
A lot of portability work. The ultimate target is to get a fully
supported version running under Conda. The main bottleneck are
serveral issues building the Windows version using MSVC with the
Conda directory structure, which is similar to what is used in
e.g., Linux. We are not there yet. We can claim robust builds
using VS2022 that support the whole system and passes all tests.
Introduce the already disussed -Dflag=value commandline option.
In addition, this patch introduces flags whose domain is a set
of atoms (oneof) and clarifies the relation to presetting a
flag and later defining it using create_prolog_flag/3. Still
needs to deal with flags that must be changed before booting
the system. Please have a look and comment on what you think
can be done better.
Some more types to must_be/2, has_type/2: compound(Term) and
type construction using conjunction, disjunction and negation.
So, we can do must_be((atom,\oneof([none])), X) to indicate
X shall be an atom, but not none.
More (and more streamlined) control over rewriting optimizations
as applied by library(apply_macros), library(debug) and library(clpfd).
More may follow.
Make show_coverage/2 using dir(SomeDir) to dump line-by-line
coverage analysis data work together with unit tests.
Several fixes to source code position reconstruction. Affects
the above mentioned coverage tools as well as the source level
debugger.
Fixes in LibBF arithmetic. In part platform independent, in
part for VS2022 and in part on 32-bit platforms.
Fixed a few crashes (undo/1, tracer ‘S’ command)
Fixed CVE-2023-45853 in included minizip source code. Isn’t
that relevant for our usage, see below for details.
Updated Emacs sweep interface.

Enjoy — Jan

P.s. Update of Ubuntu PPA is stalled due to bug
Bug #2043679 “dput is failing with 550 internal server error” : Bugs : Launchpad itself

SWI-Prolog Changelog since version 9.1.18

SECURITY: CVE-2023-45853 Upstream bug in the minizip library
used to create and load Prolog saved states. Not very relevant
to SWI-Prolog as it unlikely that SWI-Prolog is used in a way that
provides uncontrolled access to the involved parameters.

Upstream message:

Reject overflows of zip header fields in minizip.

This checks the lengths of the file name, extra field, and comment
that would be put in the zip headers, and rejects them if they are
too long. They are each limited to 65535 bytes in length by the zip
format. This also avoids possible buffer overflows if the provided
fields are too long.
ADDED: create_prolog_flag/3: type(oneof(ListOfAtoms)) This patch
also cleans up the relation to create_prolog_flag/3 and existence of
a flag with some value. Adds test cases.
MODIFIED: set_prolog_flag/2: convert Booleans If set_prolog_flag/2 is
used to create a new flag and the value is an atom that is recognised
as a Boolean, normalise the value.
ADDED: must_be/2 and friends: constructed types and compounds Allow
for e.g. compound(atom-integer) to express a pair mapping an atom to
an integer. Also introduces constructed types such as (atom;integer)
FIXED: #1205: sandbox:format_calls/3 to handle non-list format args
correctly.
ADDED: Provide a mechanism to warn on unused Prolog flags
ADDED: Allow -Dno-flag
MODIFIED: library(apply_macros) flags Now listens to optimise_apply,
which defaults to optimize, while the flag apply_macros_scope
defines whether the optimization applies to the files that import
this library or all files (default).
CLEANUP: Make optimise_debug flag library-only.
MODIFIED: Renamed Prolog flag clpfd_goal_expansion to
optimise_clpfd Improves consistency
FIXED: Source position preservation of string tereminals in DCG.
ADDED: Rewrite positions for function evaluation in meta-predicate
arguments This fixes source location handling for clauses that
call a meta-predicate where one of the non-meta arguments uses
e.g. Dict.attr.
FIXED: clause_info/4: perform full body matching.
FIXED: Clause position handling of plunit test/1,2 clauses.
FIXED: ‘$notrace’/2 + ‘$restore_trace’/2 pair to properly restore This
pair now reliably restores LCO and VMI instruction optimization such
that e.g., the coverage tools work properly after this pair is called.
ENHANCED: load library(apply_macros) globally on -O.
FIXED: #1204: undo/1: crash if a single undo is scheduled that fails.
FIXED: clause_info/5 for unification moved into the head
FIXED: clause/2: moved unifications If a unification is moved into
the head and the variable is unified against some other variable,
we must move the unification back.
MODIFIED: Control transformations by library(apply_macros). The new
Prolog flag apply_macros controls when this library rewrites the
code. Values are true (old default), false (no-op), optimise
(only wwith -O) and imported (only apply to modules where it is
explicitly imported to).
ADDED: Process -Dname=value to set Prolog flags.
FIXED: tracer S command to save the
current goal. Reported by Wolfram Diestel
(Stacktrace after hitting t S u while browsing through query solutions - #2 by wdiestel)
PORT: MSVC install: there is no libswipl.ddl.a.
PORT: Windows: deal with OpenSSL when using per-module CRT support.
ADDED: Swin_open_osfhandle()
PORT: Fixed integer addition overflow check when using MSVC
PORT: Allow using Windows HANDLE in communicating about Prolog streams.
BUILD: Make installation work again Broken with
71caaeb72188627f0d717d251cab14d3e557a241
BUILD: Simplify forced building of the library indices.
PORT: Improve support for Conda on Windows. Adds a flag
conda. Windows versions added %CONDA_PREFIX%\Library\bin and
%PREFIX%\Library\bin to dependent DLL search path. This extends
a similar work-around for MSYS2.
DOC: Non-ISO behavior of \+/1
PORT: Use uint64_t for LibBF mpz_ui_pow_ui() Switch to using the 64
bit versions on all platforms. Requires some more 64 bit init/set
versions, but overflow checking intrinsic __umulh() for MSVC only
exists of 64 bit.
FIXED: gcd function for LibBF on platforms with 32 bit long.
FIXED: mpz_ui_pow_ui() emulation on LibBF on systems with 32 bit long.
Breaks e.g. A is 2^65, succeeding with A = 0.
ADDED: PL_get_delay_list() API.
CLEANUP: PL_new_term_refs() now takes size_t argument (was int).
DOC: Fix return types of Sfread() and Sfwrite() (size_t instead of
int)

Package chr

BUILD: Another missing use_module for maplist/3.
BUILD: Avoid relying on autoloding during bootstrapping.

Package clib

ENHANCED: Make rewrite_host/3 hook work fot tcp_connect/3 Also allows
tcp_connect/3 to accept an IP number. These two enhancements avoid
the need to lookup localhost on Windows.
ADDED: socket:rewrite_host/3 hook for tcp_connect/2 This multifile
hook allows for mapping hostnames, either directly to an IP address or
another host. Currently a work-around for Windows broken handling of
localhost, mapping it directly to the IP address 127.0.0.1.
PORT: CRT handles for the process pipes must be owned by libswipl.dll.
PORT: Work around inconsistent definitions for ETIMEDOUT on Windows.
PORT: We cannot share Windows CRT stream handles with the Prolog core.
PORT: Windows: create_process/3: make stream(S) work. This patch
fixes the options stdin(stream(S)), etc. when S denotes a file stream
by enabling inheritance for the handle before calling ProcessCreate().
FIXED: process_create/3: Allow user_input (file_no 0) for stream(S)

Package cpp

PORT: Removed check_rc(). Seems not all C++ compilers can deal
with this.
ENHANCED: added unwrap() method and PlUnwrapPtr()

Package http

ENHANCED: http_stop_server/2: try to avoid connecting This sends
a signal to the HTTP accept thread. To make sure the signal is
processed it connects after the signal. Unfortunately this takes 2
seconds before raising an error on Windows. We now first check that
the signal is processed within 0.1 second. If so, we are happy.
If not, we try to connect anyway.

Package jpl

PORT: Avoid conversion messages and warnings for MSVC

Package plunit

FIXED: show_coverage/2: details for non-module files with tests.
A non-module file that defines a test was not considere a non-module
file due to the existence of the test module and as a result, coverage
details were no shown for such files.

Package ssl

PORT: Add OPENSSL_Applink() to ssl4pl.dll This function is used by
OpenSSL to get module-consistent of the MS CRT library. It is expected
at the application level. This patch tries to keep it locally in the
ssl binding with a small relay function from the SWI-Prolog executable.
PORT: Make use of OpenSSL “applink” Consistency use OpenSSL applink.c
if this is defined. This patch itself does not enable applink.
TEST: Allow running SSL test suite from main build directory.
PORT: Create test certificates when running under Conda

Package sweep

ADDED: in-buffer completions for arithmetic functions * sweep.pl
(sweep_function_functors_collection/2) (sweep_functions_collection/2):
New predicates. (sweep_context_callable_arg/4): Recognize arithmetic
functions.
- sweeprolog.el (sweeprolog-arith-functor-completion-candidates)
  (sweeprolog-arith-completion-candidates): New functions, used in…
  (sweeprolog–atom-or-functor-completion-at-point): …here.
- sweep.texi (Code Completion): Document arithmetic function
  completion.
- sweeprolog-tests.el: Test it.
ADDED: in-buffer completions for predicate options * sweep.pl
(sweep_option_functors_collection/2) (sweep_options_collection/2)
(sweep_option_arguments_collection/2): New predicates.
(sweep_context_callable_arg/4): Recognize predicate options.
- sweeprolog.el (sweeprolog-option-functor-completion-candidates)
  (sweeprolog-option-completion-candidates)
  (sweeprolog-option-arg-completion-candidates): New functions,
  used in…
  (sweeprolog–atom-or-functor-completion-at-point): …here.
- sweep.texi (Code Completion): Document predicate options completion.
- sweeprolog-tests.el: Test it.
ADDED: in-buffer completions for source file specs
- sweep.pl (sweep_alias_source_file_name_collection/2)
  (sweep_alias_source_file_name/4) (sweep_source_file_name_collection/2)
  (sweep_source_file_name/3) (sweep_path_alias_collection/2)
  (sweep_matching_path_alias/3): New predicates.
  (sweep_context_callable_/3): Simplify, and recognize predicate
  arguments that should be source file specifications.
- sweeprolog.el (sweeprolog-source-functor-completion-candidates)
  (sweeprolog-alias-source-completion-candidates)
  (sweeprolog-source-completion-candidates): New functions, used in…
  (sweeprolog–atom-or-functor-completion-at-point): …here.
  (sweeprolog-electric-layout-post-self-insert-function)
  (sweeprolog-context-menu-for-region)
  (sweeprolog-maybe-extract-region-to-predicate): Adapt.
- sweep.texi (Code Completion): Document file spec completion.
- sweeprolog-tests.el: Test it.
ENHANCED: Improve ‘sweeprolog-extract-region-to-predicate’ Teach
‘sweeprolog-extract-region-to-predicate’ about existentially quantified
goals and lambda terms.
- sweep.pl (sweep_term_variable_names/2): Remove from export list.
- sweep.pl (sweep_extract_goal/2): New public predicate.
- sweeprolog.el (sweeprolog-extract-region-to-predicate): Use it.
- sweeprolog-tests.el: Test it.
- sweep.texi (Extract Goal): Update documentation.
ADDED: new command ‘sweeprolog-query-replace-term’ This patch
extends the Term Search infrastructure to support search-and-replace
operations. Namely, we replace the predicate ‘sweep_term_search/2’
with a new predicate ‘sweep_term_replace/2’ that subsumes the
functionality of the former. The command ‘sweeprolog-term-search’
is greatly simplified and adjusted to work with this new predicate,
and we introduce a new command, ‘sweeprolog-query-replace-term’, that
exposes the new search-and-replace facility of ‘sweep_term_replace/2’
in an interactive UI inspired by ‘query-replace’.
- sweep.pl (sweep_term_search/2): Remove in favor of…
  (sweep_term_replace/2): New predicate.
- sweeprolog.el: (sweeprolog-term-replace-edits): New function.
  (sweeprolog-term-search): Rewrite.
  (sweeprolog-term-search-last-search)
  (sweeprolog-term-search-overlays)
  (sweeprolog-term-search-repeat-count)
  (sweeprolog-term-search-repeat-backward)
  (sweeprolog-term-search-repeat-forward)
  (sweeprolog-term-search-abort)
  (sweeprolog-term-search-in-buffer)
  (sweeprolog-term-search-next)
  (sweeprolog-term-search-map): Remove, unused.
  (sweeprolog-query-replace-term): New command.
  (sweeprolog-mode-map): Bind it.
- sweeprolog-tests.el (term-search)
- sweep.texi (Term Search): Adjust.
  (Term Replace): New section.
ENHANCED: Support pty top-level communication instead of TCP Add
the ability for top-level buffers to communicate with their backing
threads via pty instead of a local TCP connection. This is controlled
by a new user option, enabled by default on systems where Emacs can
creates a pty.
- sweep.pl (sweep_top_level_start_pty/2): New predicate.
- sweeprolog.el (sweeprolog-top-level-use-pty): New user option.
  (sweeprolog-top-level-buffer): Use it.
- sweep.texi (The Prolog Top-level): Document it.
ADDED: New command ‘sweeprolog-extract-region-to-predicate’ Add
a command for extracting a part of a clause body into a separate
predicate.
- sweep.pl (sweep_term_variable_names/2)
  (sweep_goal_may_cut/2): New helper predicates.
- sweeprolog.el (sweeprolog-extract-region-to-predicate): New command.
  (sweeprolog-maybe-extract-region-to-predicate): New function.
  (sweeprolog-insert-term-functions): Add it.
- sweep.texi (Insert Term DWIM, Extract Goal): Document it.

Package swipy

BUILD: Fix finding swipl for building pip Python package.
MODIFIED: py_call(Obj:Attr = Value) is now py_setattr(Obj, Attr,
Value). py_call/1 still accepts this construct, printing a deprecation
warning on the first encounter. Future versions will remove support.
PORT: Fixed MSVC emulation of __sync_bool_compare_and_swap() Could
lead to an infinite loop in the atom garbage collector thread.
ADDED: py_load/2 to define a Python module from Prolog The name should
not yet be considered “set in stone”
STANDARD: janus.cmd() Moved the implementation to C to get max
performance and return the truth value (True, False, janus.undefined).
MODIFIED: Renamed XSB compatibility functions janus.px_* to remove
the px_

Package xpce

FIXED: Huge spacing in PceEmacs on some fixed width fonts.

kwon-young · November 17, 2023, 6:56am

Thank you for your hard work !

About compound term types, the docuentation says:

Compound with same name/arity as term; checks arguments

How are arguments checked ?

jan · November 17, 2023, 8:20am

As types, so we can do compound(atom-integer) to specify a-97 to be valid and 2-42 to be invalid.

Not that this does not allow for recursive definitions. If we want to do proper typing we need a proper type language. These do exist. Possibly that should be the next step towards typing that would be acceptable to everyone.

kwon-young · November 17, 2023, 8:44am

Does this mean that we still can’t do things like this ?

compound(seg(
  compound(point(int, int)),
  compound(point(int, int)))).

Boris · November 17, 2023, 9:41am

After installing the relocatable application bundle I find on the downloads page, I see the following in my terminal:

$ swipl
Welcome to SWI-Prolog (threaded, 64 bits, version 9.1.18-70-g955571d43)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
Please run ?- license. for legal details.

For online help and background, visit https://www.swi-prolog.org
For built-in help, use ?- help(Topic). or ?- apropos(Word).

?- ^D
% halt
$ swipl --version
SWI-Prolog version 9.1.19 for fat-darwin

Note that 9.1.18-70-g955571d43 up there.

Not sure what happened.

Boris · November 17, 2023, 10:52am

We can do this:

foo(X) :-
    must_be(
        compound(
            seg(
                compound(point(integer,integer)),
                compound(point(integer,integer)))),
        X),
    format(user_error, "~w", [X]).

And with it:

?- foo(seg(point(1,2), point(3,4))).
seg(point(1,2),point(3,4))
true.

?- foo(seg(point(0.1,2), point(3,4))).
ERROR: Type error: `compound(seg(compound(point(integer,integer)),compound(point(integer,integer))))' expected, found `seg(point(0.1,2),point(3,4))' (a compound)

The error message is not super useful but at least it throws as it should.

I guess what you cannot do is define say a “stack” which would be something like (made up code):

stack(empty).
stack(Value, stack).

This stack is de facto a list but you still can’t define other recursive structures like for example a binary tree.

jan · November 17, 2023, 11:14am

Forgot to pull the git tag Thanks for noting. I should fix this in the release building script. It will remain like this for this version as security audits (and the server itself) doesn’t like executables with the same name to exists with different content.

Yes. Could be more human friendly

Indeed. To do that, you need something that allows referring to the type you are defining. As is, you can define this using the error:has_type/2 multifile predicate. I consider this a bit too ad-hoc to be considered a proper type language though. There are better type definition languages for Prolog around

kwon-young · November 17, 2023, 12:14pm

Do you have an example in mind ?
Things like lambda prolog ?
What would be the alternative of using the prolog database ?

jan · November 17, 2023, 12:53pm

Not particularly. There are type definitions in Ciao, the type library by Tom Schrijvers, Mercury and as you say lambda Prolog (I’m not familiar with its details). If we want to take typing to the next level, we need to define our goals and see what to choose or reuse.

Typing dynamic languages is a hard topic. Recent progress in other dynamic languages give some hope. But, Prolog comes with its own complications. First of all, rather satisfying some type or not, we have terms that (1) satisfy a type, (2) are compatible with a type and (3) are incompatible with a type. A term is compatible with a type if it is non-ground and can still get a shape that satisfies the type (thus, a variable is compatible with all types). So, instantiation (or modes) cannot be ignored. Second, a conjunction may refine types down the road, e.g., bla(X), atom(X), atom_concat(X, aap, Y) is type safe, even if bla/1 may succeed with non-atoms. What we would like here is to prove that bla/1 never succeeds with an atom, in which case we can warn the user this never succeeds or always succeeds with an atom, in which case we can tell the user that the atom/1 call is redundant. Probably work on dynamically typed languages do this correctly as such languages can implement control based on dynamic type tests.

My take on this is that we should provide an easy to use type, mode and determinism language. That can first be used as basis for documentation and can later be used for dynamic type checking and static type analysis. Ciao is beyond doubt most advanced here. I have the impression it could/should be done simpler, but maybe I’m wrong. It is quite a while ago I looked at the Ciao assertion language. There is a tradeoff between simplicity and preciseness here. I lean towards simplicity, accepting that we cannot proof type (in)correctness of more code.

Marco · November 17, 2023, 4:14pm

% python
Python 3.11.6 (main, Oct 17 2023, 14:36:46) [Clang 15.0.0 (clang-1500.0.40.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import janus_swi as janus
ERROR: Exported procedure janus:py_setattr/3 is not defined
>>>

Now I get this error message I didn’t get before.

Or maybe it’s my fault and I have to update something…

jan · November 17, 2023, 4:35pm

You have to re-build the Python module using pip. If you got the whole thing setup, go to packages/swipy in the source tree and run

 pip install .

jan · November 17, 2023, 5:15pm

It is to me still an open question what to do here. Should the build process (by default) check for pip and try to build the Python package? Is something like that considered good practice? One of the issue is that pip install . actually installs. Possibly better is to do pip wheel . to just build the package. How do I then install this wheel? It is also problematic that the pip build creates files in the source directory. Is there a way around that?

Marco · November 17, 2023, 5:57pm

You’re not asking me, I guess…you’re asking the community. The matter is extremely technical and it’s far above my head and my competence so to say. Nonetheless I reinstalled and now I don’t get the error message anymore

kwon-young · November 18, 2023, 2:11am

Just for curiosity, how is the list type defined in swi prolog ?

jan · November 18, 2023, 12:19pm

See library(error). “types” are defined as rules for has_type/2. The list type is simply defined by the built-in predicate is_list/1. There is much more to say about types than SWI-Prolog does …

Topic		Replies	Views
Ann: SWI-Prolog 9.1.20 Releases	23	333	December 11, 2023
Ann: SWI-Prolog 9.1.22 Releases	8	257	January 30, 2024
Ann: SWI-Prolog 9.1.16 Releases	11	420	September 30, 2023
Ann: SWI-Prolog 9.1.15 Releases	2	252	September 7, 2023
Ann: SWI-Prolog 9.1.18 Releases	3	224	November 2, 2023