Ann: SWI-Prolog 9.3.35

[jan]

Dear SWI-Prolog user,

SWI-Prolog 9.3.35 is ready for download. This is a pre-release for
10.0.0. As from now, only minor patches will be applied until
version 10 is out and a new development branch is created. Highlights:

• Sync format/1-3 with PIP-0110, adding additional tests produced
  by Joachim Schimpf. This fixes a couple of bugs and adds some
  minor features. The only incompatible change is that ~Ns now
  performs right padding with spaces if the argument is shorter than
  N characters.

• process_create/3 got a pseudo path prolog, allowing for
  process_create(prolog(‘swipl-ld’), …) to reliably call the
  SWI-Prolog linker frontend that belongs to the running Prolog
  process. After discussion with @mgondan1 to solve issues running
  the saved-state tests.

• Make the whole system compile using MSVC on Windows.

• Better MacOS Homebrew support for using the dependencies from
  Homebrew.

• Various fixes to the commandline handling. Line editing now by
  default use a bundled version of libedit that works on Windows
  and includes upstream patches to define the word characters
  that are available in most distributions yet. Also fixes
  colour handling for the flatpak version. Fix issues with
  resizing the Epilog windows.

• Added library(json_schema), providing JSON schema checking.

• XPCE:

  • Remove old PostScript generation, add PDF generation based
    on the Cairo library.
  • Make the user guide build again and update some vital sections.
  • Various minor fixes.

  Enjoy — Jan

SWI-Prolog Changelog since version 9.3.34

• ADDED: Using the option --home=dir, set SWI_HOME_DIR.
  This ensures that sub processes can access Prolog’s home and, when
  calling Prolog, execute the same Prolog system.

• TEST: Base saved-state tests on new prolog(Tool) option of
  process_create/3.

• FIXED: Allow swipl -o exe -c input.pl to pass --home=dir
  option.

• PORT: Properly install runtime dependencies for MSVC build This now
  depends in CMake > 3.21 install() option RUNTIME_DEPENDENCY_SET

• PORT: Build swipl-win.exe under MSVC Now compiles cleanly.
  swipl.exe runs fine, including line editing. The GUI is unresponsive
  though.

• FIXED: format/1-3: reset last tab stop on a newline. Spotted by
  Joachim Schimpf with the PIP tests.

• ADDED: ansi_format/4, formatting with colours to a stream.

• PORT: MSVC

• MODIFIED: format/2 handling of non-list arguments Now, the second
  argument is only considered a single argument if it is not or [|].
  Notable partial lists or lists not ending in [] are now threaded as
  (invalid) argument list.

• ENHANCED: Errors for arithmetic expression [Code] if list holds no
  single code.

• PORT: MSVC compatibility.

• TEST: Fixed test for format/2 ~6s.

• DOC: format and friends. Move documentation of deprecated writef/2
  and friends to the library documentation. Updated the format/1-3
  documentation.

• PIP: PIP-0110: make ~<n>s emit padding if text is too short.

• TEST: Use swipl.bat for testing saved states on Windows. 1. allows
  for “external” swipl.bat under Windows (see swipl.sh in l. 133)
  2. invokes saved state with -x if swipl.bat is used

• PORT: Work around broken MSVC

• TEST: PIP-0110: added PIP format/2 tests.

• COMPAT: Extended ECLiPSe test driver to accept should_output

• MODIFIED: PIP-0110: format/2 using ~N to ignore the argument.
  Most implementations ignore the argument. Others interpret it
  inconsistently. New code should use ~N~2n as replacement for
  ~3N.

• MODIFIED: PIP-0110: distributing extra spaces over ~t Used to
  distribute these spaces from the middle. Now distributes them from
  the right, compliant with SICStus and ECLiPSe.

• ADDED: PIP-0110: format/2 format specifier F. This is like f,
  but using uppercase letters for non-normal floats.

• PIP: PIP-0110: format/2: ~a to only accept atoms and compact strings.

• PIP: PIP-0110 (format) compatibility: handle expressions for *

• PORT: Better MacOS Homebrew support

• FIXED: Typo in updated history code. Caused !/0 to be redefined …

• FIXED: !<num> history handling if command contains a . in the
  middle.

• FIXED: library(ansi_term): use initialization/1 If not used, the
  setup init_color_term_flag/0 will not run when using .qlf files.

Package bdb

• PORT: Honour LIBBDB_ROOT_DIR cmake variable

Package clib

• ADDED: process_create/3: specify program as prolog(Tool) This allows
  Prolog running one of its tools, with the guarantee that we use the
  tools from the same version. This provides a hook prolog:prolog_tool/4
  that allows embedded systems to redefine how the Prolog tools should
  be executed.

• FIXED: Make library(uuid) work properly if autoloading is disabled.

Package json

• TEST: Skip JSON schema tests if we have no HTTP server

• BUILD: Only include library(json_schema) if pcre is provided.

• FIXED: Make compile if there is no HTTP client support.

• ADDED: library(json_schema) to provide checking JSON schema.

Package libedit

• PORT: Clean compilation on MSVC

• PORT: Allow using bundled libedit This is the default. The system
  version can be selected using cmake -DSYSTEM_LIBEDIT=ON.
  This should simplify building

• ADDED: el_set/2 to set the word characters This replies on el_wset()
  using EL_WORDCHARS that is provided in some recent copies of
  libedit. When available, it is used to set the word chars to only
  "_" (in addition to iswalnum()).

• FIXED: Avoid full refresh in Epilog windows on each action. Failed to
  reset the refresh request. Refreshing causes issues with wrapped
  lines.

• FIXED: Synchronise Epilog window size changes with libedit.

Package pldoc

• FIXED: Emit LaTeX for markdown code text that holds % in tables.

Package utf8proc

• PORT: Compile using MSVC

Package xpce

• PORT: Fixed emulation of C11 atomic_compare_exchange_strong()

• PORT: Compilation in MSVC This is only a partial port to MSVC.
  It does fix several issues though.

• FIXED: If there is no gui, do not setup xpce for lazy loading.

• PORT: Hack around MacOS oddities to get SIGWINCH

• PORT: Make compilation succeed if SIGWINCH is not defined.

• FIXED: Message handling. Broke emacs/0. Reported by @mike.elston

• FIXED: swipl-win to handle interrupt in the console that started
  swipl-win. What should we do if there is no terminal, i.e., if
  swipl-win is started as app directly from the GUI? It should not
  get interrupt signals in that case, but what if it gets one anyway?

• ADDED: Epilog: use pthread_kill() to update client thread of window
  size change. This fixes resizing Epilog windows cooperation with
  libedit. To work correctly, the libedit package must be updated
  as well.

• ENHANCED: Detection of when to load library(pce).

• ADDED: Epilog: Shift-Ctrl-M to inject make/0.

• MODIFIED: Removed all PostScript related support This was
  non-functional anyway. The Cairo graphics is used to implement
  graphical->pdf for generating PDF. This functionality is still
  rather minimal. It will be extended in the future.

• FIXED: Colour for class int_item up/down arrows. And some code
  cleanup.

• ENHANCED: class font_item to reflect current possibilities better.

• ENHANCED: Updated PceDraw - Use native file dialog for save, load
  and export - Remove windows metafile support - Replace Postscript
  output with PDF - Disable printing. Not supported by SDL and not
  very important.

• DOC: Revive the XPCE User Guide This commit re-adds the Makefile,
  updates the Prolog helpers and minimal updates to reflect some aspects
  of XPCE version 7 in the documentation.

  Eventually the build process must be migrated to CMake, most images
  should be regenerated and the text reviewed and updated where needed.

• ADDED: graphical->pdf to write PDF to a file

• FIXED: Always enable color on the Epilog consoles.

[mgondan1]

I experience a problem with saved states under MSYS2. Maybe more than one problem, but let’s fix it one by one.

1. After compilation, src/swipl succeeds, also check_installation, so “build_home.pl” does everything correctly.
2. ctest fails, though. It creates the saved state, but reports problems with running it. I’ll be more specific later.

$ ctest -R save

3. I added a debug(save(keep)) to test_saved_states.pl so that I can run the exe-files manually.

$ ./test_saved_1_9440.exe
ERROR: [Thread main] c:/msys64/home/matth/mingw-packages/mingw-w64-swi-prolog/src/build-ucrt64/home/library/ext/clib/filesex.pl:91: Initialization goal raised exception:
ERROR: [Thread main] Das angegebene Modul wurde nicht gefunden. 

3. (this should read as “4.“): If I manually copy files.dll in the current folder, the state executes normally.

$ cp src/files.dll .
$ ./test_saved_1_9440.exe
true.

Some “win_add_dll_directory” seems to be missing, that adds ~/mingw-packages/mingw-w64-swi-prolog/src/build-UCRT64/src to the DLL search path. But where do I put this directive? I guess it needs to be placed in the “ressource” of the saved state (?). Moreover, I don’t know from where to obtain this path.

1. For what it’s worth, the state executes normally with swipl -x (even if files.dll is not in the current folder):

$ rm files.dll .
$ src/swipl -x ./test_saved_1_9440.exe
true.

2. I can fix the problem by adding –foreign=copy when creating the state, but that’s not pretty.

[mgondan1]

Ok, I was in error, sorry. (I see you typing)

[jan]

I see. This is a bit weird as the logic for running the state has not changed. The preparation is done by set_windows_path/0 in test_saved_states.pl.

@mgondan1

Ok, I was in error, sorry. (I see you typing)

I’ll wait …

[mgondan1]

The problem is solved by set_windows_path in test_saved_states (exactly as you write). I’ll play around a bit more and then report the second problem.

[jan]

@mgondan1

The problem is solved by set_windows_path in test_saved_states.

Now I’m confused. That is already there, so what needs to change? Anyway, I guess I’ll see a PR?

[mgondan1]

I hope I can provide something meaningful.

1. The first thing that comes to my attention is that SWI_HOME_DIR is set explicitly under msys2.* This does not seem necessary anymore (?), and may even be wrong in those situations where prolog_tool points to a different prolog system with its own home directory.

*I am aware that I suggested to introduce it. Sigh.

2. All right. Found the culprit: set_stream(In, timeout(60)) does not seem to work under Windows. Do you mind if I “comment it out“ like this?

read_terms(In, List) :-
    (   current_prolog_flag(windows, true)
    ->  true
    ;   set_stream(In, timeout(60))
    ),
    read_term(In, T0, []),
    read_terms(T0, In, List).

PR is on the way

[jan]

@mgondan1

The first thing that comes to my attention is that SWI_HOME_DIR is set explicitly under msys2.* This does not seem necessary anymore (?), and may even be wrong in those situations where prolog_tool points to a different prolog system with its own home directory.

Looks safe to me. The Prolog tool is designed to run tools that belong to the same Prolog installation, so it should have the same home. Of course you can make the hook do something else …

@mgondan1

set_stream(In, timeout(60)) does not seem to work under Windows.

It is indeed something that was added in this version of the tests to ensure the test terminates. Hmm. Works fine in my build. What are the symptoms? I rather have this fixed then disabling it

[mgondan1]

That is the complaint (“not a socket”):

% [1/3] saved_state:true .......................... **FAILED (0.586 sec)
ERROR: [Thread main] c:/msys64/home/c7201178/swipl-devel/tests/save/test_saved_states.pl:229:
ERROR: [Thread main]     test saved_state:true: received error: read_term/3: I/O error in read on stream <stream>(0000021965146690) (not a socket)

It goes away if I remove the set_stream/2 command. FWIW, you might wish to reduce the timeout to 5 seconds instead of 60

[jan]

@mgondan1

That is the complaint (“not a socket”):

Ok. I was wrong. The tests worked on Windows, but after adding the timeout I did not recheck. Sorry. So, your PR is fine. Can you lift the Draft state?

And no, this cannot be fixed as th underlying poll/select functionality is only present for sockets on Windows.

@mgondan1

FWIW, you might wish to reduce the timeout to 5 seconds instead of 60

No. Some of the builds and tests are executed on extremely slow (often emulated) hardware. It is just to avoid blocking for hours.

[mgondan1]

I think swipl-9.3.35.tar.gz lacks the contents of packages/libedit/libedit. There’s just a single file, CMakeLists.txt. In contrast, swipl-devel from git has a few more.

c7201178@PC105-C720 UCRT64 ~/MINGW-packages/mingw-w64-swi-prolog/src/swipl-9.3.35
$ ls packages/libedit/libedit/
CMakeLists.txt

c7201178@PC105-C720 UCRT64 ~/MINGW-packages/mingw-w64-swi-prolog/src/swipl-9.3.35
$ ls ~/swipl-devel/packages/libedit/libedit
CMakeLists.txt  ChangeLog  Makefile.am  README.md  acinclude.m4  cmake    config.guess    config.h.in  configure     depcomp  examples    libedit.pc.in  m4       src
COPYING         INSTALL    Makefile.in  THANKS     aclocal.m4    compile  config.h.cmake  config.sub   configure.ac  doc      install-sh  ltmain.sh      missing

[jan]

@mgondan1

I think swipl-9.3.35.tar.gz lacks the contents of packages/libedit/libedit.

True. This has been fixed in scripts/make-src-tape. That is a problem for Windows builds based on the archive. On other systems it will fall back to the system libedit. The only way to fix that is releasing 9.3.36 …

[mgondan1]

Success!

With two little patches (timeout, see above, and the lack of winlibedit), swipl-9.3.35 now runs on MSYS2 with check_installation and test_installation passing all tests—except for jpl and java, which I haven’t looked at yet, and won’t look soon.