SWI-Prolog 8.5.17 is ready for download. For the normal Prolog
distribution there is not much new. Most of the work is in the WASM port
that allows running Prolog in your browser. As a result there have been
quite a few configuration and testing changes to better support single
threaded and otherwise limited builds.
For the normal Prolog version there merely two fixes. One is a
regression in the Windows version after the work on improving the
Windows file system interaction: open('NUL', ...) or in general
accessing the reserved file names in Windows was broken (reported by Paulo Moura). Second issue is
a possible memory corruption in shared tabling which used by
library(pcre).
The WASM version has been improved a lot. The interfaces should be
fairly stable now. They are documented in the main manual. One of the
reasons behind this release is to make this documentation available. The
docs should come online at https://www.swi-prolog.org/pldoc/man?section=wasm
In a nutshell the status for the WASM version is
The build process is now fairly smooth, at least on Linux and MacOS
The #wasm_demo shell (REPL) demo shows that all basic interaction
is possible. The demo itself is very limited, notably due to the
use of a simple <textarea> for editing.
The port of @PaulBrownMagicâs application from Tau-Prolog show that
event processing and DOM manipulation work. A partial Tau-Prolog
compatible library(dom) illustrates how the interaction works.
There are high level friendly interfaces to call between
JavaScript and Prolog. Calls can be in both directions and may
be nested (JavaScript calling Prolog calling JavaScript, etc.)
We can load Prolog code from a URL, e.g., from GitHub raw content.
This in addition to loading Prolog code from <script> elements,
JavaScript strings or downloading to the local filesystem and loading.
Enjoy â Jan
SWI-Prolog Changelog since version 8.5.16
DOC: More WASM documentation.
ENHANCED: notrace/1: avoid callback through C. This avoids possible
C stack overflows and allows yielding from below notrace/1.
MODIFIED: WASM: Renamed js_yield/2 to await/2 and js_can_yield/2
to is_async/2.
FIXED: Possibly dangling tref->trie pointer when shared tabling
creates the same variant table in two threads and disposes one of them.
This hopefully addresses issue#1041. It surely fixes a bug though.
FIXED: Possible corruption of tries. By using PL_BLOB_UNIQUE we
good reuse an atom after freeing the trie and creating a new one at
the same address. As we save the symbol anyway, making use of unique
atoms is meaningless here.
WASM: Make â$confirmâ/2 hookable and implement it by means of the
browser confirm() function.
WASM: Make URL compilation work on slow connections and connections
that do not provide a modification time through a HEAD request.
FIXED: Exception preservation in undo/1
WASM: Allow compiling from a URL.
WASM: js_yield/2: if the result is a single JavaScript string, return
it as a Prolog string for reducing garbage.
FIXED: source_file/2 in mode (-,+) never reported any files.
Breaks coverage analysis tool.
FIXED: on_signal/3 for SIGINT called the handler asynchronously.
This should only happen if SIGINT is bound to the debug
handler. Reported by Raivo Laanemets.
WASM: Added is_object/1,2 to test Prolog terms for being a JavaScript
object reference (of some class).
WASM: Added pseudo function instanceof() to expression evaluation.
WASM: Ported Paul Brownâs Tau-Prolog example.
ADDED: library(dom), providing Tau-Prolog compatible DOM
interaction. Partial implementation.
WASM: Added Prolog.load_string() and Prolog.load_scripts()
WASM: Typos in docs and mixup of arguments in rational numbers.
WASM: Re-implement sleep/1 using an abortable Promise.
WASM: Introduce the notion of abortable promises.
WASM: Wrap js_yield/2 to forward exceptions.
DOC: Started adding WASM documentation to main manual
WASM: Added js_can_yield/0.
WASM: Use a WeakMap rather than additional attributes to get the
reverse object --> id mapping
WASM: Return all objects except for instances of âObjectâ by reference
rather than as a dict.
WASM: Print blobs that embed JavaScript objecs as <js_Class>(Ref)
rather than <js_object>(Ref).
ADDED: js_yield/2 may be passed a promise, causing Prolog to wakeup
when the promise is resolved.
ENHANCED: PL_unify_bool() accepts same atom values as
PL_unify_bool_ex() and PL_get_bool()
ENHANCED: js_yield: get return value as term.
ENHANCED: js_yield: allow passing a term rather than a string.
INSTALL: Windows: Copied deleting the Uninstall registry key to the
64 bit section.
WASM: Much restructuring to prolog.js. Add Prolog.forEach() and
Prolog.abort() to remain responsive when running slow queries.
FIXED: Backtracking after a foreign yield.
DOC: remove outdated restriction regarding and {} as operators
ENHANCED: prolog_colour:colourise_db/3: process asserted bodies
FIXED: prolog_colour:qualify_op/3: recognize as an operator.
Use SSU rules.
CLEANUP: Moved default output routines to the core and Prolog
class. Simpified and documented the shell.html and test.html examples.
WASM: Simplify starting the WASM version by moving more defaults into swipl-web.js.
FIXED: Windows: this registry key is not deleted by Uninstaller.exe
Unistalling swipl at Windows 10 Add/Remove App window should now work
DOC: Add missing failure return info for more more foreign functions.
WASM: Use an unbound tag for dicts created from JavaScript objects.
If the class is not âObjectâ, bind the Prolog tag to the JavaScript
class name.
WASM: Introduce classes for Prolog objects we cannot represent
naturally in JavaScript such as variables, compounds, etc.
CLEANUP: Large rewrite of WASM prolog.js to use ES6 class syntax
for the Prolog class.
MODIFIED: WASM: representation of rational numbers.
WASM: Use -s BIGINT to exchange Prolog integers up to 64 bits.
Update Prolog.get_integer() to return values as JavaScript number when
possible and JavaScript bigint otherwise. Updated Prolog.toProlog()
to convert bigint to a Prolog large integer.
TEST: Added test for Windows NUL files
FIXED: Issue#153: Windows: can no longer open reserved files such as nul. Reported by Paulo Moura.
Package http
WASM: Support more the HTML related infrastructure.
PACKAGE: Include http_quasiquotations.pl and the term_html.pl imported
from the Pengines package into the minimal installation.
Package ltx2htm
FIXED: Memory error.
Package pengines
PACKAGE: Move term_html.pl to the http package. It is not specific
for Pengines.
Package xpce
FIXED: PceEmacs sgml mode: use setup_call_cleanup/3 to ensure Prolog
streams to editors are always closed.
little problem in packages/xpce (already submitted a PR). The problem did not show up earlier because I compiled with SWIPL_PACKAGES_X=OFF in the past
long path name support does not yet seem to work. It can also be reproduced with the normal swipl-win from the Windows Installer on swi-prolog.org, see below.
make_directory with paths of length 1, 200, and 200+200
?- make_directory(a).
true.
?- make_directory(a123456789b123456789c123456789d123456789e123456789f123456789g123456789h123456789i123456789j123456789k123456789l123456789m123456789n123456789o123456789p123456789q123456789r123456789s123456789t123456789).
true.
?- make_directory('a123456789b123456789c123456789d123456789e123456789f123456789g123456789h123456789i123456789j123456789k123456789l123456789m123456789n123456789o123456789p123456789q123456789r123456789s123456789t123456789/a123456789b123456789c123456789d123456789e123456789f123456789g123456789h123456789i123456789j123456789k123456789l123456789m123456789n123456789o123456789p123456789q123456789r123456789s123456789t123456789').
ERROR: directory `'a123456789b123456789c123456789d123456789e123456789f123456789g123456789h123456789i123456789j123456789k123456789l123456789m123456789n123456789o123456789p123456789q123456789r123456789s123456789t123456789/a123456789b123456789c123456789d123456789e123456789f123456789g123456789h123456789i123456789j123456789k123456789l123456789m123456789n123456789o123456789p123456789q123456789r123456789s123456789t123456789'' does not exist (No such file or directory)
I know. This seems a Windows limitation though. The low level is _wmkdir() and is correctly passed \\?\ + <the directory name> as a canonical absolute path. It works fine under Linux Wine, but indeed under Windows 11 it fails as you describe. The other issue is that most of these functions report too long names as ENOENT, meaning âexistence errorâ I donât know what else I can do. If anyone knows how to improve on this, let me know.
The bash msys shell accepts the long paths in mkdir, so there should be some way to do this. Iâll try to debug this later today and report the result.
FWIW, my system is Win 10, not 11, but I guess that does not make a difference.
Another minor thing is a uninitialized warning in packages/xpce/src/gnu/getdate.c, for your kind consideration:
Somewhere at line 850/851: if(yylen > 0) yyval = etc. The code in getdate.c seems to be parser-generated, therefore difficult to read. It seems, however, that thereâs a more recent version available, which I think isnât GPL either, e.g., here: Chrony: getdate.c | Fossies (see the comment at line 21).
There, the condition if(yylen > 0) is omitted with a comment. Maybe it can be used instead.
getdate.c says âThis code is in the public domain and has no copyrightâ Iâm happy to make minor patches to it, but otherwise I intend to avoid touching this code as much as possible. None of my compilers complain about this. If yourâs does, initializing at declaration with a comment seems the most simple solution.
Ah, ok, but the ânewâ code at the fossies archive says the same. But I see your other point
(rather not touch it)
69 /* First part of user prologue. */
70 #line 1 "getdate.y"
71
72 /*
73 ** Originally written by Steven M. Bellovin <[smb@research.att.com](mailto:smb@research.att.com)> while
74 ** at the University of North Carolina at Chapel Hill. Later tweaked by
75 ** a couple of people on Usenet. Completely overhauled by Rich $alz
76 ** <[rsalz@bbn.com](mailto:rsalz@bbn.com)> and Jim Berets <[jberets@bbn.com](mailto:jberets@bbn.com)> in August, 1990.
77 **
78 ** This code is in the public domain and has no copyright.
Actually, no. On my system, the path that is sent to _wmkdir is not absolute, and thereâs no \\?\ prefix. I was able to make this to work on my Windows 10 using the, well, pseudocode below. Sorry for my bad C, but I hope it may be helpful anyway:
int
_xos_mkdir(const char *path, int mode) // aaaa
{ TCHAR buf[PATH_MAX];
char buf1[PATH_MAX];
if ( !_xos_absolute_filename(path, buf1, PATH_MAX) ) // c:/msys64/home/matthias/aaaa
return -1;
char buf2[PATH_MAX];
if ( !_xos_os_filename(buf1, buf2, PATH_MAX) ) // c:\msys64\home\matthias\aaaa
return -1;
char buf3[PATH_MAX];
sprintf(buf3, "\\\\?\\%s", buf2) ; // \\?\c:\msys64\home\matthias\aaaa
if( !_xos_os_filenameW(buf3, buf, PATH_MAX) ) // Same in UTF16
return -1 ;
return _wmkdir(buf);
}
This were my use cases:
$ src/swipl
Welcome to SWI-Prolog (threaded, 64 bits, version 8.5.17-3-g7f90f6d1f-DIRTY)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
Please run ?- license. for legal details.
CMake built from "c:/msys64/home/c7201178/swipl-devel/build"
For online help and background, visit https://www.swi-prolog.org
For built-in help, use ?- help(Topic). or ?- apropos(Word).
make_directory(aaaa).
true.
make_directory('aaaa/aaaa').
true.
make_directory(a123456789b123456789c123456789d123456789e123456789f123456789g123456789h123456789a123456789b123456789c123456789d123456789e123456789f123456789g123456789h123456789).
true.
make_directory('a123456789b123456789c123456789d123456789e123456789f123456789g123456789h123456789a123456789b123456789c123456789d123456789e123456789f123456789g123456789h123456789/a123456789b123456789c123456789d123456789e123456789f123456789g123456789h123456789a123456789b123456789c123456789d123456789e123456789f123456789g123456789h123456789').
true.
halt.
PS. If I want to delete these newly created folders from the Windows Explorer, the Windows âWastebasketâ complains that the file path is too long
Oops. Thanks for catching this. The patch to deal with NUL, etc. broke this and I didnât notice because the tests are normally executed on Linux+Wine which is handles long files much better than Windows itself
Which fails with a strange error message under both Linux and MSYS:
ERROR: c:/msys64/home/c7201178/swipl-devel/src/tests/files/test_file_names.pl:119:
test 5 is subject to occurs check (STO):
ERROR: Finite trees (error checking): deterministic success in 0.00 seconds
ERROR: Rational trees: received error: make_directory/1: directory `aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' does not exist (File exists)
ERROR: Finite trees: received error: make_directory/1: directory `aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' does not exist (File exists)
Under Linux, it fails, as well, with a less strange error message.
I am wondering if this level of complexity* is needed for the unit test. Would you mind if I simplify them a bit?
*I mean expressions such as make_directory(/Dir/(long(100)=Dir2))
How do you run these? For me these tests run just fine. By default tests are executed under different occurs check settings. That is rather useless in this case. It seems the cleanup step is not executed correctly and thus it tries to create directories that already exist. My log on Windows 11 are below. Started by hand as I do not have ctest on that machine
PS P:\build.win64> src/swipl.exe ..\src\test.pl --no-core files
% SWI-Prolog test suite.
% To run all tests run ?- test.
%
Running scripts from files (test_file_names.pl).....[1.348sec].(test_files.pl).................[0.279sec].(test_glob.pl)..............................[0.026sec].(test_pipe.pl)...[0.085sec]. done
% Started at Thu Sep 22 15:08:04 2022
% 0.953 seconds cpu time for 1,904,281 inferences
% 7,262 atoms, 5,282 functors, 4,404 predicates, 69 modules, 185,808 VM-codes
%
% Limit Allocated In use
% Local stack: - 20 Kb 2,312 b
% Global stack: - 28 Kb 6,568 b
% Trail stack: - 30 Kb 0 b
% Total: 1,024 Mb 78 Kb 8,880 b
%
% 22 garbage collections gained 10,110,200 bytes in 0.016 seconds.
% 29 atom garbage collections gained 3,572 atoms in 0.000 seconds.
% 32 clause garbage collections gained 620 clauses in 0.016 seconds.
% Stack shifts: 2 local, 3 global, 6 trail in 0.000 seconds
% 2 threads, 0 finished threads used 0.000 seconds
All tests passed
Interesting You can run ctest -V -R files to get the full command line and then figure out which difference to the above simplified command makes the difference.
Surely the ls test, using the opendir() emulation with uses FindFirstFileW(), etc. is one of the Windows primitives that doesnât seem to like long paths very well I have no clue why. Didnât find some other way to enumerate the directory members that might work âŚ
Quick question: What about using exists_directory/1 or delete_directory/1 for testing? (instead of ls). I see that this may hide other, underlying problems, of course.
The cleanup is done by delete_directory_and_contents(TestDir), which in the end uses directory_files/2. The ls is a test to see whether we can correctly get the contents of a directory. That too uses directory_files/2. directory_files/2 is implemented using opendir() which uses FindFirstFileW().
Yes. That is also my conclusion. Now the question becomes whether it is me (still) doing something wrong or it is just Windows being broken? I think my calls to FindFirstFileW() are as good as it gets and I havenât found an alternative. On the other hand, the Windows explorer can enter the long directories. One could consider changing the working directory and use relative paths, but SetCurrentDirectoryW() is deprecated AFAIK and definitely doesnât like long paths. Microsoft is right that changing working directory in threaded applications is typically a bad idea.
but it does not like double backslashes as directory separators such as e.g. \build\packages\\*
The patch below works, although it might be a better idea not to invoke openfile with a directory that has a trailing backslash.
opendir(const char *path)
{ TCHAR buf[PATH_MAX];
TCHAR *edir;
DIR *dp = malloc(sizeof(*dp));
if ( !dp )
{ errno = ENOMEM;
return NULL;
}
/* -2: make sure there is space for the "\*" */
if ( !_xos_os_filenameW(path, buf, PATH_MAX-2) )
{ free(dp);
return NULL;
}
// In case there's already a backslash at the end, append only a "*"
edir = buf+_tcslen(buf);
if('\\' == buf[_tcslen(buf)-1])
_tcscat(edir, _T("*")) ;
else
_tcscat(edir, _T("\\*"));
if ( !(dp->data = malloc(sizeof(WIN32_FIND_DATA))) )
{ free(dp);
errno = ENOMEM;
return NULL;
}
dp->first = TRUE;
// wchar_t *pattern = buf+_xos_win_prefix_length(buf); /* see (*) */ // do not cut the \\?\
dp->handle = FindFirstFileExW(/* pattern */ buf, // use buf not pattern
FindExInfoBasic, dp->data,
FindExSearchNameMatch, NULL,
FIND_FIRST_EX_LARGE_FETCH);
if ( dp->handle == INVALID_HANDLE_VALUE )
{ *edir = 0;
if ( _waccess(buf, R_OK) ) /* Dir does not exist or is unreadable */
{ closedir(dp);
return NULL;
}
}
return dp;
}
You saved my day! Seems I overlooked several things Test now passes except that I had to prevent the test suite from trying to run the relative file access tests from deep directories for Windows. Passing "\\?\" to SetCurrentDirectoryW() gives some interesting results as that causes processes started from SWI-Prolog to get the directory including this prefix and many applications do not like that. Who knows though, maybe I missing something too
I donât know what else I can do. If anyone knows how to improve on this, let me know.
Iâm happy to make minor patches to it, but otherwise I intend to avoid touching this code as much as possible. None of my compilers complain about this. If yourâs does, initializing at declaration with a comment seems the most simple solution.