That’s it, exactly. I replaced all backslashes by forward slashes and the problem has gone. I guess we have a c function that does this in a proper way (and is reachable by pl-nt.c), which is it?
(sorry for being so helpless)…
Pushed 2a45eabceaa649d8266609f80e82f27c754512b8 to address this. If you configure for debugging using cmake -DCMAKE_BUILD_TYPE=Debug, you can run
swipl -d MSG_INITIALISE
to see some of the steps taken to find the home directory. This patch improves some of the messages.
Thanks a lot. Moving on. The next one seems suspiciously simple:
$ ~/swipl/bin/swipl --dump-runtime-variables
...
SWIPL_PACK_PATH="c:\users\matth\appdata\local\swi-prolog\pack;c:\programdata\swi-prolog\pack";
etc.
This makes perfect sense for a classical Windows installation, less so for MSYS2 in which I want the system to search for packs in c:/msys64/home/matth/.local/swi-prolog. My approach is as follows: I disable the windows folders in boot/init.pl, especially because they include a cut.
% data
'$xdg_directory'(data, Home) :-
current_prolog_flag(windows, true),
\+ current_prolog_flag(msys2, true),
catch(win_folder(local_appdata, Home), _, fail),
!.
'$xdg_directory'(data, Home) :-
getenv('XDG_DATA_HOME', Home).
'$xdg_directory'(data, Home) :-
expand_file_name('~/.local', [Local]),
'$make_config_dir'(Local),
atom_concat(Local, '/share', Home),
'$make_config_dir'(Home).
% common data
'$xdg_directory'(common_data, Dir) :-
current_prolog_flag(windows, true),
\+ current_prolog_flag(msys2, true),
catch(win_folder(common_appdata, Dir), _, fail),
!.
'$xdg_directory'(common_data, Dir) :-
'$existing_dir_from_env_path'('XDG_DATA_DIRS',
[ '/usr/local/share',
'/usr/share'
],
Dir).
Note that MSYS2 is not a Unix emulator, it is actually just MinGW + a bash and a few utilities. Therefore, I actually want swipl.exe to search for packs in the usual Windows places (at least if swipl is not invoked in the MSYS2 environment). Therefore, I add the Windows paths to the end:
(bit below)
% Append Windows folders under msys2
'$xdg_directory'(config, Home) :-
current_prolog_flag(msys2, true),
catch(win_folder(appdata, Home), _, fail).
'$xdg_directory'(data, Home) :-
current_prolog_flag(msys2, true),
catch(win_folder(local_appdata, Home), _, fail).
'$xdg_directory'(common_data, Dir) :-
current_prolog_flag(msys2, true),
catch(win_folder(common_appdata, Dir), _, fail).
'$xdg_directory'(common_config, Dir) :-
current_prolog_flag(msys2, true),
catch(win_folder(common_appdata, Dir), _, fail).
It’s not pretty, but it works:
bash session:
matth@hp UCRT64 ~/swipl-devel/build
$ ~/swipl/bin/swipl --dump-runtime-variables
...
SWIPL_PACK_PATH="c:\msys64\home\matth\.local\share\swi-prolog\pack;c:\users\matth\appdata\local\swi-prolog\pack;c:\msys64\ucrt64\share\swi-prolog\pack;c:\msys64\usr\share\swi-prolog\pack;c:\programdata\swi-prolog\pack";
...
cmd.exe:
C:\msys64\home\matth\swipl\bin>swipl --dump-runtime-variables
...
SWIPL_PACK_PATH="c:\users\matth\appdata\local\swi-prolog\pack;c:\programdata\swi-prolog\pack";
...
C:\msys64\home\matth\swipl\bin>
I pushed 388f7c8be6dd14408c0c30e7e77313f4fb7e84ea. This adds a Prolog flag xdg to
enable/disable using the XDG Free Desktop standard for config and data directories. See docs.
Still a bit messy. I don’t like lots of conditionals on msys/conda/… in the code though. That should be abstracted as much as possible.
I see your point, and I wouldn’t mind if the whole MSYS2 thing is taken out and published separately as a set of patches. Maybe not right now, but at some point in the future, maybe in the 9.4 series.
It also looks more impressive to have a few files in the MSYS2 sources, not just the PKGBUILD with the cmake command line 
Works fine for SWIPL_PACK_PATH, but there’s an inconsistency with file_search_path(pack, X):
matth@hp UCRT64 ~/swipl-devel/build
$ ~/swipl/bin/swipl --dump-runtime-variables
PLBASE="c:/msys64/home/matth/swipl/lib/swipl";
SWIPL_PACK_PATH="c:\msys64\home\matth\.local\share\swi-prolog\pack;c:\users\matth\appdata\local\swi-prolog\pack;c:\msys64\ucrt64\share\swi-prolog\pack;c:\msys64\usr\share\swi-prolog\pack;c:\programdata\swi-prolog\pack";
...
matth@hp UCRT64 ~/swipl-devel/build
$ ~/swipl/bin/swipl
Welcome to SWI-Prolog (threaded, 64 bits, version 9.3.0-13-g84784c878-DIRTY)
1 ?- file_search_path(pack, X).
X = 'c:/users/matth/appdata/local/swi-prolog/pack' ;
X = app_data(pack).
And, in fact, pack_install(environ). still installs into the xdg folders. Backtracking of pack_install/2 indicates that the “return value” of file_search_path/2 is used.
Edit: There’s also a prolog_config/2 that lists the paths, in a different order, though.
prolog_config(pack_path, V).
V = 'c:\\users\\matth\\appdata\\local\\swi-prolog\\pack;c:\\msys64\\home\\matth\\.local\\share\\swi-prolog\\pack;c:\\users\\matth\\appdata\\local\\swi-prolog\\pack;c:\\msys64\\ucrt64\\share\\swi-prolog\\pack;c:\\msys64\\usr\\share\\swi-prolog\\pack;c:\\programdata\\swi-prolog\\pack' .
Surely, this is a bit of a nuisance. It does allow people to build the latest version in almost any environment though. The Windows version is a particular mess because we have both Windows native way of doing things and the various environments that try to make Windows look a bit more like Unix such as MSYS2, Cygwin and Conda (may be more …). Something similar holds for MacOS where we have native MacOS bundles and package managers that do it the classical Unix way like Macports and Homebrew. The configuration tries to abstract ways from these things and allow combining a set of features to match the desired layout. That is a bit of a moving target though 
prolog_config/2 is used by --dump-runtime-variables. I suspect the issue is with attach_packs/0 in boot/packs.pl, calling set_pack_search_path/0 in that file. The --dump-runtime-variables does not call attach_packs/0. Possibly it should call at least set_pack_search_path/0 for consistency.
I don’t really see what is going on. Guess you need a couple of writeln/1 calls and see what happens.
Found it. Finally. I am not 100% sure, though.
My theory says: It’s the coexisting, “normal”, Windows installation that happens to be on my system. Upon initialization, MSYS2 swipl correctly searches for packs in the msys2 folders; later, it also looks into the Windows folders. As it finds some packs there, the Windows folders are added to file_search_path(pack, X).* The problem disappeared after I removed c:\users\matth\appdata\local\swi-prolog.
*which, probably by the way absolute_file_name/3 works with pack(.), puts the windows folders in the beginning of the “pack” search path.
One way to solve it would be to change the behavior of prolog flag xdg:
\definition{Flag is not defined}
First search the Windows directories, then the XDG directories. This is the default for the Windows binaries.
\definition{Flag is \const{true}}
First search the XDG directories, then the Windows directories. This is the default on MSYS2 and Conda.
→ change to Search the XDG directories only.
\definition{Flag is \const{false}}
Search the Windows directories only.
I should add that I may not see all the implications of this patch.
1 Like
Makes sense. Implemented. Co-existence of differently structured versions is always a bit problematic. That is not unique to SWI-Prolog
2 Likes
Getting there
I’ve submitted the respective PR, sorry for the delay (I just noticed now that I had forgotten it).
Now it gets a bit tricky:
$ swipl
Welcome to etc.
?- check_installation.
(no complaints)
?- pack_install(environ).
Using CMake exports from c:/msys64/ucrt64/lib/cmake/swipl
% CMake Error at C:/msys64/ucrt64/lib/cmake/swipl/SWIPLTargets.cmake:95 (message):
% The imported target "swipl::swipl" references the file
% "/ucrt64/bin/swipl.exe"
% but this file does not exist. Possible reasons include:
% * The file was deleted, renamed, or moved to another location.
% * An install or uninstall procedure did not complete successfully.
% * The installation package was faulty and contained
% "C:/msys64/ucrt64/lib/cmake/swipl/SWIPLTargets.cmake"
% but not all the files it references.
Edit: It’s an MSYS2 issue, not related to swipl.
1 Like
MSYS2, still. Is the behavior below expected (i.e., that janus links to two python dlls)?
matth@hp UCRT64 ~/swipl-devel/build
$ ldd src/janus.dll
ntdll.dll => /c/Windows/SYSTEM32/ntdll.dll (0x7ff877130000)
...
libpython3.dll => /ucrt64/bin/libpython3.dll (0x7ff86e270000)
...
libpython3.11.dll => /ucrt64/bin/libpython3.11.dll (0x7fffd75d0000)
...
Low priority, not urgent or important.
Yes. Well, if ldd on MSYS looks like ELF ldd as we find it on e.g. Linux. janus.dll links to libpython3.dll, a “version independent” intermediate DLL. That one links to the current (libpython3.11.dll) version. As ldd lists both direct and indirect dependencies, it shows both.
(Continuation of my problems with threads on Windows)
I think there actually is a problem (not just my broken MSYS). I can reproduce it with this tiny prolog script using the current SWI-Prolog 9.3.2 from the download site. This is the minimalistic version of the program (test.pl):
:- use_module(library(thread)).
at_exit_work :-
thread_exit(true).
test :-
thread_create(at_exit_work, Id, []),
thread_join(Id, exited(true)).
This is from CMD.EXE:
C:\msys64\home\matth\threads>"\Program Files\swipl\bin\swipl.exe"
Welcome to SWI-Prolog (threaded, 64 bits, version 9.3.2)
etc.
1 ?- [test].
true.
2 ?- trace.
true.
[trace] 2 ?- test.
Call: (12) test ? creep
^ Call: (13) thread_create(at_exit_work, _3962, []) ? creep
^ Exit: (13) thread_create(at_exit_work, <thread>(3,000001ec0b7e7960), []) ?
Call: (1) at_exit_work ? creep
Call: (13) thread_join(<thread>(3,000001ec0b7e7960), exited(true)) ? creep
Call: (2) thread_exit(true) ? creep
creep
C:\msys64\home\matth\threads>
Note that Prolog stops and we are back at the Windows prompt.
With gdb, I see
$ gdb /c/Program\ Files/swipl/bin/swipl.exe
GNU gdb (GDB) 14.1
...
(No debugging symbols found in C:/Program Files/swipl/bin/swipl.exe)
(gdb) run
Starting program: C:\Program Files\swipl\bin\swipl.exe
[New Thread 10120.0x2fec]
[New Thread 10120.0x2608]
[New Thread 10120.0x29d8]
Welcome to SWI-Prolog (threaded, 64 bits, version 9.3.2)
...
[New Thread 10120.0x2984]
1 ?- [test].
true.
2 ?- test.
[New Thread 10120.0x33c]
Thread 6 received signal SIGSEGV, Segmentation fault.
[Switching to Thread 10120.0x33c]
0x00007ffe240f13e2 in ntdll!RtlVirtualUnwind () from C:\windows\SYSTEM32\ntdll.dll
(gdb) bt
#0 0x00007ffe240f13e2 in ntdll!RtlVirtualUnwind () from C:\windows\SYSTEM32\ntdll.dll
#1 0x00007ffe240f0b5a in ntdll!RtlVirtualUnwind () from C:\windows\SYSTEM32\ntdll.dll
#2 0x00007ffe240efc88 in ntdll!RtlUnwindEx () from C:\windows\SYSTEM32\ntdll.dll
#3 0x00007ffe241241ed in ntdll!RtlUnwind () from C:\windows\SYSTEM32\ntdll.dll
#4 0x00007ffe2254804b in msvcrt!_setjmpex () from C:\windows\System32\msvcrt.dll
#5 0x00007ffdfa1f584e in pthread_exit () from C:\Program Files\swipl\bin\libwinpthread-1.dll
#6 0x00007ffd8ab29697 in libswipl!PL_unify_thread_id () from C:\Program Files\swipl\bin\libswipl.dll
#7 0x00007ffd8aacd994 in libswipl!PL_exception () from C:\Program Files\swipl\bin\libswipl.dll
#8 0x00007ffd8ab039fc in libswipl!PL_check_data () from C:\Program Files\swipl\bin\libswipl.dll
#9 0x00007ffd8ab28a58 in libswipl!PL_write_term () from C:\Program Files\swipl\bin\libswipl.dll
#10 0x00007ffdfa1f4b5b in pthread_create_wrapper () from C:\Program Files\swipl\bin\libwinpthread-1.dll
#11 0x00007ffe2250e634 in msvcrt!_beginthreadex () from C:\windows\System32\msvcrt.dll
#12 0x00007ffe2250e70c in msvcrt!_endthreadex () from C:\windows\System32\msvcrt.dll
#13 0x00007ffe2241257d in KERNEL32!BaseThreadInitThunk () from C:\windows\System32\kernel32.dll
#14 0x00007ffe2412aa58 in ntdll!RtlUserThreadStart () from C:\windows\SYSTEM32\ntdll.dll
#15 0x0000000000000000 in ?? ()
(gdb)
If I recompile the MSYS version with debugging turned on, the segmentation fault disappears, but reappears in other tests :-/
Hard to say. Does work fine in Linux as well as on the public Windows binary. If the stack is not misleading it looks like some mismatch in how Windows and MSYS look at the stack. This may indicate there is a problem with the gcc flags. When cross-compiling the releases there is not much special to the flags:
FLAGS = -O2 -gdwarf-2 -g3 -std=gnu11 -fvisibility=hidden -D__WINDOWS__ -D_WIN32_WINNT=0x0600 -DWIN64 -Wall -fno-strict-aliasing -fprofile-use -Wno-maybe-uninitialized -fprofile-dir=/home/swipl/src/swipl-devel/build.win64/PGO-data -include /home/swipl/src/swipl-devel/build.win64/src/pgo-timestamp.h
The various profiling options should not make a difference.
If it is hard to fix, just remove thread_exit/1. It is deprecated anyway as it can easily deadlock the system.
One thing, pl-thread.c, line 2355:
if ( c_stack )
{ stack = round_pages(c_stack); // stack is not used in the later course
func = "pthread_attr_setstacksize";
(unrelated to the problem)
Thanks. Fixed. Typo. Indeed unrelated.
In fact, this line crashes the public Windows binary (9.3.2 on my system):
thread_create(thread_exit(true), _Id).
For a second, a response is shown; afterwards, the window is closed. I tried to place pthread_exit(NULL) at several places in the code (by duplicating the respective functions and calling the duplicates for my specific program) to obtain an “alternative” backtrace.
- pl-thread.c: callProlog at line 2067. If I put a pthread_exit in any line < 2067, the thread exits normally, there’s just a little complaint about dead threads when I “halt” the system.
- pl-pro.c: PL_get_next_solution at line 475.
- pl-wam.c: If I put pthread_exit in the very beginning of the function body of PL_get_next_solution, it already segfaults.
Unsure if this helps. Unfortunately, as you might guess, PL_get_next_solution is above my head. Moreover, I don’t understand the LD macros.
That indeed reproduces on Windows 11. There is no message when the main thread exits, so it is hard to tell more. The example works fine when running under the Wine emulator . Just putting pthread_exit(NULL) at different places is not very likely to result in anything useful. As you see, it sometimes crashes, some of these crashes are probably to be expected.
Not sure what to do. I have very little interest in trying to debug this deprecated functionality. Looks like a pthread-win32 issue. Disabling thread_exit/1 on Windows is probably the simplest.
The LD macro returns the “Local Data”, a pointer to a big struct that contains all Prolog thread-specific data. The tricky macro deals with the fact this may be fixed (when compiled without threads), passed around as parameter between functions or is extracted from one of several OS/compiler specific way to get it.
It appears as if I can suppress the problem by the cmake switch -DVMI_FUNCTIONS=ON. You might consider this for the Windows build, as well (?).
Did you do some timing? AFAIK using -DVMI_FUNCTIONS implies a serious slowdown on gcc. Only on clang it works fine because clang is not good in optimization large functions and the slowdown of the function calls is compensated for by the better optimization of the instruction code. But, clang based SWI-Prolog on x86_64 (amd64) is about 30% worse than gcc. On the Apple M1 xCode’s clang is about equal to gcc.
Anyway, this should be completely unrelated from pthread_exit(). If that is the “solution” my vote would be to remove thread_exit/1.