$ cd swipl-devel
$ mkdir build
$ cd build
$ cmake ..
$ make --trace
Version B (fails on some systems)
$ cd swipl-devel
$ mkdir build
$ cd build
$ R --vanilla
> system("cmake ..")
> system("make --trace")
fails in those places where the newly built swipl is invoked:
[ 63 percent] Generating ../home/boot.prc
cd /home/matthias/swipl-devel/build/src && /usr/bin/cmake -E remove -f /home/matthias/swipl-devel/build/home/boot.prc
cd /home/matthias/swipl-devel/build/src && ./swipl -q -O -o /home/matthias/swipl-devel/build/home/boot.prc -b /home/matthias/swipl-devel/build/home/boot/init.pl
print_message/2: recursive call: /home/matthias/swipl-devel/build/home/boot/init.pl:697 debug_mode(on)
[5] system:$set_predicate_attribute/3 <foreign>
[0] system:$c_call_prolog/0 [PC=0 in top query clause]
print_message/2: recursive call: /home/matthias/swipl-devel/build/home/boot/init.pl:697 trace_mode(on)
[5] system:$set_predicate_attribute/3 <foreign>
[0] system:$c_call_prolog/0 [PC=0 in top query clause]
print_message/2: recursive call: /home/matthias/swipl-devel/build/home/boot/init.pl:697 error(domain_error(predicate_property,transact),context(system: $set_predicate_attribute/3,_124))
[5] system:$set_predicate_attribute/3 <foreign>
[0] system:$c_call_prolog/0 [PC=0 in top query clause]
/home/matthias/swipl-devel/build/home/boot/init.pl:697: directive failed
For those who wonder why I should invoke the build commands from R: I see the same problem when compiling the R package rswipl. Any idea how I could debug this?
Edit: current workaround is cmake -DSWIPL_SHARED_LIB=OFF -DSWIPL_STATIC_LIB=ON -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DINSTALL_DOCUMENTATION=OFF ..
$ cd swipl-devel
$ mkdir build
$ cd build
$ cmake -DSWIPL_PACKAGES_X=OFF -DINSTALL_DOCUMENTATION=OFF ..
$ make
$ sudo make install
...
[100%] Built target library_index_library_ext_readline_
Install the project...
-- Install configuration: "RelWithDebInfo"
-- Set runtime path of "/usr/local/lib/swipl/bin/x86_64-linux/swipl" to "/usr/local/lib/swipl/lib/x86_64-linux"
-- Set runtime path of "/usr/local/lib/swipl/lib/x86_64-linux/libswipl.so.9.1.21" to "/usr/local/lib/swipl/lib/x86_64-linux"
-- Set runtime path of "/usr/local/lib/swipl/bin/x86_64-linux/swipl-ld" to "/usr/local/lib/swipl/lib/x86_64-linux"
$ /usr/local/bin/swipl
Welcome to SWI-Prolog (threaded, 64 bits, version 9.1.21-67-gcaddd3e33-DIRTY)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
Please run ?- license. for legal details.
For online help and background, visit https://www.swi-prolog.org
For built-in help, use ?- help(Topic). or ?- apropos(Word).
?- halt.
$ R --vanilla
R version 4.3.1 (2023-06-16) -- "Beagle Scouts"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> system("/usr/local/bin/swipl")
[FATAL ERROR: at Sat Jan 13 07:48:44 2024
/usr/local/lib/swipl/boot.prc: incompatible VM-signature (file: 0x408bfff4; Prolog: 0x9fa44fc5)]
Aborted (core dumped)
> quit()
Save workspace image? [y/n/c]: y
matthias@mcclass:~/swipl-devel/build$
So, your new Prolog gets its shared object from the publicly installed older version I suspect R sets LD_LIBRARY_PATH to include the system directory at /lib/x86_64-linux-gnu, overruling the compiled in path to the correct location.
If this is correct, this problem happens if there is a “standard” Debian apt version installed as the Debian maintainers do not like shared objects in application private directories, but want them in the public shared object directory.
This is hard to fix. Either we should ensure R doesn’t put the system library in LD_LIBRARY_PATH or we should ensure that the major version of libswipl.so changes with every release. Semantic versioning does not apply very well to SWI-Prolog, notably not to its shared object.
$ R
> system("LD_PRELOAD=/usr/local/lib/swipl/lib/x86_64-linux/libswipl.so.9 /usr/local/bin/swipl")
Welcome to SWI-Prolog (threaded, 64 bits, version 9.1.21-67-gcaddd3e33-DIRTY)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
Please run ?- license. for legal details.
For online help and background, visit https://www.swi-prolog.org
For built-in help, use ?- help(Topic). or ?- apropos(Word).
?- halt.
> quit()
$
That seems asking for lots of trouble. LD_PRELOAD is not intended for that. It is probably mostly intended to inject instrumentation into dynamically linked executables and can be a work around for systems using ldopen() and running into troubles due to the visibility rules.
The issue (if my analysis correct) is threefold: (1) R setting LD_LIBRARY_PATH to a weird path. One should typically try to avoid setting that and I don’t think it should include the system libraries. The variable is mostly used to make an application choose between shared objects in various non-standard places, such as selecting a specific JVM. (2) SWI-Prolog using two approaches: normally it has its libswipl.so in its own tree, where it is found using the Rpath setting of the executable, but Debian insists in it being installed in the global directory and (3) SWI-Prolog’s shared object version control, which is fine for the default local installation, but dangerous for using the public directory.
You can “fix” the build by deleting LD_LIBRARY_PATH from the R environment, or at least remove the system directory from the path. That is still likely to cause problems for running the result. I consider extending the test for possible conflicts in the environment, but that mostly makes diagnosing this easier. Changing the versioning of libswipl is worth considering. I don’t know the consequences though. We have quite a few platforms to consider …
I hope I am not too perseverative: Could the latter added to the cmake file (where boot.prc is being built, as well as library/INDEX.pl) without breaking things?
As is, I consider this an R issue. Simply removing LD_LIBRARY_PATH is in general also not the right solution. When building the Java interface it must often be set to point at the Java directory holding the shared objects. There are probably other setups where LD_LIBRARY_PATH is needed to allow finding libraries. One could set the src directory as first, but this will (I think) break PGO building as that uses libswipl.so from another location for the instrumented run.
The only thing we can do from the Prolog side is to make sure every libswipl.so has a new major version. That would mean that we have to encode the full version into the major version, e.g. use 090121 or something similar. This would (correctly) express that libswipl is tight to a specific version.
Another option is to stop using triples that suggest semantic versioning for Prolog completely. As is, the semantic versioning claim only holds for patch levels of the stable versions: these are guaranteed to be fully compatible. Patch levels of the development releases may introduce arbitrary incompatibilities (of course, we must try to avoid that). That would require a new numbering for stable vs development though. Any suggestions?
Yet another step could be to stop creating libswipl.so.<version>, but only libswipl.so and get the package maintainers to accept that the shared object is in the Prolog tree and uses the Rpath setting of the executable to be found.
N.b. Java uses the solution above (putting libjvm.so inside the distribution tree). Python calls the shared object e.g., libpython3.10.so.1.0. It is not mmediately clear what Node.js does.
For SWI-Prolog that would be libswipl9.1.21.so.1.0. That has advantages and disadvantages. Where for Python the C binding changes frequently and thus you want extensions to be linked against specific Python versions, SWI-Prolog is keen to keep the C binding binary compatible and extensions can thus safely link against libswipl and will work with a large range of Prolog versions. The impact depends on the linker semantics. On ELF systems (Linux), extensions do not link to libswipl, so there is no issue. On XCOFF (Apple) they do, but plugins on this platform are hard to use with other versions anyway. On Windows, plugins links against libswipl.dll and this is found in the same directory as swipl.exe, so you do not want a versioned DLL.
An option might be to introduce Python-like versioning iff the shared object is installed in the public library directory. That will also allow to use multiple SWI-Prolog versions to installed with this scenario. If SWI-Prolog is installed in a self-contained directory, simply call the library libswipl.
No action needed. Thank you for helping me understand the problem.
I agree. I won’t discuss this the R developers, though. They’ll tell me to build a static library, as it’s recommended in their package development guideline. Been there, done that (just to notice that you can’t load modules on top of a static library in systems other than linux).
I suspect R sets