Swipl embedded in R package

Help! Pack
1

Dear developers,

I am trying to connect swipl to R (i.e., invoking swipl from R). Most things are working fine, but there’s a problem with dependencies which is a bit complicated to describe. Also because I am a hobbyist. The problem occurs under linux.

I can consult a simple knowledge base and query it, for example:

LD_LIBRARY_PATH=/usr/local/lib/swipl/lib/x86_64-linux SWI_HOME=/usr/local/lib/swipl R
library(rolog)
[SWI welcome message]
rolog_consult(“likes”)
rolog_findall(quote(likes(sam))

works fine. This is just to show that I got the basics running. But whenever I want to use a command that relies on a swipl module (e.g., something requiring sgml2pl.so [I think it is html/3]), I get a “symbol not found” for PL_new_atom used in sgml2pl.so.

And, indeed,

matthias@DESKTOP-A2T8IFC:~/swipl-devel/build/packages/sgml$ ldd sgml2pl.so
linux-vdso.so.1 (0x00007ffe489d5000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fbc9a9e9000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbc9a7ff000)
/lib64/ld-linux-x86-64.so.2 (0x00007fbc9aa4d000)
matthias@DESKTOP-A2T8IFC:~/swipl-devel/build/packages/sgml$

shows that the object is not linked to libswipl.so. Which makes sense, because it is a “plugin”, so the missing symbols are provided by swipl itself. Unfortunately, I don’t know why the resolution of the missing symbols does not work from the R package.

I found two ways to fix this:

  1. Invoke R using the command

LD_PRELOAD=/usr/local/lib/swipl/lib/x86_64-linux/libswipl.so R

But this seems a bit cruel.

  1. Less cruel (but who knows):

cd /swipl-devel/build/packages/sgml

VERBOSE=1 make
and then notice that -lswipl is rather early in the linker command:

/usr/bin/cc -fPIC -O2 -gdwarf-2 -g3 -L/usr/local/lib/swipl/lib/x86_64-linux -lswipl -shared -o sgml2pl.so CMakeFiles/plugin_sgml2pl.dir/parser.c.o CMakeFiles/plugin_sgml2pl.dir/util.c.o CMakeFiles/plugin_sgml2pl.dir/charmap.c.o CMakeFiles/plugin_sgml2pl.dir/catalog.c.o CMakeFiles/plugin_sgml2pl.dir/model.c.o CMakeFiles/plugin_sgml2pl.dir/xmlns.c.o CMakeFiles/plugin_sgml2pl.dir/utf8.c.o CMakeFiles/plugin_sgml2pl.dir/xml_unicode.c.o CMakeFiles/plugin_sgml2pl.dir/quote.c.o CMakeFiles/plugin_sgml2pl.dir/sgml2pl.c.o CMakeFiles/plugin_sgml2pl.dir/xsd.c.o CMakeFiles/plugin_sgml2pl.dir/error.c.o -lpthread

So I moved it to the end:

/usr/bin/cc -fPIC -O2 -gdwarf-2 -g3 -L/usr/local/lib/swipl/lib/x86_64-linux -shared -o sgml2pl.so CMakeFiles/plugin_sgml2pl.dir/parser.c.o CMakeFiles/plugin_sgml2pl.dir/util.c.o CMakeFiles/plugin_sgml2pl.dir/charmap.c.o CMakeFiles/plugin_sgml2pl.dir/catalog.c.o CMakeFiles/plugin_sgml2pl.dir/model.c.o CMakeFiles/plugin_sgml2pl.dir/xmlns.c.o CMakeFiles/plugin_sgml2pl.dir/utf8.c.o CMakeFiles/plugin_sgml2pl.dir/xml_unicode.c.o CMakeFiles/plugin_sgml2pl.dir/quote.c.o CMakeFiles/plugin_sgml2pl.dir/sgml2pl.c.o CMakeFiles/plugin_sgml2pl.dir/xsd.c.o CMakeFiles/plugin_sgml2pl.dir/error.c.o -lswipl -lpthread

And then, indeed,

matthias@DESKTOP-A2T8IFC:~/swipl-devel/build/packages/sgml$ ldd sgml2pl.so
linux-vdso.so.1 (0x00007ffe8ef08000)
libswipl.so.8 => /usr/local/lib/libswipl.so.8 (0x00007f6aaf7fe000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6aaf7dc000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6aaf5f2000)
libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6 (0x00007f6aaf5c2000)
libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f6aaf53f000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f6aaf522000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f6aaf51a000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f6aaf3cb000)
/lib64/ld-linux-x86-64.so.2 (0x00007f6aaf9f1000)
matthias@DESKTOP-A2T8IFC:~/swipl-devel/build/packages/sgml$

And the “autoloading” of sgml2pl.so works fine. I am wondering if there’s an even less cruel way to do this.

Sorry for being not more specific, but that’s what I was able to find out. The library is found at github, mgondan/rolog, but I am unsure if this is really helpful at this early stage.

Best regards,

Matthias

2

Third option, this one is definitely less cruel: Added libswipl to SWIPL_LIBRARIES for Linux (see packages/cmake/PrologPackage.cmake, line 50). Note that this is actually in conflict with the ideas expressed in the comment in that file.

cmake -DSWIPL_LIBRARIES=libswipl ..
3

This is a rather dangerous route. It looks like R loads libswipl.so using the RTLD_LOCAL flag of dlopen(). That keeps the Prolog symbols local to Prolog. If you now add libswipl.so as explicit dependency to sgml2pl.so I fear you are loading libswipl.so twice. If that is the case I’m not sure what happens to SWI-Prolog global variables. You might get two copies of them :frowning:

The clean solution is to use RTLD_GLOBAL. Most systems that can load shared objects for plugins allow for choosing, indeed typically use RTLD_LOCAL as default to avoid multiple plugins from interfering.

On ELF systems we do not want Prolog plugins to depend on libswipl.so as it is not needed and keeps plugins independent from the exact Prolog version and location.

4

I see your point regarding the global variables.

Just to help my understanding: Did you mean R loads its packages in Mode RTLD_LOCAL? And, since the package depends on libswipl.so, it also loads libswipl.so locally?

I’ll see if I can find out if that’s indeed the case and whether it is possible to fix it.

Thank you & best wishes,

Matthias

5

It’s working, wonderful!

Thank you for the hint.