Tried to build a static executable (x86_64) with -DSTATIC_EXTENSIONS (and a static libswipl.a) but got the following error:
[...]
/usr/sbin/ld: src/libswipl.a(pl-load.c.o):(.data+0x1a8): undefined reference to `install_tex'
/usr/sbin/ld: src/libswipl.a(pl-load.c.o):(.data+0x2b8): undefined reference to `install_bdb4pl'
collect2: error: ld returned 1 exit status
[5/280] Building CXX object packages/swipl-win/CMakeFiles/swipl-win.dir/Completion.cpp.o
ninja: build stopped: subcommand failed.
Some packages are not yet ready for this. There are two demands: the install function of a library must be called install_<libname> and the Prolog side must use use_foreign_library/1 as
:- use_foreign_library(foreign(libname)).
Some packages (still) name the install function simply install(), but this obviously cannot work as you get a name conflict.
There may be other problems 
Only working on the Emscripten version with this option. PRs welcome 
After fixing some install_<name> naming problems and replacing 'autoload(…)withuse_foreign_library(foreign(libname))` I am getting some errors:
[..]
ERROR: /home/u/tmp/swipl-devel/build.static/home/xpce/prolog/boot/pce_principal.pl:71:
ERROR: '$activate_static_extension'/1: foreign_extension `shlib' does not exist
any idea where to look? thanks!
After commenting out autoload(library(shlib),[load_foreign_library/1]) I am left with these errors:
5/167] Generating lib/shlib.tex
FAILED: man/lib/shlib.tex man/lib/summaries.d/shlib.tex /tmp/swipl-devel/build.static/man/lib/shlib.tex /tmp/swipl-devel/build.static/man/lib/summaries.d/shlib.tex
cd /tmp/swipl-devel/build.static/man && /tmp/swipl-devel/build.static/src/swipl -f none --no-packs -x /tmp/swipl-devel/build.static/man/pldoc2tex -- --source=/tmp/swipl-devel/man --out=lib/shlib.tex --subsection --summaries "library(shlib)"
ERROR: /tmp/swipl-devel/man/pldoc2tex.pl:48: pltotex:main source_sink `library(shlib)' does not exist
[14/167] Generating lib/readutil.tex
FAILED: man/lib/readutil.tex man/lib/summaries.d/readutil.tex /tmp/swipl-devel/build.static/man/lib/readutil.tex /tmp/swipl-devel/build.static/man/lib/summaries.d/readutil.tex
cd /tmp/swipl-devel/build.static/man && /tmp/swipl-devel/build.static/src/swipl -f none --no-packs -x /tmp/swipl-devel/build.static/man/pldoc2tex -- --source=/tmp/swipl-devel/man --out=lib/readutil.tex --summaries "library(readutil)"
Warning: /tmp/swipl-devel/build.static/home/library/readutil.pl:89:
Warning: /tmp/swipl-devel/build.static/home/library/prolog_xref.pl:86:
Warning: library(shlib): No such file
ERROR: /tmp/swipl-devel/build.static/home/library/readutil.pl:89:
ERROR: prolog_xref:process_foreign/2: Unknown procedure: prolog_xref:current_foreign_library/2
Warning: Halting with status 1 due to 1 errors and 1 warnings
ninja: build stopped: subcommand failed.
obviously library(shlib) won’t be used for static binary, so we need some kind of conditional compile time flag that we can use like this:
:- if (...dynamic_binary...).
:- use_foreign_library(foreign(shlib)).
:- endif
we can use this flag for prolog_xref when it calls `current_foreign_library/2? or better, current_foreign_library/2 should work with static extensions linked in the executable?
Good question. Some of this can probably be resolved by avoiding the use of old stuff from library(shlib). I’m also in doubt whether using this should imply complete static linking on all platforms ort it should still allow for dynamic linking and only statically link the extensions? That would give two flags, one to decide whether or not to include the extensions statically and one to decide whether or not to support dynamic linking (if the platform allows for it).
I’ll push a few commits to ltx2htm and bdb packages to fix some of this.
I think the most useful use case is complete static linking (even allowing the user to statically link in some packs that use foreign modules) for unix like systems. This, by the way, may make it easier to embed a static swi-prolog in an android apk, making it much more easy to move forward with android apps.
Windows has no need of static linking since it is very good at binary compatibility already, so no need for static linking on windows.
Good points. For now I’ve pushed some changes that make the system compile out of the box on MacOS (I’m traveling) using -DSTATIC_EXTENSIONS. JPL is disabled (needs dynamic linkng). All tests pass and fthe result is a single executable that only depends on the system shared libs
Thanks! It compiles on X86 too, xpce is not working, but the rest seems ok.
Of course, It is not a statically linked binary yet, but great progress. Here is what I get:
$ ldd src/swipl
linux-vdso.so.1 => linux-vdso.so.1 (0x00007f9eeef63000)
libtcmalloc_minimal.so.4 => /usr/lib/libtcmalloc_minimal.so.4 (0x00007f9eee626000)
libncursesw.so.6 => /usr/lib/libncursesw.so.6 (0x00007f9eeee9f000)
libformw.so.6 => /usr/lib/libformw.so.6 (0x00007f9eeee8b000)
libgmp.so.10 => /usr/lib/libgmp.so.10 (0x00007f9eee583000)
libz.so.1 => /usr/lib/libz.so.1 (0x00007f9eeee71000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007f9eee49b000)
libcrypt.so.2 => /usr/lib/libcrypt.so.2 (0x00007f9eeee3d000)
libossp-uuid.so.16 => /usr/lib/libossp-uuid.so.16 (0x00007f9eeee2c000)
libnsl.so.3 => /usr/lib/libnsl.so.3 (0x00007f9eeee23000)
libarchive.so.13 => /usr/lib/libarchive.so.13 (0x00007f9eee3d4000)
libodbc.so.2 => /usr/lib/libodbc.so.2 (0x00007f9eee362000)
libdb-5.3.so => /usr/lib/libdb-5.3.so (0x00007f9eee1a3000)
libpcre2-8.so.0 => /usr/lib/libpcre2-8.so.0 (0x00007f9eee106000)
libyaml-0.so.2 => /usr/lib/libyaml-0.so.2 (0x00007f9eee0e4000)
libssl.so.1.1 => /usr/lib/libssl.so.1.1 (0x00007f9eee04f000)
libcrypto.so.1.1 => /usr/lib/libcrypto.so.1.1 (0x00007f9eedc00000)
libXinerama.so.1 => /usr/lib/libXinerama.so.1 (0x00007f9eeee1c000)
libXpm.so.4 => /usr/lib/libXpm.so.4 (0x00007f9eeee06000)
libXext.so.6 => /usr/lib/libXext.so.6 (0x00007f9eee03a000)
libXt.so.6 => /usr/lib/libXt.so.6 (0x00007f9eedfcf000)
libX11.so.6 => /usr/lib/libX11.so.6 (0x00007f9eedabd000)
libSM.so.6 => /usr/lib/libSM.so.6 (0x00007f9eedfc4000)
libICE.so.6 => /usr/lib/libICE.so.6 (0x00007f9eedfa7000)
libXft.so.2 => /usr/lib/libXft.so.2 (0x00007f9eedf8f000)
libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0x00007f9eed9ee000)
libfontconfig.so.1 => /usr/lib/libfontconfig.so.1 (0x00007f9eedf3f000)
libjpeg.so.8 => /usr/lib/libjpeg.so.8 (0x00007f9eed96b000)
libedit.so.0 => /usr/lib/libedit.so.0 (0x00007f9eedf04000)
libreadline.so.8 => /usr/lib/libreadline.so.8 (0x00007f9eed915000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007f9eed72e000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f9eed400000)
libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f9eedee2000)
/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f9eeef65000)
libtirpc.so.3 => /usr/lib/libtirpc.so.3 (0x00007f9eed701000)
libacl.so.1 => /usr/lib/libacl.so.1 (0x00007f9eed6f7000)
libexpat.so.1 => /usr/lib/libexpat.so.1 (0x00007f9eed6c6000)
liblzma.so.5 => /usr/lib/liblzma.so.5 (0x00007f9eed69c000)
libzstd.so.1 => /usr/lib/libzstd.so.1 (0x00007f9eed357000)
liblz4.so.1 => /usr/lib/liblz4.so.1 (0x00007f9eed679000)
libbz2.so.1.0 => /usr/lib/libbz2.so.1.0 (0x00007f9eed666000)
libltdl.so.7 => /usr/lib/libltdl.so.7 (0x00007f9eed65b000)
libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f9eed654000)
libxcb.so.1 => /usr/lib/libxcb.so.1 (0x00007f9eed32c000)
libuuid.so.1 => /usr/lib/libuuid.so.1 (0x00007f9eed64b000)
libXrender.so.1 => /usr/lib/libXrender.so.1 (0x00007f9eed63e000)
libpng16.so.16 => /usr/lib/libpng16.so.16 (0x00007f9eed2f5000)
libharfbuzz.so.0 => /usr/lib/libharfbuzz.so.0 (0x00007f9eed1f9000)
libbrotlidec.so.1 => /usr/lib/libbrotlidec.so.1 (0x00007f9eed1eb000)
libgssapi_krb5.so.2 => /usr/lib/libgssapi_krb5.so.2 (0x00007f9eed196000)
libXau.so.6 => /usr/lib/libXau.so.6 (0x00007f9eed637000)
libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0x00007f9eed18e000)
libgraphite2.so.3 => /usr/lib/libgraphite2.so.3 (0x00007f9eed16a000)
libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0x00007f9eed02d000)
libbrotlicommon.so.1 => /usr/lib/libbrotlicommon.so.1 (0x00007f9eed00a000)
libkrb5.so.3 => /usr/lib/libkrb5.so.3 (0x00007f9eecf31000)
libk5crypto.so.3 => /usr/lib/libk5crypto.so.3 (0x00007f9eecf01000)
libcom_err.so.2 => /usr/lib/libcom_err.so.2 (0x00007f9eecefb000)
libkrb5support.so.0 => /usr/lib/libkrb5support.so.0 (0x00007f9eeceea000)
libkeyutils.so.1 => /usr/lib/libkeyutils.so.1 (0x00007f9eecee3000)
libresolv.so.2 => /usr/lib/libresolv.so.2 (0x00007f9eeced1000)
libpcre.so.1 => /usr/lib/libpcre.so.1 (0x00007f9eece5a000)
and here is the output of check_installation:
1 ?- check_installation
| .
% Checking your SWI-Prolog kit for common issues ...
%
% Version: ............. 8.5.15-96-gc5e130043-DIRTY
% Address bits: ........ 64
% Architecture: ........ x86_64-linux
% Installed at: ........ /tmp/swipl-devel/build.static/home
% Cores: ............... 16
%
% Checking tcmalloc ............................ ok
% Checking gmp ................................. ok
% Loading library(archive) ..................... ok
% Supported filters: bzip2, compress, gzip, grzip, lrzip, lzip, lzma, lzop, none, rpm, uu, xz
% Supported formats: 7zip, ar, cab, cpio, empty, gnutar, iso9660, lha, mtree, rar, raw, tar, xar, zip
% Loading library(cgi) ......................... ok
% Loading library(crypt) ....................... ok
% Loading library(bdb) ......................... ok
% Loading library(double_metaphone) ............ ok
% Loading library(filesex) ..................... ok
% Loading library(http/http_stream) ............ ok
% Loading library(http/json) ................... ok
% Loading library(http/jquery) ................. ok
% jQuery from /tmp/swipl-devel/build.static/home/library/http/web/js/jquery-3.6.0.min.js
% Loading library(isub) ........................ ok
Warning: library(jpl) .......................... NOT FOUND
Warning: See http://www.swi-prolog.org/build/issues/jpl.html
% Loading library(memfile) ..................... ok
% Loading library(odbc) ........................ ok
% Loading library(pce) ......................... FAILEDUnknown procedure: send/2
Warning: Unknown procedure: get/3
Warning: However, there are definitions for:
Warning: det/1
Warning: get/1
Warning: get0/1
Warning: get/2
Warning: get0/2
Warning:
Warning: Unknown procedure: pce_error/1
Warning: Unknown procedure: pce_info/1
Warning: Unknown procedure: pce_warn/1
Warning: Unknown procedure: pce_error/1
Warning: Unknown procedure: pce_error/1
Warning: Unknown procedure: get/3
Warning: However, there are definitions for:
Warning: det/1
Warning: get/1
Warning: get0/1
Warning: get/2
Warning: get0/2
Warning:
Warning:
Warning: See http://www.swi-prolog.org/build/issues/xpce.html
% Loading library(pcre) ........................ ok
% Loading library(pdt_console) ................. ok
% Loading library(porter_stem) ................. ok
% Loading library(process) ..................... ok
% Loading library(protobufs) ................... ok
% Loading library(editline) .................... ok
% Loading library(readline) .................... ok
% Loading library(readutil) .................... ok
% Loading library(rlimit) ...................... ok
% Loading library(semweb/rdf_db) ............... ok
% Loading library(semweb/rdf_ntriples) ......... ok
% Loading library(semweb/turtle) ............... ok
% Loading library(sgml) ........................ ok
% Loading library(sha) ......................... ok
% Loading library(snowball) .................... ok
% Loading library(socket) ...................... ok
% Loading library(ssl) ......................... ok
% Loading library(crypto) ...................... ok
% Loading library(syslog) ...................... ok
% Loading library(table) ....................... ok
% Loading library(time) ........................ ok
Warning: library(tipc/tipc) .................... NOT FOUND
Warning: See http://www.swi-prolog.org/build/issues/tipc/tipc.html
% Loading library(unicode) ..................... ok
% Loading library(uri) ......................... ok
% Loading library(uuid) ........................ ok
% Loading library(zlib) ........................ ok
% Loading library(yaml) ........................ ok
[...]
xpce works for me. Whether or not to link the executable dynamically or statically against the system libraries is another decision. Fully static linking should not be hard. Not sure how useful this is, but it is good to know that dynamic linking is no longer really needed and for WASM this seems the preferred route.
Attaching user packs is another matter. For the whole thing to work the extension needs to use the cmake infrastructure that is used for the bundled extensions. That requires following much more strict conventions.
I did a ninja clean && ninja, now xpce works fine.
The only strange thing I noticed is that libpcre is linked twice, both the new libpcre2 and the old libpcre are linked. Wonder why?
$ ldd src/swipl | grep pcre
libpcre2-8.so.0 => /usr/lib/libpcre2-8.so.0 (0x00007f4d4c89c000)
libpcre.so.1 => /usr/lib/libpcre.so.1 (0x00007f4d4b5d3000)
There are two use cases where I see static linking as useful:
Android. Full static linking would be specially useful to run swi-prolog in Android. You could compile swi-prolog in some ARM compatible board and then include the static binary in your java Android app to run it on a phone. They could communicate via sockets, with the Java android app providing the gui (maybe just a webview and swiprolog acting as a web server) and the prolog code doing all the rest.
The static linking would shield swi-prolog from most of the non-standard changes they have made in Android, because the binary would be simply making direct system calls to the linux kernel without dependency on anything else. As long as they have not blocked/changed kernel system calls the binary should work. I have done this with a statically linked Ada executable and it works well.
Stable deployment of prolog executables. A fully statically linked binary would be very useful for deploying single binary prolog applications, without worrying about breaking things because we installed a new version of SWI prolog. I have single file prolog saved states that I run and some times I have to recompile them because of a WAM/VM change in libswipl.so or some such change. This would not be needed at all if I just used the static binary, and I could install the cutting-edge version of SWI-Prolog while all my saved states and prolog applications would continue to work without trouble.
What is needed to convert a user made pack into an extension? Probably use cmake, but what else would be needed?
No clue. Not on my Mac … Also the CMakeCache.txt only shows 2-8,
Not much. The dependencies and api to the Prolog system are the same. If you dump the Prolog and C/C++ files in a subdir of packages and add a cmake config it should work. Actually, you could simply maintain the layout of the pack and add a CMakeLists.txt at the top. The only issue is that pack_install/1 also triggers on a cmake config, but in a quite different way. That could be changed using some naming convention, I guess.
think I found out where it comes from, I have this in CMakeCache.txt:
PKG_FONTCONFIG_STATIC_LIBRARIES:INTERNAL=fontconfig;freetype;bz2;z;png16;m;m;z;harfbuzz;m;graphite2;glib-2.0;m;pcre;sysprof-capture-4;brotlidec;brotlicommon;expat;m
So it must come from fontconfig.
good to know, thanks.