[swi/prolog 8.2.1] : bugs on FreeBSD

didier31 · August 22, 2020, 2:23pm

I don’t report bug in bug database because it appears on FreeBSD and I don’t know if you plan to support FreeBSD.
Nevertheless, I report you 2 bugs (only whom one, corrected) when building and testing swi/prolog that, I guess, could be solved easily for an experienced user.

The first one occurs at build :
SWI-cpp.h:42

#ifndef `__APPLE__`
#include <malloc.h>
#endif

clang complains malloc.h is replaced by stdlib.h when __STDC__ is defined.

I suggests to correct like this :

#if not(defined(`__APPLE__`) || defined(`__FreeBSD__`))
#include <malloc.h>
#endif

A bug at test:

Test report is encouraging except a sole test, failing on SEGFAULT :

Test project /home/didier/swipl-8.2.1/build
      Start  1: swipl:basic
 1/64 Test  #1: swipl:basic ......................   Passed    3.21 sec
      Start  2: swipl:unprotected
 2/64 Test  #2: swipl:unprotected ................   Passed    4.13 sec
      Start  3: swipl:core
 3/64 Test  #3: swipl:core .......................   Passed    8.47 sec
      Start  4: swipl:attvar
 4/64 Test  #4: swipl:attvar .....................   Passed    0.42 sec
      Start  5: swipl:debug
 5/64 Test  #5: swipl:debug ......................   Passed    1.30 sec
      Start  6: swipl:tabling
 6/64 Test  #6: swipl:tabling ....................   Passed    4.20 sec
      Start  7: swipl:library
 7/64 Test  #7: swipl:library ....................   Passed    2.43 sec
      Start  8: swipl:charset
 8/64 Test  #8: swipl:charset ....................   Passed    0.25 sec
      Start  9: swipl:eclipse
 9/64 Test  #9: swipl:eclipse ....................   Passed    0.23 sec
      Start 10: swipl:clp
10/64 Test #10: swipl:clp ........................   Passed    0.36 sec
      Start 11: swipl:GC
11/64 Test #11: swipl:GC .........................   Passed    1.96 sec
      Start 12: swipl:save
12/64 Test #12: swipl:save .......................   Passed    1.92 sec
      Start 13: swipl:files
13/64 Test #13: swipl:files ......................   Passed    0.26 sec
      Start 14: swipl:xsb/basic_tests
14/64 Test #14: swipl:xsb/basic_tests ............   Passed    0.85 sec
      Start 15: swipl:xsb/ai_tests
15/64 Test #15: swipl:xsb/ai_tests ...............   Passed    0.63 sec
      Start 16: swipl:xsb/ptq
16/64 Test #16: swipl:xsb/ptq ....................   Passed    0.55 sec
      Start 17: swipl:xsb/neg_tests
17/64 Test #17: swipl:xsb/neg_tests ..............   Passed    0.57 sec
      Start 18: swipl:xsb/delay_tests
18/64 Test #18: swipl:xsb/delay_tests ............   Passed    0.91 sec
      Start 19: swipl:xsb/wfs_tests
19/64 Test #19: swipl:xsb/wfs_tests ..............   Passed    0.65 sec
      Start 20: swipl:xsb/table_tests
20/64 Test #20: swipl:xsb/table_tests ............   Passed    1.13 sec
      Start 21: swipl:xsb/incremental_tests
21/64 Test #21: swipl:xsb/incremental_tests ......   Passed    0.35 sec
      Start 22: swipl:xsb/nonmt_tests
22/64 Test #22: swipl:xsb/nonmt_tests ............   Passed    0.39 sec
      Start 23: swipl:xsb/sub_tests
23/64 Test #23: swipl:xsb/sub_tests ..............   Passed    0.50 sec
      Start 24: swipl:thread
24/64 Test #24: swipl:thread .....................   Passed    7.18 sec
      Start 25: chr:chr
25/64 Test #25: chr:chr ..........................   Passed    1.09 sec
      Start 26: clib:cgi
26/64 Test #26: clib:cgi .........................   Passed    0.20 sec
      Start 27: clib:crypt
27/64 Test #27: clib:crypt .......................   Passed    0.19 sec
      Start 28: clib:memfile
28/64 Test #28: clib:memfile .....................   Passed    0.22 sec
      Start 29: clib:process
29/64 Test #29: clib:process .....................   Passed    0.37 sec
      Start 30: clib:readutil
30/64 Test #30: clib:readutil ....................   Passed    0.21 sec
      Start 31: clib:socket
31/64 Test #31: clib:socket ......................   Passed    4.53 sec
      Start 32: clib:af_unix
32/64 Test #32: clib:af_unix .....................   Passed    0.35 sec
      Start 33: clib:stream
33/64 Test #33: clib:stream ......................   Passed    0.40 sec
      Start 34: clib:time
34/64 Test #34: clib:time ........................   Passed    1.33 sec
      Start 35: clib:uri
35/64 Test #35: clib:uri .........................   Passed    0.30 sec
      Start 36: http:cgi_stream
36/64 Test #36: http:cgi_stream ..................   Passed    0.47 sec
      Start 37: http:http
37/64 Test #37: http:http ........................   Passed    0.71 sec
      Start 38: http:json
38/64 Test #38: http:json ........................   Passed    0.46 sec
      Start 39: http:multipart
39/64 Test #39: http:multipart ...................   Passed    0.41 sec
      Start 40: http:proxy
40/64 Test #40: http:proxy .......................   Passed    0.46 sec
      Start 41: http:websocket
41/64 Test #41: http:websocket ...................   Passed    0.38 sec
      Start 42: nlp:nlp
42/64 Test #42: nlp:nlp ..........................   Passed    0.23 sec
      Start 43: pengines:pengines
43/64 Test #43: pengines:pengines ................***Exception: SegFault  0.92 sec
      Start 44: pengines:term_html
44/64 Test #44: pengines:term_html ...............   Passed    0.36 sec
      Start 45: protobufs:protobufs
45/64 Test #45: protobufs:protobufs ..............   Passed    0.21 sec
      Start 46: RDF:rdf
46/64 Test #46: RDF:rdf ..........................   Passed    0.41 sec
      Start 47: RDF:write
47/64 Test #47: RDF:write ........................   Passed    0.49 sec
      Start 48: semweb:con
48/64 Test #48: semweb:con .......................   Passed    0.28 sec
      Start 49: semweb:litmap
49/64 Test #49: semweb:litmap ....................   Passed    0.33 sec
      Start 50: semweb:load
50/64 Test #50: semweb:load ......................   Passed    0.41 sec
      Start 51: semweb:ntriples
51/64 Test #51: semweb:ntriples ..................   Passed    0.28 sec
      Start 52: semweb:rdf11
52/64 Test #52: semweb:rdf11 .....................   Passed    0.32 sec
      Start 53: semweb:rdf_db
53/64 Test #53: semweb:rdf_db ....................   Passed    0.37 sec
      Start 54: semweb:subprop
54/64 Test #54: semweb:subprop ...................   Passed    1.67 sec
      Start 55: semweb:turtle2
55/64 Test #55: semweb:turtle2 ...................   Passed    0.29 sec
      Start 56: semweb:turtle
56/64 Test #56: semweb:turtle ....................   Passed    2.07 sec
      Start 57: sgml:sgml
57/64 Test #57: sgml:sgml ........................   Passed    0.31 sec
      Start 58: sgml:sgml_write
58/64 Test #58: sgml:sgml_write ..................   Passed    0.20 sec
      Start 59: sgml:xsd
59/64 Test #59: sgml:xsd .........................   Passed    0.24 sec
      Start 60: sgml:c14n
60/64 Test #60: sgml:c14n ........................   Passed    0.31 sec
      Start 61: zlib:zlib
61/64 Test #61: zlib:zlib ........................   Passed    0.38 sec
      Start 62: pcre:pcre
62/64 Test #62: pcre:pcre ........................   Passed    0.22 sec
      Start 63: yaml:yaml
63/64 Test #63: yaml:yaml ........................   Passed    0.35 sec
      Start 64: ssl:ssl
64/64 Test #64: ssl:ssl ..........................   Passed    4.16 sec

98% tests passed, 1 tests failed out of 64

Total Test time (real) =  69.83 sec

The following tests FAILED:
         43 - pengines:pengines (SEGFAULT)

Best regards
Didier.

jan · August 22, 2020, 2:48pm

I tend to include patches for portability for any platform unless involved patches are required that would seriously impact the readability of the code.

Thanks. Added (to swipl-devel.git; the development branch).

This is not good of course. If you want to figure out what is going on, run this test verbosely using

 ctest -V -R pengines

That should show the command and output. Next re-run this under the C debugger, possibly after recompiling for debugging (see CMAKE.md).

didier31 · August 22, 2020, 5:42pm

ctest -V -R pengines

This is what I did and I ran gdb.

gdb /home/didier/swipl-8.2.1/build/src/swipl 
GNU gdb (GDB) 9.2 [GDB v9.2 for FreeBSD]
Copyright (C) 2020 Free Software Foundation, Inc.

Reading symbols from /home/didier/swipl-8.2.1/build/src/swipl...
(gdb) run "-p" "foreign=:/home/didier/swipl-8.2.1/build/packages/clib:/home/didier/swipl-8.2.1/build/packages/sgml:/home/didier/swipl-8.2.1/build/packages/http" "-f" "none" "--no-packs" "-s" "/home/didier/swipl-8.2.1/packages/pengines/test_pengines.pl" "-g" "test_pengines" "-t" "halt"
Starting program: /home/didier/swipl-8.2.1/build/src/swipl "-p" "foreign=:/home/didier/swipl-8.2.1/build/packages/clib:/home/didier/swipl-8.2.1/build/packages/sgml:/home/didier/swipl-8.2.1/build/packages/http" "-f" "none" "--no-packs" "-s" "/home/didier/swipl-8.2.1/packages/pengines/test_pengines.pl" "-g" "test_pengines" "-t" "halt"
[New LWP 100407 of process 20950]
% PL-Unit: local_pengines ld-elf.so.1: /home/didier/swipl-8.2.1/build/packages/clib/uuid.so: Undefined symbol "uuid_make"
[LWP 100407 of process 20950 exited]
[Inferior 1 (process 20950) exited with code 01]

(gdb) quit

I’ve checked uuid_make is defined in ossp/uuid.h and in .so library too.

Never encountered a similar error in my life after decades.

I must go further before I bore you again.

Thanks for your answer, Jan.

Good evening
And best regards
Didier.

PS: I’ll tell you.

jan · August 23, 2020, 7:41am

Indeed weird. Doesn’t seem related to the segv though. Seems more an environment/dynamic linking issue running under gdb. Success!

didier31 · August 23, 2020, 8:58am

Hi Jan,

Catched !

As we suspected, it is related to dlib linking inconsistency.

make  -f packages/clib/CMakeFiles/plugin_uuid.dir/build.make packages/clib/CMakeFiles/plugin_uuid.dir/build
[ 67%] Building C object packages/clib/CMakeFiles/plugin_uuid.dir/uuid.c.o
cd /home/didier/swipl-8.2.1/build/packages/clib && /usr/bin/cc -Dplugin_uuid_EXPORTS -I/usr/local/include/ossp -I/home/didier/swipl-8.2.1/build/packages/c
lib -I/home/didier/swipl-8.2.1/src/os -I/home/didier/swipl-8.2.1/src -g -fPIC   -Wall -D__SWI_PROLOG__ -o CMakeFiles/plugin_uuid.dir/uuid.c.o   -c /home/d
idier/swipl-8.2.1/packages/clib/uuid.c
[ 67%] Linking C shared module uuid.so
cd /home/didier/swipl-8.2.1/build/packages/clib && /usr/local/bin/cmake -E cmake_link_script CMakeFiles/plugin_uuid.dir/link.txt --verbose=1
/usr/bin/cc -fPIC -g  -shared  -o uuid.so CMakeFiles/plugin_uuid.dir/uuid.c.o  -Wl,-rpath,/usr/local/lib: /usr/local/lib/libuuid.so
[ 67%] Built target plugin_uuid

/usr/local/lib/libuuid.so is not the ossp’s one as cmake has selected. It’s /usr/local/lib/libossp-uuid.so, instead to link to.

Have a nice day, Jan
Cheers

Boris · August 23, 2020, 9:09am

I have suffered similar issues with uuid for SWI-Prolog on Archlinux (I don’t remember how I solved it. I think I might have installed from source and passed something to ./configure).

Also Perl is somehow confused by ossp-uuid on Archlinux. I don’t know enough about these things but I am starting to suspect that there is something peculiar with the configuration/building/installing for ossp-uuid because it causes similar trouble even across operating systems.

didier31 · August 23, 2020, 9:26am

The problem resides in swipl-8.2.1/packages/clib/cmake/FindLibUUID.cmake:57,63

string(REPLACE "/include" "/lib" i_libuuid_libdir ${LIBUUID_INCLUDE_DIR})
find_library(UUID_LIBRARY
57:   NAMES ossp-uuid uuid
		 HINTS ${i_libuuid_libdir}
			   ${PC_LIBUUID_LIBDIR}
			   ${PC_LIBUUID_LIBRARY_DIRS}
		 NO_CMAKE_SYSTEM_PATH)
if(NOT UUID_LIBRARY)
  find_library(UUID_LIBRARY
63:	    NAMES ossp-uuid uuid)
endif()

libossp-uuid and libuuid are distinct libraries with distinct interfaces. In particular, uuid_make doesn’t exist in libuuid.

jan · August 23, 2020, 9:46am

uuid libraries is the nicest mess around for portability . There are quite a few libraries floating around and they in part share the same name and they all have different APIs. I guess we must also check the API … Can you suggest a patch?

didier31 · August 23, 2020, 6:22pm

Can you suggest a patch?

with cmake --debug-finds, I’ve found out it takes advantage of uuid-config.

CMake Debug Log at packages/clib/cmake/FindLibUUID.cmake:7 (find_program):
find_program called with the following settings:

VAR: UUID_CONFIG
NAMES: "uuid-config"
Documentation: OSSP uuid config tool
Framework
  Only Search Frameworks: 0
  Search Frameworks Last: 0
  Search Frameworks First: 0
AppBundle
  Only Search AppBundle: 0
  Search AppBundle Last: 0
  Search AppBundle First: 0
CMAKE_FIND_USE_CMAKE_PATH: 1
CMAKE_FIND_USE_CMAKE_ENVIRONMENT_PATH: 1
CMAKE_FIND_USE_SYSTEM_ENVIRONMENT_PATH: 1
CMAKE_FIND_USE_CMAKE_SYSTEM_PATH: 1

find_program considered the following locations:

/sbin/uuid-config
/bin/uuid-config
/usr/sbin/uuid-config
/usr/bin/uuid-config
/usr/games/uuid-config
/usr/local/sbin/uuid-config

The item was found at

/usr/local/bin/uuid-config

When I run uuid-config, this is the result.

didier@NomadBSD ~/s/build> uuid-config --cflags 
-I/usr/local/include/ossp
didier@NomadBSD ~/s/build> uuid-config --libs
-luuid

It’s clearly inconsistent : output of the last uuid-conf invocation should be -lossp-uuid => cmake is fooled by uuid-config.

As a result, it seems to be a uuid-config bug.

jan · August 24, 2020, 7:18am

Thanks for sorting this out. Considering it is a /usr/local program confusing things I guess this is just a case of bad luck and you should set an appropriate variable to force it to find the right version.

Boris · August 24, 2020, 7:40am

@didier31 I am sharing this info in the hope that it is relevant and helps.

I now looked more carefully. Apparently on Archlinux there is a community-maintained installation script for ossp-uuid that I eventually ended up using to resolve my SWI-Prolog+ossp-uuid trouble.

Notably, it includes two patches. They have stuff in there looking like:

diff -up uuid-1.6.1/uuid-config.in.BAD uuid-1.6.1/uuid-config.in
--- uuid-1.6.1/uuid-config.in.BAD	2008-03-06 11:56:13.000000000 -0500
+++ uuid-1.6.1/uuid-config.in	2008-03-06 11:56:25.000000000 -0500
@@ -121,7 +121,7 @@ do
             output_extra="$output_extra $uuid_ldflags"
             ;;
         --libs)
-            output="$output -luuid"
+            output="$output -lossp-uuid"
             output_extra="$output_extra $uuid_libs"

(and so on)

The whole package with the patches is available here: https://aur.archlinux.org/uuid.git (this is a read-only git repo).

But then again this is the opposite to what @jan proposed as a solution (if I understood correctly).

jan · August 24, 2020, 7:46am

I’m not proposing anything I think it is about ok as it is and the remaining trouble is something for the various uuid libraries to sort out between them. I am happy to accept patches (most likely to the CMake files) that adds tighter tests such that we do not accidentally end up with some inconsistent configuration and/or is smarter to ensure the configuration is consistent (i.e., we really have the OSSP version of both the header and library).

Boris · August 24, 2020, 8:33am

I understand now.

As it was, the issue for me was that I was not able to fix the SWI-Prolog CMake file on my own, and the half-solutions that I used for ossp-uuid broke other things (for example Perl). The clean solution in my case turned out to be: use a patched ossp-uuid. This solution works with SWI-Prolog as it is right now.

PS: ossp-uuid itself is frozen (no one is maintaining it but it doesn’t seem to need it), the original authors no longer even host the code, there are like a dozen “mirrors” of that code floating around on github and other such places, and so on. I haven’t looked into how Ubuntu/Debian/CentOS deal with uuid because I haven’t needed it, but I would guess that if they provide ossp-uuid, they are patching it too.

didier31 · August 24, 2020, 9:33am

Thanks @jan
Thanks @Boris for your informations.

Have a nice day
Cheers
Didier.

didier31 · August 24, 2020, 10:08am

Epilogue

The faulty uuid-config is shell script with hard-coded values…
.
.
.
uuid_includedir="/usr/local/include/ossp"
.
.
.
–libs)
output="$output -luuid"
output_extra="$output_extra $uuid_libs"
;;

Workaround : patching uuid-config

Topic		Replies	Views
Ann: SWI-Prolog 8.1.18 Releases	1	444	December 15, 2019
Ann: SWI-Prolog 8.1.3 Releases	4	710	March 27, 2019
Ann: SWI-Prolog 8.3.1 Releases	0	361	June 8, 2020
Ann: SWI-Prolog 8.1.29 Releases	1	475	April 22, 2020
Ann: SWI-Prolog 9.1.1 Releases	0	330	December 6, 2022

[swi/prolog 8.2.1] : bugs on FreeBSD

Related topics