Problems compiling Rocksdb

I’ve rebuilt rocksdb with g++ version 11, and no warning messages. These are the flags that were used (they’re generated by rocksdb/Makefile):
-W -Wextra -Wall -Wsign-compare -Wshadow -Wunused-parameter -Werror -Woverloaded-virtual -Wnon-virtual-dtor -Wno-missing-field-initializers -Wno-invalid-offsetof

I also rebuilt rocksdb4pl.cpp, resulting in a two warnings (I’ll push a fix for one of them; the other is safe, but will take a bit more work). I turned on some more flags and got warnings about deprecated copies, which I’ll look into because they might indicate a flaw in my changes to SWI-cpp.h.

I don’t have g++ version 12; I suppose I could build it from source, but I’d rather wait until it’s available in a PPA.

1 Like

Yes, this works for me too. I also had to add the -Wno-error to compile - so guess this extra flag comes from the difference between g++ 11 and 12.

I had to point the include flags to the last version of swi itself and get the SWI-cpp.h from the latest package. After this it works from latest stable as well.

Thanks for following up!

Please send me the warning messages from your compilation (by private message or by putting them in a snippet) – chances are, they indicate some other subtle problem that I should resolve.
I’ve pushed a couple of changes to my PRs for SWI-cpp.h and rocksdb4pl.cpp, which now give me a clean compile with g++ 11.

Loosely related, can someone point me to a simple example of prolog using rocksdb? Just, say, opening a database and putting something into a table or so.


Just RocksDB.

In the RocksDB pack there is

  1. a demo directory with example code.

  2. a test directory with test cases.

How to run tests for RocksDB pack (Click triangle to expand)
?- use_module(pack(rocksdb/test/test_rocksdb)).
true.

?- test_rocksdb.
% PL-Unit: rocks ......... done
% PL-Unit: terms . done
% PL-Unit: types .... done
% PL-Unit: merge .. done
% PL-Unit: builtin_merge ... done
% PL-Unit: properties . done
% PL-Unit: enum . done
% All 21 tests passed
true.

RocksDB pack is also used for rocks-predicates.

The code for rocks-predicates includes two examples

  1. a WordNet - example. You will need https://wordnetcode.princeton.edu/3.0/WNprolog-3.0.tar.gz
  2. a RDF - example

I also posted another example


There is also the SWI-Prolog pack bio_db but I could not get it to work.

The unit tests are pretty simple:

For my reference, this is what it took to install g++ 12 (based off linux - how to install gcc-12 on ubuntu - Stack Overflow ). Note that the gcc sources are over 1Gb and take a while to download (apparently throttled to 700 KiB/s or less)

sudo apt install libmpfrc++-dev libmpc-dev libgmp-dev gcc-multilib flex
git clone -j8 --recurse https://gcc.gnu.org/git/gcc.git
cd gcc
git checkout remotes/origin/releases/gcc-12
mkdir ../gcc-12-build
cd ../gcc-12-build
../gcc/configure --prefix=$HOME/.local --enable-languages=c,c++ --disable-multilib
make -j4  # number of CPUs/threads
make install
export LD_LIBRARY_PATH=$HOME/.local/lib64:$LD_LIBRARY_PATH
gcc --version

I’ve rebuilt everything with gcc 12.2.1, and there were no warnings, either from rocksdb4pl.cpp or from rebuilding rocksdb (with GitHub - facebook/rocksdb: A library that provides an embeddable, persistent key-value store for fast storage. as of Apr 7: d52b520d5, which seems to be the latest). Are you perhaps using some additional flags?
The flags I used (from the Makefile):

-Wall -Wextra -Wsign-compare -Wshadow -Wunused-parameter -Woverloaded-virtual -Wnon-virtual-dtor -Wno-missing-field-initializers -Wno-invalid-offsetof  -std=c++17 -O2 -fPIC -pthread

Hm, what I get is the one below. It disappears if I run without the cflag: -Wno-error

My complete CFLAGS is this: CFLAGS=‘-fPIC -pthread -I"/data/git/swipl-devel/src" -Wno-error’

Maybe you can try to run with Werror?

There might some default settings on gcc which is different here, maybe?

./include/rocksdb/file_system.h:757:8: error: ‘read_req.rocksdb::FSReadRequest::scratch’ may be used uninitialized [-Werror=maybe-uninitialized]
757 | struct FSReadRequest {
| ^~~~~~~~~~~~~
db/blob/blob_file_reader.cc: In member function ‘void rocksdb::BlobFileReader::MultiGetBlob(const rocksdb::ReadOptions&, rocksdb::autovectorrocksdb::BlobReadRequest*&, uint64_t*) const’:
db/blob/blob_file_reader.cc:417:19: note: ‘read_req’ declared here
417 | FSReadRequest read_req;
| ^~~~~~~~
cc1plus: all warnings being treated as errors
make: *** [Makefile:2442: db/blob/blob_file_reader.o] Error 1

The warning is from rocksdb, not from the Prolog wrapper function, and it appears to be harmless. (Strangely, I didn’t get this warning, so perhaps it’s due to the particular version of gcc you have? Or possibly because you don’t have -O2?)

Agree and agree, O2 does however not make any difference. Same issue when I try to use clang also, and no traces of general compile-settings while I do an env-command. Could be the defaults of the os it looks like.

To take this in a slightly different direction but towards what I perceive as a final goal.

Is not one of the desired goals when building with SWI-Prolog to just have one CMakeLists.txt file for each source code directory?

If so then should we not be collecting evidence and feedback for creating and enhancing such files?

rocksdb is a pack, so it uses a Makefile, not CMakeLists.txt. The form of the Makefile isn’t well documented; pack_install/2 uses some standard targets, such as install, check, etc. to handle any foreign function actions (e.g., for rocksdb, one action uses git submodule update to ensure that the rocksdb submodule is the appropriate version).

Have you seen this?

From looking at the source of prolog_pack.pl, I don’t see where the 2nd argument of is_foreign_pack/2 is used. As far as I can tell, these files are used to indicate that there are foreign files, but it’s not clear to me what library/build/tools.pl uses – according the documentation for build_steps/3, the appropriate toolchain is used, based on what files exist.

But I don’t think we’d want packs to require cmake. (Although it’s easier to set up nowadays than it used to be.) Also, in the case of rocksdb, building librocksdb.so, which we have no control over, uses make.

Not sure myself at present. I don’t know if you saw this which list two packs that have CMakeLists.txt. Since Jan W. created one it seems that the intent is for this to work even if it is not yet fully working in the code.

EDIT

The code is in directory library/build

library/build/tools.pl

This module implements the build system that is used by pack_install/1
and pack_rebuild/1. The build system is a plugin based system where each
plugin knows about a specific build toolchain. The plugins recognise
whether they are applicable based on the existence of files that are
unique to the toolchain.

library/build/cmake.pl

Manage a CMake project. This prefers the ninja generator if available
in $PATH.

library/build/make.pl

This build plugin deals with GNU style packages. It knows about the
following programs:

  • automake to create Makefile.in from Makefile.am
  • autoheader to create config.h.in from configure.in
  • autoconf to create configure from configure.in
  • configure to create Makefile
  • make for the make step

I don’t think it should be a requirement but it would be nice to have them. I really did not understand CMake until a few weeks ago when I took time to due the tutorial and then do it with a few other generators such as Visual Studio 17 2022, Ninja, NMake Makefiles, Unix Makefiles, MSYS Makefiles, MinGW Makefiles.

While it does make building more consistent, there is still much work that has to be done to get correct builds, still learning the ropes.