@nicos noticed that my RocksDB interface is broken with the current Mint/Ubuntu version of librocksdb-dev (5.17.2). The C++ interface compiles without warning, but linking results in
I don’t see what I’m doing wrong. If you have more experience with C++ debugging, please help. To reproduce, install the rocksdb library. On Debian based systems using
The mystery is solved. As is, the packaged rocksdb libraries are practically useless:
The shared lib is not compiled with rtti (runtime type information), which causes the subclassing of the rocksdb::MergeOperator to fail. See https://github.com/facebook/rocksdb/issues/3811
The shared lib is linked against tcmalloc, which crashes if SWI-Prolog is also linked against tcmalloc (default when avallable). Compiling SWI-Prolog without tcmalloc makes rocksdb load, but crash during the tests.
The static lib is not compiled with -fPIC and can thus not be used to create a shared object.
The only solution is probably to include rocksdb in the package and build it the way we need. The main disadvantage is that that makes the packages rather heavyweight to download and build.
I’ve done a successful pack_install/1 from Ubunto 20.04 and MacOSX. The main downside is that it takes very long to download and build.
Enjoy!
Its one of many key-value alternatives. These external stores (we also have the BDB interface) map a blob to a blob. The Prolog interface allows various native types to be used for the blob (string, int, float) and a Prolog term. Term identity is based on variant. That is, if you use f(X) as key, you can only retrieve this using f(X) and not using e.g., f(a).
The main advantage of RocksDB is that it scales quite well.
That 's great. Thanks for fixing this.
I just did a pack_install(rocksdb) on latest swipl-devel
which installed cleanly.
It was lengthy, but it automated what I was doing before,
so it is a bonus as far as I am concerned.
I haven’t tested with my code yet.
I tried to test with ?- pack('rocksdb/demo/merge')].
however it is not trivial what i should be seeing.
Please note that the first line of the file reads: :- use_module(rocksdb).
module() is missing.
It would be nice if someone provides a generic translation layer between k-v and tables.
db_facts, would one place for it. bio_db does some of that, but I only had time to
do what was necessary for the bio tables. As far as I recall there where some
differences between rocksdb and bdb interfaces.
Fixed (well, library() was missing). Code needs some comments …
Sure. This isn’t trivial though. All these external K-V stores have their own proprietary interface and notably their own extensions that make them valuable. For example (if I recall correctly), BDB can have multiple values per key. RocksDB has only a single value per key. RocksDB has this nifty value merging API that make it more suitable for dealing with complex key values. Still, certainly when using these databases for more than small beer, you carefully need to design your data to fit the strong points of the DB and you need to tune it to do the best job.
If you have a fairly simple db with few writes mapping a term variant to a single term they are indeed pretty much the same.
I wonder how one could offer a generic translation from a backend k-v store that is non-backtrackable to a prolog front-end that is relational and backtrackable.
Both have an enumeration functionality that does do the usual Prolog unification. As BDB can also have multiple values per key, all works nicely as a Prolog fact. Using RocksDB you would have to associate a list with each key. Only, the DB is not indexed this way You have to do something similar to Prolog itself to fix that: create additional indexes. With some persistence I don’t see why this should not be possible. Would be interesting to see how well it performs.
You have to do something similar to Prolog itself to fix that: create additional indexes. With some persistence I don’t see why this should not be possible. Would be interesting to see how well it performs.