Now call this from C(++). The first PL_next_solution() says TRUE, but the cleanup is not executed. Now close the query. That runs the cleanup handler and should raise error. If the goal in the setup_call_cleanup/3 completed (fail, exception, deterministic true), the cleanup handler has done its work before control gets back to Prolog and thus PL_next_solution() already generates the exception.
Yes. To do it all by the book though, we must either use no flag (exception will never reach C(++)) or use PL_Q_CATCH_EXCEPTION as there is nothing to pass to. If we do so however, we must deal with the exception before we close the query (after which we could throw some other exception).
If we use PL_Q_PASS_EXCEPTION here, the exception will be considered “not caught” as there is no enclosing PlQuery and (thus) the system will trap the debugger rather than passing the exception to C.
If that poses problems we can probably provide the same semantics as PL_Q_PASS_EXCEPTION but without trapping the debugger. Possibly we could supply both flags for that, meaning it is caught, but it should survive closing the query.
I think that will work. It’ll take me a bit of time to make all the code changes and test cases, but the overall API probably won’t change much, if at all. (I’ll also update swipl-win, rocksdb, and rolog)
As for PlEngine – I think we can deal with that separately, possibly by making some macros that are similar to what PREDICATE does. Or – because C++ now has lambdas – possibly we could do this without macros; I already have one use of lambda, in bool PlRewindOnFail(std::function<bool()> f). The current support for PlEngine is fairly minimal, anyway (as was PlQuery, so making a good API for PlEngine could require quite a bit of thinking).
ADDED:
Here’s what the code inside PREDICATE looks like (some parts removed or changed slightly for brevity/clarity):
term_t ex = PL_exception(0);
if (ex)
{ // PL_clear_exception(); /* See below */
throw PlException(ex);
}
In the latter situation, the frame’s exception has already been set, so the PL_raise_exception() in PREDICATE will act as a no-op. On the other hand, if the PlCheck() code does PL_clear_exception(), the “catch” in PREDICATE would just reinstate the same exception term. So, the net effect is the same, either way.
However, if the code surrounding the PlCheck() has its own try/catch and deals with the exception, it would also need to do PL_clear_exception() or else there will be a “did not clear exception” message (I presume that the message is caused by PL_raise_execption(...); return TRUE).
So, I think that PlCheck(...) should call PL_clear_exception() before throw PlException(ex).
Seems to make sense. My claim “using PL_clear_exception() is almost always wrong” probably only applies to languages that have no exception handling such as C
Still, the Prolog exceptions should in almost all cases be propagated to Prolog …
It turns out that C++ exceptions and the Prolog debugger don’t like each other. When a predicate that’s called via PlQuery (which uses PL_open_query() / PL_next_solution()) creates a Prolog error, the debugger gets involved before the C++ code sees it. It appears that the debugger resumes execution using a longjmp() - and this bypasses the catch in PREDICATE, which results in a fatal runtime error because the exception isn’t handled. (Or it gets a segmentation error – I’ve seen both behaviours.)
This problem can be avoided by running the query inside catch(Goal,E,true), which avoids the debugger getting involved (the debugger looks up the stack for a “catch” before the top level; if it sees a “catch”, it just continues execution without a longjmp)
Possibly a PL_FA_CPP flag in the PREDICATE could be used to prevent invoking the debugger? – that’s not a proper solution, but at least it would prevent crashes.
The conventional advice for C++ and longjmp is “don’t do that”:
The other issue that I’ve been dealing with is the lifetime of Prolog terms and the lifetime of C++ objects. This turns out to be quite tricky:
throwing a PlException exception invokes the PlException copy constructor (and the destructor on the original object). This would probably be easier in Rust.
if the PlQuery predicate creates a Prolog exception (via PlQuery::next_solution()), the error term’s lifetime is limited to the foreign frame in which the predicate runs, so the error term is invalid at the point where the C++ code sees it (which is, I suppose, why PL_open_query() has the PL_Q_PASS_EXCEPTION and PL_Q_CATCH_EXCEPTION flags).
I’ve solved these problems for now by having PlException use PL_record_external() to serialize the error term. This is more overhead than I’d like, but it’s a fairly simple solution; a lower overhead solution looks as if it would be quite tricky to get right. (I tried PL_record() but got various crashes, presumably because I was trying to use terms that had become invalid.)
Only if you set PL_Q_PASS_EXCEPTION. You are telling Prolog that you will pass on exceptions to the outer Prolog environment. If that doesn’t catch the exception it is considered uncaught and the debugger gets involved. What happens depends on the user interaction with the debugger. In any case, SWI-Prolog no longer uses setjmp()/longjmp() unless the old PL_throw_exception() is used (rather than PL_raise_exception()). It also uses setjmp()/longjmp() to deal with GMP allocation failures, but that should be local and never interleaved with C++.
So, if you want to catch the exception you must tell the system so using PL_Q_CATCH_EXCEPTION. That means you must deal with it before closing teh query though. As suggested earlier we could define a flag that will preserve the exception and claims it is caught.
That should be ok, no? That only affects the term_t reference, not the term itself.
PL_record_external() should never be needed for preserving terms in the same process. It is there to copy terms to other instances, i.e…, it notably serialized atoms rather then using the atom handle. If one works and the other doesn’t you probably have a memory corruption. I still think there is something fundamentally wrong if these functions are needed.
The new SWI-cpp2.h is almost ready – all that remains is some tidying up and documentation. packages/swipl-win and rocksdb are compiling and passing their tests (for swipl-win, my “test” is to run the executable and run a few simple Prolog queries in the window).
I’ve made a few name changes (e.g., PlCheck() is now PlCheckFail()); I’ve also created wrappers for all the PL_*() functions in SWI-Prolog.h, so that there’s no need to check for Prolog errors after a call; they’ll throw a C++ error if something goes wrong.
@mgondan – what is the status of rolog? It appears that you’ve been keeping the cpp2 source in parallel with your other changes - if so, I can make the necessary changes. (I don’t know how to test rolog, however).
Hmmm … things are now working. “I didn’t change anything”.
It’s possible that my lifetime problem will go away if I only call PL_exception() when PL_Q_CATCH_EXCEPTION is set (I also use that flag to decide whether to do PL_exception(qid) or PL_exception(0)).
Eventually, I may be able to get rid of the serialization; but for now, it removes one potential source of error. Getting the PlException object correct has been tricky - it’s similar to getting a smart pointer correct, and there’s no C++ “borrow checker”.
@jan For testing, I’m provoking an exception by calling a non-existent predicate. Is this sufficient, or should I also provoke some other errors, such as divide-by-zero or must_be/2?
PS: the top level of swipl-win does the wrong thing with catch(foo(bar),E,true) – it doesn’t call catch/3 but just outputs the error message – but it does the right thing if I define f(E):-catch(foo(bar),E,true) and then call f(E) at the top level.
I renamed rolog_cpp2.txt to rolog_cpp2.cpp and it compiled cleanly with my latest SWI-cpp2.h. But I seem to be missing some things for testing …
$ git status
On branch main
Your branch is up to date with 'origin/main'.
nothing to commit, working tree clean
$ git pull origin --rebase=no
Already up to date.
$ git remote -v
myfork git@github.com:kamahen/rolog.git (fetch)
myfork git@github.com:kamahen/rolog.git (push)
origin git@github.com:mgondan/rolog.git (fetch)
origin git@github.com:mgondan/rolog.git (push)
$ git log -n 1
commit 4ab15be6891a1e4b640fcff962448e65e1c08642 (HEAD -> main, origin/main, origin/HEAD)
Author: mgondan <Matthias.Gondan-Rochon@uibk.ac.at>
Date: Sun Jan 29 23:30:49 2023 +0100
Update interval.pl
$ (cd ..; R CMD check rolog)
* using log directory ‘/home/peter/src/rolog.Rcheck’
* using R version 4.0.4 (2021-02-15)
* using platform: x86_64-pc-linux-gnu (64-bit)
* using session charset: UTF-8
* checking for file ‘rolog/DESCRIPTION’ ... ERROR
Required field missing or empty:
‘Author’
* DONE
Status: 1 ERROR
See
‘/home/peter/src/rolog.Rcheck/00check.log’
for details.
(00check.log has the same contents as the error messages)
and got a different error (needs package rswipl, plus some suggested packages):
$ (cd .. && R CMD check rolog)
* using log directory ‘/home/peter/src/rolog.Rcheck’
* using R version 4.0.4 (2021-02-15)
* using platform: x86_64-pc-linux-gnu (64-bit)
* using session charset: UTF-8
* checking for file ‘rolog/DESCRIPTION’ ... OK
* checking extension type ... Package
* this is package ‘rolog’ version ‘0.9.12’
* package encoding: UTF-8
* checking package namespace information ... OK
* checking package dependencies ... ERROR
Package required but not available: ‘rswipl’
Packages suggested but not available:
'DiagrammeR', 'DiagrammeRsvg', 'rsvg', 'testthat'
The suggested packages are required for a complete check.
Checking can be attempted without them by setting the environment
variable _R_CHECK_FORCE_SUGGESTS_ to a false value.
See section ‘The DESCRIPTION file’ in the ‘Writing R Extensions’
manual.
* DONE
Status: 1 ERROR
See
‘/home/peter/src/rolog.Rcheck/00check.log’
for details.
I have moved all the swipl sources from rolog to rswipl. If you just want to “install” rolog, you can connect to swipl on your system. If you want to test it, you need the above-mentioned R packages, therefore,
Anyway, I’m still unable to test … it seems that Debian is a bit conservative in its use of packages and I need to learn some magic incantations for adding PPAs.
Also, there seem to be some additional package dependencies:
r-base-core : Depends: libicu66 (>= 66.1-1~) but it is not installable
Depends: libjpeg8 (>= 8c) but it is not installable
I have an Ubuntu machine with R 4.2.3 on it, but it also has some problems.
Maybe I’ll just have to leave this to @mgondan to test.
Here’s what I get on Debian (ChromeBook):
> warnings()
Warning messages:
1: package ‘rswipl’ is not available for this version of R
‘rswipl’ version 9.1.8 is in the repositories but depends on R (>= 4.2)
A version of this package for your version of R might be available elsewhere,
see the ideas at
https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Installing-packages
2: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘isoband’ had non-zero exit status
3: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘farver’ had non-zero exit status
4: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘rsvg’ had non-zero exit status
5: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘igraph’ had non-zero exit status
6: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘influenceR’ had non-zero exit status
7: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘scales’ had non-zero exit status
8: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘vroom’ had non-zero exit status
9: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘ggplot2’ had non-zero exit status
10: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘dplyr’ had non-zero exit status
11: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘readr’ had non-zero exit status
12: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘tidyr’ had non-zero exit status
13: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘viridis’ had non-zero exit status
14: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘DiagrammeR’ had non-zero exit status
15: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘testthat’ had non-zero exit status
$ R CMD check rolog
Warning: ‘rolog’ is neither a file nor directory, skipping
[peter@penguin rolog (exceptions)]$ cd ..
[peter@penguin src]$ R CMD check rolog
* using log directory ‘/home/peter/src/rolog.Rcheck’
* using R version 4.0.4 (2021-02-15)
* using platform: x86_64-pc-linux-gnu (64-bit)
* using session charset: UTF-8
* checking for file ‘rolog/DESCRIPTION’ ... OK
* checking extension type ... Package
* this is package ‘rolog’ version ‘0.9.12’
* package encoding: UTF-8
* checking package namespace information ... OK
* checking package dependencies ... ERROR
Package required but not available: ‘rswipl’
Packages suggested but not available: 'DiagrammeR', 'rsvg', 'testthat'
The suggested packages are required for a complete check.
Checking can be attempted without them by setting the environment
variable _R_CHECK_FORCE_SUGGESTS_ to a false value.
See section ‘The DESCRIPTION file’ in the ‘Writing R Extensions’
manual.
* DONE
And on my Ubuntu 20.04 machine:
Warning messages:
1: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘curl’ had non-zero exit status
2: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘rsvg’ had non-zero exit status
3: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘igraph’ had non-zero exit status
4: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘V8’ had non-zero exit status
5: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘influenceR’ had non-zero exit status
6: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘DiagrammeRsvg’ had non-zero exit status
7: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘vroom’ had non-zero exit status
8: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘readr’ had non-zero exit status
9: In install.packages(c("rswipl", "DiagrammeR", "DiagrammeRsvg", ... :
installation of package ‘DiagrammeR’ had non-zero exit status
It’s a bit strange that your Linux distributions don’t allow you to install a current R. Anyway, I wouldn’t mind a pull request with your much appreciated results (even an incomplete one).
There seem to be some extra steps needed with key signing for adding a PPA to Debian, and I haven’t spent the time figuring out how to do it. (This also affects the PPA for SWI-Prolog; but I just build it form source.)
As soon as I have the new SWI-cpp2.h ready for review, I’ll inform you. (There have been a lot of changes, although I think that most of them are backwards compatible.)
SWI-cpp2-plx.h contains wrapper functions (imported from SWI-cpp2.h)
throw PlException for Prolog errors
most SWI-Prolog.h functions have a wrapper.
verify() methods removed from PlAtom, PlTerm, etc.
the wrapper functions do the checking.
Some executable code has been moved to SWI-cpp2.cpp
can be inlined, if desired.
PlException is now a subclass of std::exception, not a subclass of PlTerm.
PlTypeError, PlDomainError, etc. are no longer subclasses of PlTerm, but are functions for creating suitable PlException objects.
The string comparison operators are deprecated; use as_string() and std::string comparison instead, which allows specifying the encoding.
Added PlRecord, PlRecordExternal, PlControl (used by PREDICATE_NONDET), PlStream.
Fixed numerous bugs and misfeatures; added tests.
I’ve also made PRs for swipl-win, rocksdb, and rolog.
I apologize for the large size of the PR, but I didn’t see an easy way to break it into smaller pieces.
The documentation is still a bit rough; I’ll try to review it and improve it “in the future”.
There are still a few things that need to be done, mostly to do with text encodings. However, their lack shouldn’t be a significant issue, and missing functionality should be easy to add.
I’m mostly finished with changes to SWI-cpp2.h. My changes have been merged and will probably be available in the next development release (9.1.9).
The main things left to do are:
Improve the interfaces for blobs and PL_options()
Add encoding information for methods that accept or return strings (char*, wchar_t*, std::string, std::string) - this is partially done but needs to be finished.
Improve the efficiency of PlException - currently it uses the PL_record() et al to ensure that the exception term doesn’t go out of scope but some changes to PlQuery might make this unnecessary. (Also, more test cases are needed)
Improve the documentation. PRs gratefully accepted.
I intend to use the improved C++ API with the rocksdb package and the related rocks-predicates package, for removing size limitations when scaling Prolog databases (redis is another possibility).
