Library(persistency) struggling with concurrency

Help!
1

I’m using: SWI-Prolog version 8.1.2, library(persistency) for a web application.

I want the code to:
Accept 8+ async requests via AJAX that do a read, retract and write on a persistent predicate stored in the attached db file. I want the db file to periodically sync, as this will be running as a web application if the process were to end it’s likely going to be in an unwanted and uncontrolled manner.

But what I’m getting is:
I’ll get about 6/8 requests succeed, but for a couple it’ll fail, telling me the database file doesn’t exist. I’m assuming this is because at that moment a gc operation is running. I’m hoping that I simply need to change the db_sync(gc) to something else, but I’m somewhat confused as to what the best option would be. Only one module touches this database via 4 predicates that could potentially be called simultaneously.

My code looks like this:

:- persistent project(id:atom, situation:any).

do_action(Action) :-
    user_current_project(ID),
    with_mutex(project_db,
            ( project(ID, Sit)
            , do(Action, Sit, NewSit)
            , retractall_project(ID, _)
            , assert_project(ID, NewSit)
            )
     ),
    db_sync(gc).
2

You need one mutex to control all related db action: writing, syncing, reading. The latter only if the writes do things like retract/assert pairs to update. Without locking the read you may be reading just between the retract and assert and read nothing. You definitely do not want two db(syncs) running on the same db and a db(sync) concurrently with a write may loose information.

You also do not want to call db_sync(gc) often. This basically writes a new database from the facts in core and swaps the file (atomicaly if the OS can do so). Decide on the flush (close, flush, buffered), typically using close for really infrequently updated databases (so you can easier manipulate them on the file system while the server runs), flush for most work and buffered for high performance where you do not care loosing some data in case of a crash/power failure/.

3 Likes
3

Hi Jan,
Thank you! I’ve moved my calls to db_sync/1 into the mutex and everything is working swimmingly. That was a lot less painful to resolve than I had feared.