Asked Claude, this is what it replied.
Note: There is a Summary Table at the end of the reply.
Suite-Level Isolation with Order-Independent Test Assertions
This is the proper testing strategy for persistent databases (RocksDB, library(persistency)) where you can’t easily clear data without deleting the entire database.
Core Concept
Suite-level isolation = Each test suite gets a fresh database, but tests within that suite share the same database instance.
Order-independent assertions = Tests check for specific data existence rather than exact counts, so they pass regardless of what previous tests added.
Example 1: Basic Suite-Level Isolation
% The suite setup runs ONCE before all tests
% The suite cleanup runs ONCE after all tests
:- begin_tests(my_database_tests, [
setup(setup_test_db(mydb)), % Creates fresh DB
cleanup(cleanup_test_db(mydb)) % Deletes DB
]).
test(first_test) :-
rdb_open('dbs/test_mydb', _),
rdb_assertz('dbs/test_mydb', item(apple)),
rdb_close('dbs/test_mydb').
test(second_test) :-
rdb_open('dbs/test_mydb', _),
rdb_assertz('dbs/test_mydb', item(banana)),
% Database now has: item(apple) from first_test
% item(banana) from this test
rdb_close('dbs/test_mydb').
test(third_test) :-
rdb_open('dbs/test_mydb', _),
% Database has: item(apple), item(banana)
findall(X, rdb_clause('dbs/test_mydb', item(X), true), Items),
% Items = [apple, banana] or [banana, apple]
% DON'T check exact count or order!
rdb_close('dbs/test_mydb').
:- end_tests(my_database_tests).
What happens:
setup_test_db(mydb) runs → Creates fresh dbs/test_mydb/ directoryfirst_test runs → Adds item(apple)second_test runs → Sees item(apple) from first test, adds item(banana)third_test runs → Sees both itemscleanup_test_db(mydb) runs → Deletes entire dbs/test_mydb/ directory
Example 2: Order-Independent Assertions (WRONG vs RIGHT)
WRONG - Brittle to test order
test(count_users) :-
rdb_assertz('dbs/test_app', user(alice)),
rdb_assertz('dbs/test_app', user(bob)),
findall(U, rdb_clause('dbs/test_app', user(U), true), Users),
assertion(length(Users, 2)). % FAILS if previous test added users!
Why it fails: If a previous test added user(charlie), this test expects 2 users but finds 3.
CORRECT - Check for specific data
test(has_users) :-
rdb_assertz('dbs/test_app', user(alice)),
rdb_assertz('dbs/test_app', user(bob)),
% Check that OUR specific users exist
rdb_clause('dbs/test_app', user(alice), true),
rdb_clause('dbs/test_app', user(bob), true).
% Don't care if other users exist too!
Why it works: Only checks that alice and bob exist, regardless of other data.
ALSO CORRECT - Check membership, not count
test(multiple_users) :-
rdb_assertz('dbs/test_app', user(alice)),
rdb_assertz('dbs/test_app', user(bob)),
findall(U, rdb_clause('dbs/test_app', user(U), true), Users),
% Check our users are in the list
memberchk(alice, Users),
memberchk(bob, Users).
% Don't check length(Users, 2) - there might be more!
Example 3: Testing Persistence (The POINT of Shared DB)
:- begin_tests(persistence_tests, [
setup(setup_test_db(persist)),
cleanup(cleanup_test_db(persist))
]).
% Test 1: Write data and close
test(write_and_close) :-
rdb_open('dbs/test_persist', _),
rdb_assertz('dbs/test_persist', config(version, '1.0')),
rdb_close('dbs/test_persist').
% Test 2: Reopen and verify data SURVIVED the close
test(reopen_and_read) :-
rdb_open('dbs/test_persist', _),
% This SHOULD see config(version, '1.0') from previous test!
rdb_clause('dbs/test_persist', config(version, V), true),
assertion(V == '1.0'), % Verifies persistence!
rdb_close('dbs/test_persist').
% Test 3: Add more data
test(accumulate_data) :-
rdb_open('dbs/test_persist', _),
rdb_assertz('dbs/test_persist', config(author, 'Eric')),
% Database now has TWO config facts
findall(K-V, rdb_clause('dbs/test_persist', config(K,V), true), Configs),
memberchk(version-'1.0', Configs), % From test 1
memberchk(author-'Eric', Configs), % From this test
rdb_close('dbs/test_persist').
:- end_tests(persistence_tests).
The point: These tests intentionally share data to verify that:
- Data survives
rdb_close/rdb_open cycles
- Multiple facts can coexist
- The database is truly persistent
Example 4: Multiple Test Suites (Isolation BETWEEN Suites)
% Suite 1: User management tests
:- begin_tests(user_tests, [
setup(setup_test_db(users)), % Fresh 'dbs/test_users'
cleanup(cleanup_test_db(users))
]).
test(add_user) :-
rdb_assertz('dbs/test_users', user(alice)).
test(list_users) :-
findall(U, rdb_clause('dbs/test_users', user(U), true), Users),
memberchk(alice, Users). % Sees alice from add_user
:- end_tests(user_tests).
% Suite 2: Product management tests
:- begin_tests(product_tests, [
setup(setup_test_db(products)), % Different fresh 'dbs/test_products'
cleanup(cleanup_test_db(products))
]).
test(add_product) :-
rdb_assertz('dbs/test_products', product(widget)).
test(list_products) :-
findall(P, rdb_clause('dbs/test_products', product(P), true), Products),
assertion(Products == [widget]).
% Can check exact count! No previous tests in THIS suite added products
:- end_tests(product_tests).
Isolation:
user_tests suite uses dbs/test_users/ (isolated from products)product_tests suite uses dbs/test_products/ (isolated from users)- Each suite starts with completely fresh database
- Tests within each suite share their suite’s database
Example 5: Real-World Scenario - RDF Triples
:- begin_tests(rdf_tests, [
setup(setup_test_db(rdf)),
cleanup(cleanup_test_db(rdf))
]).
% ❌ WRONG - Expects exact count
test(store_two_triples_WRONG) :-
rdb_assertz('dbs/test_rdf', rdf(subject1, pred1, obj1)),
rdb_assertz('dbs/test_rdf', rdf(subject1, pred2, obj2)),
findall(S-P-O, rdb_clause('dbs/test_rdf', rdf(S,P,O), true), Triples),
assertion(length(Triples, 2)). % FAILS if previous test added triples!
% âś… CORRECT - Check specific triples exist
test(store_two_triples_RIGHT) :-
rdb_assertz('dbs/test_rdf', rdf(subject1, pred1, obj1)),
rdb_assertz('dbs/test_rdf', rdf(subject1, pred2, obj2)),
% Just verify OUR triples were stored
rdb_clause('dbs/test_rdf', rdf(subject1, pred1, obj1), true),
rdb_clause('dbs/test_rdf', rdf(subject1, pred2, obj2), true).
% âś… CORRECT - Query specific subject
test(query_by_subject) :-
rdb_assertz('dbs/test_rdf', rdf(person1, knows, person2)),
rdb_assertz('dbs/test_rdf', rdf(person1, likes, pizza)),
% Check we can query by subject
rdb_clause('dbs/test_rdf', rdf(person1, knows, person2), true),
rdb_clause('dbs/test_rdf', rdf(person1, likes, pizza), true).
:- end_tests(rdf_tests).
Example 6: When You DON’T Want Order-Independent (Use Exact Counts)
:- begin_tests(isolated_test, [
setup(setup_test_db(iso)),
cleanup(cleanup_test_db(iso))
]).
% This is the ONLY test in the suite, so we can check exact counts!
test(exactly_three_items) :-
rdb_assertz('dbs/test_iso', item(a)),
rdb_assertz('dbs/test_iso', item(b)),
rdb_assertz('dbs/test_iso', item(c)),
findall(X, rdb_clause('dbs/test_iso', item(X), true), Items),
assertion(length(Items, 3)), % Safe! No other tests in suite
assertion(Items == [a, b, c]).
:- end_tests(isolated_test).
When exact counts are okay:
- Only one test in the suite
- First test in the suite (no previous tests added data)
- You’re testing a complete workflow in a single test
Summary Table
| Approach |
Isolation Level |
When to Use |
Example |
| Suite-level isolation |
Clean DB per suite |
Persistent databases (RocksDB, persistency) |
setup(setup_test_db(name)) on begin_tests |
| Order-independent assertions |
Tests share DB |
When tests accumulate data |
memberchk(pred/1, List) instead of length(List, N) |
| Test-level isolation |
Clean DB per test |
Non-persistent databases (dynamic facts) |
setup(retractall(fact(_))) on each test |
| Subprocess isolation |
Complete process isolation |
Rare cases needing total isolation |
process_create(swipl, ...) |
The key insight: For persistent databases, suite-level isolation IS the correct pattern. Trying to clear data per test defeats the purpose of testing persistence!