Learning persistency for complex terms

I’m using: SWI-Prolog version 7.7.19

I’m trying to add dynamic discovery/configuration of devices to my SWI based home automation system. When a new device (display for starters) boots in my house it will publish a MQTT message with a JSON payload that declares its location and capabilities and I’d like my SWI-Prolog code to capture that configuration info into persistent terms it can then use to control those devices. All the MQTT and JSON part of this is already working.

For example a small LCD display might come online and the resulting term might be like:
device(‘lcd1’,‘mac_address1’, lcd(480,320, 16))
and an oled device might be:
device(‘oled1’,‘mac_address2’, oled(128,128))

I just came across the persistency library which might be used to allow Prolog to keep a database of these devices, but I’m not sure if that library allows a term within a term in the persistent declaration. I’m not totally sure on the form of my terms above, but my thought was that a device has generic attributes like name and mac address, but then has attributes specific to the type of device, width and height for both lcd’s and oled’s and bits per pixel for lcd. In the future other types of devices might not be displays at all, but maybe a sensor type.

Is the persistency library a good fit for this or should I resort to something else like writing terms to a regular file? The reason I’m looking for Prolog to persist this data is that maybe the device boots and advertises itself and then Prolog is restarted and then needs to restore itself so it can function before the next time the device advertises itself again.

As a side note, my HA system currently uses MQTT for all messaging between the components (arduino/ESP8266, Java, Prolog, Python) so using MQTT to do this device configuration/capability advertisement was an easy first step. But the ESP8266 devices are also discoverable via mDNS (which is how their OTA update function works) so eventually I’ll instead have central service discover devices via mDNS and then make a REST call to each new device to return a JSON description of its capabilities. The additional feature that will provide is that I believe mDNS can also help notify when a device goes away so I’ll be able to catch devices as they come and go on my network.

I think the persistency library is a great for this. It can handle arbitrary terms as arguments if you do not declare the type or declare it as any.

Thanks Jan! It might be a nice addition to the documentation to point to what the syntax and possible values are in a persistent declaration.

There is a (text) reference to library(error) at the bottom of the docs for persistent/1. You then have to read through it and look at the table in the docs for must_be/2.

If you look at the “simple example” code at the top of the section, you’d see, among other things, role:oneof(...). This is also documented in library(error).

Thanks Boris! I was reading section A.28 of the manual (which seemed like the majority of the documentation of the library) and hadn’t followed the link to persistent/1 so I didn’t see the reference to library(error). I’ve got it now and will try it out soon.

That is not the full list. To see the full list one needs to look at the code for has_type/2


To OP, you may find this of interest as it relates to library persistence.

Solving two consecutive dependent goals from command line

Interesting, I’ve gotten some basic stuff working with the code:
:- dynamic
display/3.

:- persistent
display(hostname:string, mac:string, tech:any).

I asserted and db_synched one entry then changed the MAC by one digit and reran to add a second entry, so good so far. Then I retracted one of the entries and synched again and here’s where I was surprised, the db looks like:

created(1573706954.9064286).
assert(display(“LCDst7735-d1e72”,“4c:11:ae:0d:1e:72”,lcd(160,128))).
assert(display(“LCDst7735-d1e72”,“4c:11:ae:0d:1e:73”,lcd(160,128))).
retractall(display(,“4c:11:ae:0d:1e:73”,),1).

So the database doesn’t really make a term completely go away, it appears to keep a log of all the transactions that ever happened. That’s good in some cases, but for the use I’m contemplating that would mean that the DB file just keeps growing as entries come and go.

Is there an option to really purge an item that gets deleted instead of having a create and delete transaction logged? UPDATE: found it, db_sync(gc).

1 Like

There is no need for this. The persistent directive does this. Right now it is harmless, but it is always possible that the implementation changes.

Yes, the db file keeps growing. For that purpose you have db_sync/1, which can perform a number of operations, one of which is gc. This safely writes a new database and safely uses this to replace the old (i.e., gc is safe against failure of the process or computer it runs on).

For a low frequence change DB like this, you can simply call db_sync(gc) after any retract. If the amount of garbage exceeds 50% it will actually cleanup.

1 Like

Thanks, I actually meant to remove the dynamic declaration part, guess I was just tired.

The gc solution is kinda cool.