Jev, Prolog, Pi, and the dream of probabilistic logic programming

Discussion
1

Hi all,

Jev, a new model (not an LLM, bit rather a modern version of BERT), has been making the rounds on the usual outlets these past few days. As I believe the core concepts of Jev map very neatly onto Prolog predicates, I’ve decided to integrate it into DeepClause (a runtime and SDK built on top of SWI-Prolog to combine LLMs/Agents and Prolog in a small DSL). The write-up can be found here. If there are people in this forum working on probabilistic logic programming, then I’d be very curious about their thoughts on this.

If you want to try it out yourself, the easiest way to get started is using the Pi coding agent and the deepclause-pi extension via “pi install npm:deepclause-pi”.

Thanks,
Andreas

2

I love the idea! It would be great for protect management agents

3

@apfadler, thank you for the interesting project and write-up. I think that you are onto something with respect to integrating logic programming and decision models.

What do you think of the following initial sketch involving noul?

capital_of(X, Y) :- noul("Is `x` the capital of `y`?", #{x:X, y:Y}, 0.75).

I would be interested in brainstorming on these topics and sketching more ideas occurring over the course of a discussion, including with respect to choice or score.

You might find interesting the Decisions API, an early design sketch which could become a feature in Web browsers.

4

The sketch with noul wrapping a natural-language question and binding X and Y through the map argument is a clean way to keep the predicate relational rather than collapse it into a single boolean call. Tying a 0.75 threshold to it raises the question of whether the score should be an output unification target instead, so callers can branch on the value rather than just succeed or fail. Choice and score semantics would matter a lot there. I’m curious how the 0.75 interacts with backtracking when multiple bindings are plausible.

5

Using such map arguments to provide structured state objects can enable using the backticks features in questions or instructions. State can be a string, array, or object. See the SWI Prolog JSON library for how we might, more broadly, go about representing state.

"Is `x` the capital of `y`?"

"Is `x.text` the capital of `y.text`?"

"Is `x.names[0].text` the capital of `y.names[0].text`?"

It seems that you are describing:

capital_of(X, Y) :- noul("Is `x` the capital of `y`?", #{x:X, y:Y}, P), P >= 0.75 .

Also, a noul can, optionally, receive text-string descriptions for its true and false criteria.

capital_of(X, Y) :- noul("Is `x` the capital of `y`?", #{x:X, y:Y}, #{true: "`x` is the capital of `y`.", false: "`x` is not the capital of `y`."}, P), P >= 0.75 .
6

The Topic is intressting. On which based the trreshold Value 0.75 - I think the Value comes from the Numbers how times the sentence is used, and should be a very lower value?

7

The threshold value of 0.75 was arbitrary. It could be any value desired by a developer, e.g., 0.5 .

8

The value should not be “any desired value by the developer”.
By building a database for the “captial of …” should be given a value from an existing System.
a E.g.: The Countries should have Codes (0 … 999) 0 = unspecified, 1 = Canada, 2 = US, …
b E.g.: The Regions should have Codes (e.g.: 0 .. 999) 0 = unspecified, 1 = Texas, …
c E.g.: The Region sizes should have Codes (e.g. 0 .. 999) 0 = unspecified/unknown, 2 = 2 Million km^2

So, you have 3 simple criterias:
a = 000 … 999
b = 000 … 999
c = 000 … 999

These criterias should be combined a + b + c: 002 | 001 | 002 | …

So, you have a threshold experimental value of: #0.002001002

9

@paule32, the example capital_of/2 was also “arbitrary”. By “arbitrary”, I meant just for purposes of example as syntactic possibilities were under discussion. I was intending to briefly focus on syntactic possibilities for potential predicates:noul, choice, and score.

As interesting, here is an article on setting threshold values with models like Jev: How should you use Jev’s probabilities to set decision thresholds? - Vercel .

One could also, as easily, express a predicate like:

hf(X, Y) :- noul("Is `x` a historical figure of `y`?", #{x:X, y:Y}, P), P > 0.5 .

Also, AI decision models can do more than answer factual questions. They can answer questions about natural-language content provided to them in state data, e.g., about user-provided messages.

Here is a quick example of whether a sentence mentions a thing, a noun, subject, or topic:

mentions(S, T) :- noul("Does `s` mention `t`?", #{s:S, t:T}, P), P > 0.6 .

Here is another quick example of a paraphrase/2 predicate using a decision model:

paraphrase(A, B) :- noul("Is `a` a paraphrase of `b`?", #{a:A, b:B}, P), P > 0.6 .

One might want to use guard clauses, are A and B valid sentences:

paraphrase(A, B) :- sentence(A), sentence(B), noul("Is `a` a paraphrase of `b`?", #{a:A, b:B}, P), P > 0.6 .

One could also go “meta” about discussing such predicates. That is, for example, shouldn’t paraphrase/2 be symmetric, paraphrase(B, A) being the same as paraphrase(A, B)?

One could also include in the state data the paragraphs or other document excerpts from which sentences were drawn, as context.

Also, one could ask, using noul or choice, questions like whether a given sentence is implied by an array of other sentences or how a set of evidence otherwise relates to a claim.

As one can see, there are many interesting possibilities with respect to (probabilistic) logic programming and AI decision models such as Jev.

10

Hello,

I do not know if is for interests:
I implement my own Version of Prolog with goal of SWIPL compatiblity.
For learning Prolog, and other reasons I started a Project written in Python 3+.
You can take a look at:

https://github.com/paule32/Commodore_C64_DISM

It is a little bit twisted, and most parts are localised for native German Readers, so I have still to do translating it to English native Readers - but I think that should not be a Problem.

Why do I point you to this:
I looking forward for peoples that help me to built a set of learning, and applicate Software in one thing…

You are welcome on Board to write great Software, not simply producing Scripts for Console usage.

I can not gurantee that is the non plus ultra - but this could be a little playground to learn something new, and I thinking on that we dont should move, and jump on the ride of big players, because the bubble is on the Limit, and we need Software that make us independent…

Thanks for reading.

11

@AdamSobieski Thank you for interest and initiating the discussion!

The current primitives which are available in deepclause-sdk should be helpful for experimenting with your idea. Please do feel free to play around and submit PRs, e.g. for the new Decision API or the noul predicate.

Instead of backticks, deepclause currently supports {} based string interpolation. So you can write e.g. X…=45, task(“Generate a number larger than {X} and return it to me in {Y}”, integer(Y)), Y > 72… deepclause does some preprocessing before the execution (based on the string interpolation pack) and keeps track of variable names so we can use variable names directly in prompts. Makes the code look nice, but your approach might be a bit more robust.

I’ve recently deployed a web only version of deepclause at https://web.deepclause.ai, which might help with the experimentation, but is more geared towards building harnesses that can defined in the form of Prolog module. Thanks to SWI’s WASM build it runs completely in browser (without a backend), all you need to provide is an API key for your provider (or have a local LLM running).

If you prefer to have everything directly in SWI, then this also shouldn’t be hard to try and test. The latest round of models is very good at writing and debugging Prolog already (at least this is my impression), so many ideas can be tested in a matter of minutes.

12

In my opinion, combining both {}-based string interpolation and backticks, a feature of Jev and other decision models for referring to components of structured state arguments, would be useful for developers.

Also, I have enjoyed rapidly prototyping some SWI-Prolog for experimentation purposes, using Ollama and a model named “nimble”. As interesting, here is some source code:

:- use_module(library(http/http_client)).
:- use_module(library(http/http_json)).
:- use_module(library(json)).
:- use_module(library(json_convert)).

noul(M, Q, S, P) :- http_post("http://localhost:11434/v1/systemone", json(_{model:M, state:S, questions:_{q: _{type:"noul", instructions:Q}}}), Y, [request_header('Content-Type'='application/json'), json_object(dict)]), P = Y.answers.q.noul .

choice(M, Q, S, C, R, P) :- http_post("http://localhost:11434/v1/systemone", json(_{model:M, state:S, questions:_{q: _{type:"choice", instructions:Q, criteria:C}}}), Y, [request_header('Content-Type'='application/json'), json_object(dict)]), R = Y.answers.q.choice, P = Y.answers.q.confidence .

mentions(S, T) :- noul("nimble", "Does `s` mention `t`?", _{s:S, t:T}, P), P > 0.5 .

paraphrase(A, B) :- noul("nimble", "Is `a` a paraphrase of `b`?", _{a:A, b:B}, P), P > 0.5 .

entails(E, C) :- E = [], string(C), !, noul("nimble", "Is the statement `c` a tautology?", _{c:C}, P), P > 0.5 .
entails(E, C) :- E = [F], string(C), string(F), !, noul("nimble", "Does the statement `e` entail the claim `c`?", _{e:F, c:C}, P), P > 0.5 .
entails(E, C) :- E = [_|_], string(C), !, noul("nimble", "Do the statements `e` entail the claim `c`?", _{e:E, c:C}, P), P > 0.5 .
entails(E, C) :- string(E), string(C), !, noul("nimble", "Does the statement `e` entail the claim `c`?", _{e:E, c:C}, P), P > 0.5 .

example(T, L) :- choice("nimble", "Which label fits this ticket?", T, _{billing: "Payments and refunds.", bug: "Software errors.", account: "Login and account access."}, L, P), P > 0.5 .

Thank you for that hyperlink to deepclause. I will take a closer look!