Llm: swi-prolog and large language models

[swi]

@jan

• Write a clean nice Prolog friendly wrapper and document it properly. Quite a bit of work, expertise is rare in our community. Fast and nice for the Prolog user.
• Use the Python interface. If it is complete (enough) and well documented, this is really easy for the Prolog programmer. Typically it does require a small Prolog abstraction to get a clean Prolog interface to the stuff you need (as opposed to wrapping everything cleanly in the first option).
• Use the SWI-Prolog ffi package. This is a little less maintenance to the Python ctypes interface as it extracts all information from the header (structs, constants, enums, etc.). It is a little tedious and error prone to use directly, so you’ll end up writing a cleaner user-facing interface on top of this that needs to be comprehensive and documented.

There is a fourth option, which I think might be optimal at this point:

• to write a prolog API which under the hood calls the main executable and controls it through command line arguments passing and receiving data through stdin/stdout/stderr pipes. Most of the features of llama.cpp are exposed in the main executable. This solution has none of the disadvantages of the python interface and is much less work than the prolog wrapper or using the c ffi package. It also gives an opportunity for the prolog api to mature, and later on could be replaced with prolog wrappers. It is also a way to make something available to the community with which expertise can be gained in the new field. (btw, I used to think that using stdin/stout for interfaces was hacky until I did some work with erlang; they recommend most extensions to be done this way).