What fruits are or even what a JSON list looks like is still internal knowledge stored in the weights. The point I think @stassa.p is making is that it giving the correct answer in this one case is essentially a coincidence, not evidence of understanding. It’s entirely possible some simple permutation of the input could make it output something wildly different (note that it included a JSON list in the output, but its response was not in fact a JSON list as requested!). You’re shaking a magic eight-ball and sometimes the “answers” seem to resonate.
1 Like
Thanks for sharing this, it looks interesting. I was able to install it and get the basic examples working.
1 Like
This is great, thank you for sharing! I see in your examples you show Prolog playing the role of a knowledge base in a RAG-type system, very cool and makes sense! There are some other cool features of Prolog that I think could be useful to LLMs, such as a reasoning agent for certain types of reasoning and/or constraint problems. While it’s already great to use Prolog as a knowledgebase for a RAG, have you thought of any other applications you could use your library for?
My initial idea was to use Prolog as a reasoning engine to control the reasoning process and workflow of LLM-based agents. However, I decided to take a simpler approach and integrate the library into an existing ecosystem (LangChain/LangGraph) rather than building a standalone solution.
As it stands today, the library can serve as a tool for an LLM-based agent to extend its knowledge or enhance its reasoning capabilities. Another potential use case is as a guardrail system, ensuring that LLM responses comply with specific restrictions to prevent inappropriate, dangerous, or off-topic interactions.
My goal is to develop real-world examples that demonstrate the library’s potential and spark interest in both the library itself and Prolog. Any help or collaboration in this effort would be greatly appreciated!
The prolog interface has already been included in LangChain documentation:
Does this mean that pip install langchain-prolog installs swi-prolog as well? Because I don’t see the documentation saying that dei-prolog needs to be installed as a prerequisite
No. SWI-Prolog is a prerequisite.
The read the docs langchain-prolog’s page and the github repository’s readme do mention it:
but the langchain documentation indeed fails to mention this. I will see if they agree to include it. Thank you for bringing this up.
1 Like
I developed a small demo to showcase how easy is to integrate Prolog with an LLM based application:
It runs in Google Colab and it does not require any local installation.
I’d love to hear your thoughts!
The translation from natural language to a prolog query is useful, assuming the knowledge base already exists.
Do you plan to add features for extracting prolog facts from text and asserting them into the knowledge base?
James
Knowledge graphs and ontological reasoning are promising for applying Prolog’s solution search capabilities. However, extracting knowledge from text and other types of media is the difficult part.
Some people are using LLMs, which shine in natural language ‘understanding,’ but the extracted knowledge typically requires validation to prevent hallucinations and incorrect relationships from being inserted into the database.
I think the most effective approach would be to use LLMs and Prolog as part of a pipeline where the LLM extracts initial structured knowledge, and a Prolog rule-based systems verify and constrains the output by linking to existing knowledge bases or applying a scoring system.
I am still figuring out what to prioritize, and this is on the shortlist, but it will take me a few months of research—because the last time I worked on something like this, ELIZA was still a thing.
Click on “ Open With” and select “Google Colaboratory”.
If you don’t see “Google Colaboratory” in the drop-down menu, select “Connect more apps” and you should see “Google Colaboratory” in the app list.
Fixed!
2 Likes