Prolog totally missed the AI Boom

[j4n_bur53]

@stassa.p

Inductive logic programming at 30
https://arxiv.org/abs/2102.10556

The paper contains not a single reference to autoencoders!
Still they show this example:

I guess ILP is 30 years behind the AI boom. An early autoencoder
turned into transformer was already reported here (*):

SERIAL ORDER, Michael I. Jordan - May 1986
https://cseweb.ucsd.edu/~gary/PAPER-SUGGESTIONS/Jordan-TR-8604-OCRed.pdf

Well ILP might have its merits, maybe we should not ask
for a marriage of LLM and Prolog, but Autoencoders and ILP.
But its tricky, I am still trying to decode the da Vinci code of

things like stacked tensors, are they related to k-literal clauses?
The paper I referenced is found in this excellent video:

The Making of ChatGPT (35 Year History)
https://www.youtube.com/watch?v=OFS90-FX6pg

[j4n_bur53]

Hi,

One idea I had was that autoencoders would become
kind of invisible, and work under the hood to compress
Prolog facts. Take these facts:

% standard _, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
data(seg7, [0,0,0,0,0,0,0], [0,0,0,0,0,0,0]).
data(seg7, [1,1,1,1,1,1,0], [1,1,1,1,1,1,0]).
data(seg7, [0,1,1,0,0,0,0], [0,1,1,0,0,0,0]).
data(seg7, [1,1,0,1,1,0,1], [1,1,0,1,1,0,1]).
data(seg7, [1,1,1,1,0,0,1], [1,1,1,1,0,0,1]).
data(seg7, [0,1,1,0,0,1,1], [0,1,1,0,0,1,1]).
data(seg7, [1,0,1,1,0,1,1], [1,0,1,1,0,1,1]).
data(seg7, [1,0,1,1,1,1,1], [1,0,1,1,1,1,1]).
data(seg7, [1,1,1,0,0,0,0], [1,1,1,0,0,0,0]).
data(seg7, [1,1,1,1,1,1,1], [1,1,1,1,1,1,1]).
data(seg7, [1,1,1,1,0,1,1], [1,1,1,1,0,1,1]).
% alternatives 9, 7, 6, 1
data(seg7, [1,1,1,0,0,1,1], [1,1,1,1,0,1,1]).
data(seg7, [1,1,1,0,0,1,0], [1,1,1,0,0,0,0]).
data(seg7, [0,0,1,1,1,1,1], [1,0,1,1,1,1,1]).
data(seg7, [0,0,0,0,1,1,0], [0,1,1,0,0,0,0]).

https://en.wikipedia.org/wiki/Seven-segment_display

Or more visually, 9 7 6 1 have variants trained:

The auto encoder would create a latent space, an encoder,
and a decoder. And we could basically query ?- data(seg7, X, Y)
with X input, and Y output, 9 7 6 1 were corrected:

The autoencoder might also tolerate deviations in the input
that are not in the training data, giving it some inferential
capability. And then choose an output again not in the training

data, giving it some generative capabilities.

Bye

[stassa.p]

@j4n_bur53

I guess ILP is 30 years behind the AI boom. An early autoencoder
turned into transformer was already reported here (*):

I’m not sure what you’re saying here but it sounds like you mean that AI means autoencoders and there’s no AI without autoencoders. AI research has gone on since at least the 1940’s and autoencoders are just one approach to one AI task, pattern recognition. You’re not going to do, e.g. automated theorem proving with autoencoders, no matter how hard you try. But you can do automated theorem proving, with, say, SLD-Resolution, and therefore Prolog, and I’m pretty sure there’s very little doubt that automated theorem proving is an AI task, from the dawn of AI research to date. So how did Prolog miss the AI boom?

[stassa.p]

OK, so you’re saying that Prolog missed the AI Boom because it’s not Deep Leaning?

Well of course it isn’t. It’s Prolog. It has nothing to do with Deep Learning, including autoencoders.

Neither is anything else that isn’t Deep Learning and doesn’t use autoencoders, like Planning and Scheduling, SAT-solving, Program Verification and Model checking, Knowledge Representation and Reasoning etc. Did all classic AI “miss the AI Boom”?

It’s a different kind of AI. What exactly do you want to do with it? Turn it into something it isn’t, and can’t be? Why not just accept that classical AI and modern AI are two different sets of approaches with different capabilities, both of which are very useful -like jets and helicopters are both flying machines, both very useful and each very different from the other. Did helicopters “miss the Jet Age”?

@j4n_bur53

image1067×537 79.1 KB

The later is basically “relational learning”, which would fit Prolog very well…

I don’t think so. Prolog is “relational”, as well as many other things like “declarative”; but what it primarily is, is a programming language where the programs are definite logic theories executed by an SLD-Refutation proof of a Horn goal. That’s nothing to do with autoencoders nor can you replicate that result with autoencoders.

Nor should you try. There is a basic tenet of good engineering that says “use the right tool for the right job”. It’s right up there with “if it ain’t broke don’t fix it” and “don’t reinvent the wheel”. When you need to hammer a nail, use a hammer, not a screwdriver. When you need pattern recognition, use Deep Learning and possibly autoencoders. If you need theorem proving, use Prolog, or some other automated theorem prover.

[stassa.p]

@j4n_bur53

The
autoencoder adds hidden units in an auto-regressive way, and is

a prototypical element of self-supervised learning.

Funny you should mention self-supervised learning. This is what I’m working on:

vanilla/lib/poker at master · stassa/vanilla

It’s a self-supervised form of ILP. No autoencoders anywhere at all.

[j4n_bur53]

@stassa.p

It’s a self-supervised form of ILP. No autoencoders anywhere at all.

And, this only proofs my point that ILP doesn’t solve the problem
to make autoencoders and transformers available directly in Prolog.
Which was the issue I posted at the top of this thread.

Subsequently I would not look into ILP for Prolog autoencoders and
transformers is my point exactly. Because mostlikely ILP is unaware of the
concept of latent space. Latent space has quite some advantages:

• Dimensionality Reduction: It captures the essential structure of high-
  dimensional data in a more compact form.

• Synthetic Data: Instead of modifying raw data, you can use the
  latent space, to generate variations for further learning.

• Domain Adaptation: Well-structured latent space can help transfer
  knowledge from abundant domains to underrepresented ones.

If you don’t mention autoencoders and transformers at all, you are
possibly also not aware of the above advantages and other properties
of autoencoders and transformers.

In ILP mostlikely the concept of latent space is dormant or blurred,
since the stance is well we invent predicates, ergo relations. There
is no attempt to break down relations further:

https://www.v7labs.com/blog/autoencoders-guide

Basically autoencoders and transformers, by imposing some hidden
layer, are further structuring relations into an encoder and a decoder.
So a relation is seen as a join. The H is the bottleneck on purpose:

relation(X, Y) :- encoder(X, H), decoder(H, Y).

The values of H go through the latent space which is invented
during the learning process. It is not simply the input or output space.
This design has some very interesting repercussions.

[stassa.p]

@j4n_bur53

And, this only proofs my point that ILP doesn’t solve the problem
to make autoencoders and transformers available directly in Prolog.
Which was the issue I posted at the top of this thread.

My argument is that the “problem” you describe is not a problem that anyone has. There is no problem that absolutely requires autoencoders to solve, that we need to solve with Prolog, or with ILP.

@j4n_bur53

Dimensionality Reduction: It captures the essential structure of high-
dimensional data in a more compact form.

Dimensionality reduction is needed in Deep Learning because deep neural nets suffer from “the curse of dimensionality”, which however does not affect Prolog, or ILP because we’re not trying to learn from flat feature vectors, but from logic programs. To paraphrase Laplace, we don’t need that concept.

@j4n_bur53

In ILP mostlikely the concept of latent space is dormant or blurred,
since the stance is well we invent predicates, ergo relations. There
is no attempt to break down relations further:

On the contrary, that’s exactly what predicate invention does. But maybe you are using “predicate invention” to mean something other than what it really means? It means that we construct a new predicate that is necessary to construct the predicate of the examples, but that is not, itself, in the examples nor in the background theory. We pull it out of thin air. And the job that does is, indeed, that it breaks up relations into sub-relations or sub-routines, if you prefer.

For instance, if you look at this logic program with invented predicates, learned by Poker, the system I linked above:

q0(A,B):-empty(A,B).
q0(A,B):-zero(A,C),q0(C,B).
q0(A,B):-one(A,C),inv_1(C,B).
inv_1(A,B):-zero(A,C),inv_1(C,B).
inv_1(A,B):-one(A,C),q0(C,B).

zero([0|A],A).
one([1|A],A).
empty([],[]).

You’ll see that the concept of even parity (more specifically, bit strings with an even number of ‘1’'s) is constructed by first inventing the concept of odd parity (inv_1/2 is an invented predicate defining odd parity). That is “breaking down relations further”. There is no need to project anything into any “latent spaces” to do that. “Latent space” is not a concept that exists in First Order Logic, or higher order logics, nor is it a concept that is necessary to learn new concepts.

@j4n_bur53

Synthetic Data: Instead of modifying raw data, you can use the
latent space, to generate variations for further learning.

As to that, the system I linked to generates new positive and negative examples during the learning process. Again, see an example from the link I posted above, here:

% Learning even parity.
?- time( poker:learn(q0/2) ).
Hypothesis:
q0(A,B):-empty(A,B).
q0(A,B):-zero(A,C),q0(C,B).
q0(A,B):-one(A,C),inv_1(C,B).
inv_1(A,B):-zero(A,C),inv_1(C,B).
inv_1(A,B):-one(A,C),q0(C,B).
Positive examples:
q0([1,1,1,1],[]).
q0([0,1,1,0],[]).
q0([0,0,0,0],[]).
q0([],[]).
q0([0],[]).
q0([0,0],[]).
q0([1,1],[]).
q0([0,0,0],[]).
q0([0,1,1],[]).
q0([1,0,1],[]).
q0([1,1,0],[]).
q0([1,1,0,0],[]).
q0([1,0,1,0],[]).
q0([1,0,0,1],[]).
q0([0,1,0,1],[]).
q0([0,0,1,1],[]).
q0([0,0,1,1,0,0],[]).
q0([0,0,1,0,0,1],[]).
Negative examples:
q0([1,1,1,0],[]).
q0([1,1,1],[]).
q0([1,1,0,1],[]).
q0([1,0,1,1],[]).
q0([1,0,0,0],[]).
q0([1,0,0],[]).
q0([1,0],[]).
q0([1],[]).
q0([0,1,1,1],[]).
q0([0,1,0,0],[]).
q0([0,1,0],[]).
q0([0,1],[]).
q0([0,0,1,0],[]).
q0([0,0,1],[]).
q0([0,0,0,1],[]).
% 38,069,358 inferences, 2.594 CPU in 5.757 seconds (45% CPU, 14677343 Lips)
true.

Above, the logical atoms (Prolog terms) listed under “Negative Examples” are negative instances generated automatically by the system in order to learn its target theory. The “Positive examples” listing has a mix of given and generated examples (specifically, the first three positive examples are generated, the remaining positive examples, given).

If you can find any system, based on autoencoders, Deep Learning, or anything else, that can do the same thing I demonstrate above with the same kind of accuracy, sample efficiency, and computational efficiency as the system I linked above, please let me know. You won’t. Autoencoders can’t do anything like that. Nor can most Deep Learning systems, except maybe supervised LSTMs with an external memory, and that again only if they’re given thousands of both positive and negative examples. Learn from a handful of positive examples only? Invent a sub-concept not present in the training data? With Deep Learning? You can forget about that.

@j4n_bur53

Domain Adaptation: Well-structured latent space can help transfer
knowledge from abundant domains to underrepresented ones.

I’m sorry but I don’t know what that means. In any case there is no reason to try and emulate autoencoders or anything else. Prolog, SLD-Resolution, is a powerful algorithm and there isn’t anything else that comes close to it. And now we can also use it for learning.

[j4n_bur53]

But can you do the 7-segment challenge with ILP?

@stassa.p

no reason to try and emulate autoencoders

Well you don’t need to emulate autoencoders. When I asked for directly
available in Prolog, I didn’t ask for “emulation”, I asked for directly available
in Prolog. You can implement them directly in Prolog with not so much effort.

For example you can follow the idea of so called eForests:

AutoEncoder by Forest
Ji Feng, Zhi-Hua Zhou - 26 Sep 2017
[1709.09018] AutoEncoder by Forest

The paper once won an AAAI award. Lets first look at supervised
training. In supervised training you would give the latent space:

% https://en.wikipedia.org/wiki/Seven-segment_display
data(enc, [0,0,0,0,0,0,0], [1,1,1,1]).
data(enc, [1,1,1,1,1,1,0], [0,0,0,0]).
data(enc, [0,1,1,0,0,0,0], [0,0,0,1]).
data(enc, [1,1,0,1,1,0,1], [0,0,1,0]).
data(enc, [1,1,1,1,0,0,1], [0,0,1,1]).
data(enc, [0,1,1,0,0,1,1], [0,1,0,0]).
data(enc, [1,0,1,1,0,1,1], [0,1,0,1]).
data(enc, [1,0,1,1,1,1,1], [0,1,1,0]).
data(enc, [1,1,1,0,0,0,0], [0,1,1,1]).
data(enc, [1,1,1,1,1,1,1], [1,0,0,0]).
data(enc, [1,1,1,1,0,1,1], [1,0,0,1]).

data(dec, [1,1,1,1], [0,0,0,0,0,0,0]).
data(dec, [0,0,0,0], [1,1,1,1,1,1,0]).
data(dec, [0,0,0,1], [0,1,1,0,0,0,0]).
data(dec, [0,0,1,0], [1,1,0,1,1,0,1]).
data(dec, [0,0,1,1], [1,1,1,1,0,0,1]).
data(dec, [0,1,0,0], [0,1,1,0,0,1,1]).
data(dec, [0,1,0,1], [1,0,1,1,0,1,1]).
data(dec, [0,1,1,0], [1,0,1,1,1,1,1]).
data(dec, [0,1,1,1], [1,1,1,0,0,0,0]).
data(dec, [1,0,0,0], [1,1,1,1,1,1,1]).
data(dec, [1,0,0,1], [1,1,1,1,0,1,1]).

It takes a few milliseconds to learn a Bayes Classifier for the encoder
and a Bayes Classifier for the decoder. Works even in SWI-Prolog,
the output C is a decision tree, since its quite big I do not show it:

?- time(tree_assign_list(4, enc, S, C)).
% 39,126 inferences, 0.000 CPU in 0.022 seconds (0% CPU, Infinite Lips)
S = 0,
C = ...

?- time(tree_assign_list(7, dec, S, C)).
% 4,857 inferences, 0.000 CPU in 0.001 seconds (0% CPU, Infinite Lips)
S = 0,
C = ....

The scoring S = 0 shows that both decision trees have no error. But now
unsupervised training, it is based on the seg7 data here, which has also
some alternate digits. Its quite fun to watch how the score reaches zero:

It uses backpropagation, but not with neural networks, but instead
with decision trees. Here is an example latent space that the hidden
layer algorithm of an autoencoder finds:

?- tree_guess_list(4, seg7, 7, 100, S, E), write(S), nl,
   write('----------'), nl, data(seg7, X, _),
   tree_current_list(E, X, Y), write(Y), nl, fail; true.
0
----------
[1,1,1,0]
[1,1,1,1]
[0,1,1,1]
[0,0,1,1]
[0,1,1,0]
[1,0,1,0]
[1,0,1,1]
[0,0,0,1]
[1,0,0,1]
[0,1,0,1]
[0,0,0,0]
[0,0,0,0]
[0,1,0,0]
[1,1,0,1]
[1,0,0,0]
true.

Very much fun!

[stassa.p]

Decision tree learners are propositional learners that cant learn logic programs. As far as I can tell from a quick skim of the paper you link, it uses gradient boosted trees that are indeed propositional. Btw, so are autoencoders and deep neural nets. The motivation for ILP is to learn from relational data, or more precisely from logic programs (as I point out above).

FOIL is a Decision Tree learner that can learn first order programs (the name stands for First Order Inductive Learner) but, (in)famously, FOIL cannot learn parity. Nor can propositional decision trees. Those systems have trouble escaping from flat plains of their objective function, for example the string “111111” is an even-parity string but it is also a string of 6 ones, a string that starts with a 1, a string that ends with a 1, a string that is all ones etc etc, all competing hypotheses that have to be gotten rid of during learning. Decision trees can’t do that because there is zero gain from those examples and they don’t know which way to go.

@j4n_bur53

But can you do the 7-segment challenge with ILP?

I’m not sure what that is but if the target is a regular, context free or context-sensitive language then the answer is “yes”.

[stassa.p]

I’m sorry, I don’t understand what that is. Are you giving examples, an objective function… ? I don’t understand the “challenge”. Obviously ILP does not learn latent spaces, encoders or decoders, as it doesn’t learn neural nets- because it doesn’t need those things, as I’ve argued in my post above.

[stassa.p]

@stassa.p

Obviously ILP does not learn latent spaces, encoders or decoders, as it doesn’t learn neural nets- because it doesn’t need those things, as I’ve argued in my post above.

Sorry, I wrote that while you were replying. I think you’re arguing again that we should be trying to do with ILP something that is completely unnecessary to do with ILP. Well that’s not necessary.

It’s a bit like asking someone to use a Neural Net to learn a logic program. Neural nets don’t work that way. They learn weights, not programs. So there’s no point in asking “can you learn a program”?

I think you’re asking me to do the same thing: use a screwdriver as a hammer.

[stassa.p]

@j4n_bur53

% standard _, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
data(seg7, [0,0,0,0,0,0,0], [0,0,0,0,0,0,0]).
data(seg7, [1,1,1,1,1,1,0], [1,1,1,1,1,1,0]).
data(seg7, [0,1,1,0,0,0,0], [0,1,1,0,0,0,0]).
data(seg7, [1,1,0,1,1,0,1], [1,1,0,1,1,0,1]).
data(seg7, [1,1,1,1,0,0,1], [1,1,1,1,0,0,1]).
data(seg7, [0,1,1,0,0,1,1], [0,1,1,0,0,1,1]).
data(seg7, [1,0,1,1,0,1,1], [1,0,1,1,0,1,1]).
data(seg7, [1,0,1,1,1,1,1], [1,0,1,1,1,1,1]).
data(seg7, [1,1,1,0,0,0,0], [1,1,1,0,0,0,0]).
data(seg7, [1,1,1,1,1,1,1], [1,1,1,1,1,1,1]).
data(seg7, [1,1,1,1,0,1,1], [1,1,1,1,0,1,1]).
% alternatives 9, 7, 6, 1
data(seg7, [1,1,1,0,0,1,1], [1,1,1,1,0,1,1]).
data(seg7, [1,1,1,0,0,1,0], [1,1,1,0,0,0,0]).
data(seg7, [0,0,1,1,1,1,1], [1,0,1,1,1,1,1]).
data(seg7, [0,0,0,0,1,1,0], [0,1,1,0,0,0,0]).

Is this your training data? Or your learning target? Can you explain the learning problem?

For instance - explain what are the examples, and what kind of logic program you expect to be learned from them. No need to give the exact definition, just give me a set of examples and explain to me what it’s supposed to be representing.

In fact you don’t need to explain anything, just a set of examples is enough, but it would help me to understand what you are trying to do.

[stassa.p]

@j4n_bur53

And the autoencoder instance that the learning algorithm
for autoencoding produced:

To clarify, what learning algorithm was that?

[stassa.p]

Actually, you know what? I retract my previous comment (you can still see it if you click on the edit button) I don’t think your example makes any sense at all, not as a “challenge” for an ILP system. I thought about it for a few more minutes and I don’t understand what you’re trying to achieve.

The “autoencoder” you show above learned… an arbitrary mapping from 7-segment led activation strings to some binary strings, that are not the numbers to be displayed in binary form. It’s like it assigned a made-up symbol to each of the 7-led strings. What is the point of that?

I appreciate that’s how autoencoders work, I’m asking: what is the point of that from the point of view of learning with an ILP system? In ILP you want to learn a program that is consistent with your training data and background theory. I could trivially “solve” your problem by learning some completely arbitrary mapping like:

q0(b,[1,1,1,1,1,1,0]).
q0(c,[0,1,1,0,0,0,0]).
q0(d,[1,1,0,1,1,0,1]).
q0(e,[1,1,1,1,0,0,1]).
q0(f,[0,1,1,0,0,1,1]).
q0(g,[1,0,1,1,0,1,1]).
q0(h,[1,0,1,1,1,1,1]).
q0(i,[1,1,1,0,0,0,0]).
q0(j,[1,1,1,1,1,1,1]).
q0(k,[1,1,1,1,0,1,1]).
% Alternativers
q0(k,[1,1,1,0,0,1,1]).
q0(i,[1,1,1,0,0,1,0]).
q0(g,[0,0,1,1,1,1,1]).
q0(c,[0,0,0,0,1,1,0]).

So just learn a bunch of ground facts? That is not challenging and it isn’t interesting. ILP systems learn logic programs with both facts and rules, with recursion and with all the bells and whistles. Learning a bunch of facts is primitive.

I think I will wait for you to explain the problem, and the motivation, better before I attempt to solve it because currently I don’t understand what you mean and I think you’re still trying to do something with logic programming that is trivial to do, just because it’s hard to do with neural nets. That’s just not an interesting problem. Sorry.

[j4n_bur53]

@j4n_bur53

Thats not chain of thought (CoT), but one-shot thought already! You can
use it to build chain of thought and you are back to automated theorem
proving, etc… So the direct availability in Prolog systems could benefit a

lot of algorithms, like proof search, game search, etc…

The fastest chess engines now use neural networks, I suspect
this is a spin off from AlphaGo by DeepMind:

Efficiently updatable neural network
https://en.wikipedia.org/wiki/Efficiently_updatable_neural_network

Monte Carlo tree search
https://en.wikipedia.org/wiki/Monte_Carlo_tree_search

Monte Carlo tree search (MCST) was already around in 2006,
Efficiently updatable neural network (NNUE) is from 2018. You can
generate the ground facts to train the neural network via

synthetic data, so play chess with a deeper level. The update
refers to incremental computation of the output of a neural network.
But I observed that in eForest, the evaluation of a decision tree is

quite cheap, only linear in the number of given decision variables.
So maybe perfect to compress Prolog facts, an alternative to what
the tabling directive in SWI-Prolog does with its trie datastructure.

[logicmoo]

The fastest chess engines now use neural networks, I suspect
this is a spin off from AlphaGo by DeepMind:

Clarification on Neural Networks in Modern Chess Engines

I wanted to clarify a common misconception about modern chess engines. While neural networks play a critical role in engines like Stockfish, Leela Chess Zero (Lc0), and AlphaZero, these engines do not rely exclusively on neural networks for move calculations.

In reality, all major chess engines still use rule-based, Prolog-like systems for move generation and search—even those that incorporate MCTS and deep learning.

Here’s how:

1. Stockfish – Primarily relies on Alpha-Beta Pruning with Iterative Deepening, which is a deterministic rule-based search much like Prolog’s inference engine. The NNUE (neural network) only improves position evaluation, but it does not alter the fundamental move calculation process, which is still based on explicit logical rules and deterministic backtracking.

2. Lc0 & AlphaZero – Unlike Stockfish, these engines use Monte Carlo Tree Search (MCTS), but MCTS is still a structured, rule-based approach. It systematically explores move trees, evaluating each branch like a Prolog inference system. While it introduces probabilistic weighting, the tree itself is logically structured in a way that mirrors Prolog’s depth-first search with backtracking and state evaluation. The neural network only influences which branches are prioritized, but the actual move calculations still follow predefined game rules, much like a Prolog rule system evaluating different predicates.

3. Move Generation vs. Move Selection –

   • Move generation is completely rule-based and follows deterministic logical inference, just like Prolog.
   • Move selection (choosing the best move) is where neural networks and probabilistic techniques come into play, refining rather than replacing the rule-based search.

Even engines that rely on MCTS + neural networks (like AlphaZero and Lc0) still use a Prolog-like system for move exploration. The neural network does not calculate moves directly—it simply provides a heuristic guide for which logical branches to explore deeper.

Thus, modern chess engines are hybrids—they use rule-based systems (similar to Prolog) for move calculations and neural networks to refine evaluation and guide search heuristics.

[logicmoo]

The Code Tells the Story:

If we analyze the actual codebase, the difference is striking:

• Neural network components (such as NNUE in Stockfish or deep learning models in Lc0) typically consist of a few hundred lines of code—mostly matrix operations and evaluation heuristics.
• Rule-based search logic, which includes move generation, tree exploration, pruning techniques (like Alpha-Beta), and tactical calculations, spans tens of thousands of lines of code.
• Even in Lc0 and AlphaZero, which use Monte Carlo Tree Search (MCTS) instead of brute-force search, the MCTS itself is a structured, rule-based system—similar to Prolog’s inference engine, where logical constraints define move legality and structured search explores possible outcomes.

How Much is Actually Neural Network-Based?

Looking at the numbers, one could say that less than 3% of the total implementation is NN-based or stochastic, while over 97% remains rule-based deterministic logic.

I seriously doubt that a purely neural network or LLM could ever win a single game against these 97% logic-based chess systems.

[stassa.p]

@j4n_bur53

It doesn’t make any sense if you remove the perceptual structure
of the input column and represent it as atoms instead bit vectors. If you
remove this structure you wont have any benefits of an autoencoder!

Well that’s just the thing. What are “the benefits of an autoencoder”? If I look back on this thread I get the impression you’re arguing that you need an autoencoder because you want to learn a latent space and you want to learn a latent space because you need an autoencoder. So why do you need an autoencoder, again?

You gotta have some extrinsic motivation for a machine learning approach or any other technology that you want to use or implement and your example doesn’t explain what that is. You fail to motivate the proposed approach, I’d say. Your 7-segment LED example is trivially solvable in Prolog without any autoencoders at all.

Btw, you should try to understand why that is. You mentioned joins earlier in the conversation I believe. Well, in Prolog we can do “joins” pretty easily because we can unify variables. So if we have two literals p(X,Y) and q(Y,Z) we can go from X to Z via Y, just by unification.

But autoencoders, in fact statistical machine learning algorithms in general, including neural nets and gradient boosted trees, can’t do anything like that because they don’t have the ability to represent variables, or variable substitutions … let alone unification! So in your autoencoder example, the autoencoder is forced to construct some arbitrary structure in latent space that is really nothing but a constant, because otherwise it cannot connect its input bit-strings to its output bit-strings. Not if the input and output don’t always match.

And, yes, the “latent space” your example demonstrates is in every sense of the words a set of new constants. The whole thing is nothing but constants: it’s trying to do reasoning with only ground facts, no rules, no variables, no nothing at all. It’s an extremely restrictive setting and the only reason it’s worked so well over the years is that people have persisted at flogging it like the deadest of dead horses, and because after forty some years of working on that kind of thing they can now throw a metric shit-ton of data and compute at it, and it can do useful stuff at last; forty years later.

Can you imagine the kind of AI we would have if, instead of throwing all their resources at neural nets in a sustained human wave attack, Google, Facebook, Microsoft etc, had instead invested a 10th of that effort in logic programming and Prolog-like languages? They didn’t, and that’s why there’s neural nets all over the place, and “AIs” that are dumb as bricks. It was a political decision and nothing to do with technology.

Might I even suggest that those large tech corporations certainly benefit from championing an approach that requires the kind of resources -data, compute and engineers- that they are the best at procuring, and that nobody else can compete with them at? Just entirely coincidental I’m sure.

But you and I don’t have any of that. At least I know I don’t have bazillions in money and oodles of data and compute. But I have Prolog, and that’s enough for me.

[logicmoo]

@logicmoo

while over 97% remains rule-based deterministic logic.

Oops: When i said “deterministic logic” here … i meant to say nondeterministic logic! … As in Chess programs (regardless of NNs or not) do a search that includes Prolog style multiple choices. As in that code is making sure multiple candidates and choices are available to be evaluated/compared.

I would not mean to (I didn’t mean to, oops) apply Prolog meaning of determinism/non-determinism to NNs or LLMs.

[logicmoo]

I’ve been thinking about the ongoing conversation, and I wanted to clarify something. Are you, Jan, suggesting that because LLMs have turned out to be such a good idea, maybe autoencoders (or a similar approach) could also be surprisingly effective in ways we haven’t fully explored yet?

From my perspective, LLMs work well because they index statistical relationships in massive token datasets, allowing for efficient next-token prediction. That doesn’t necessarily mean that autoencoders—or other compression-based methods—would benefit from the same kind of scaling and brute-force approach.

Are you thinking that autoencoders could play a bigger role in tasks like language modeling, retrieval, or structured representation learning? Or are you just drawing a general parallel between their unexpected effectiveness?

Would love to hear more about what you’re thinking!

Best,
Douglas