Glitch in term_factorized/3, stack overflow

[ridgeworks]

Summarizing: term_factorized/3 uses library(rbtrees) which depends heavily on compare/3 which is buggy given cyclic terms so, in hindsight, not surprising it isn’t reliable.

In fact many (most, all?) of the problems encountered so far seem to be traceable back to the same bug, so I think a useful step would be to raise a new issue on using compare/3 with left-recursive rational trees (may not be the only issue but it’s simple) based on the following example:

?- X=(X,a), Y=(Y,b), Z=(Y,c), compare(A,X,Y), compare(B,Y,Z), compare(C,X,Z).
X = (X, a),
Y = (Y, b),
Z = (Y, c),
A = B, B = (<),
C = (>).

specifically, the violation of the transitive property of comparison system predicates.

I think Jan essentially localized the problem:

@jan

The cyclic comparison uses the same trick as for cyclic unification I learned from Bart Demoen and that is described in pl-prims.c in a comment starting at line 71. The idea is that while going through compounds you create a reference link from the compound at one side to the other one. Now, for unification and ==/2 this works fine as the argument order does not matter. For comparison, the argument order does matter.

Is there any other useful input that would help resolve this issue?