Careful with compare/3 and Brent algorithm

The Brent algorithm, when also equipped with some post processing
shown on Wikipedia, can determine the start μ of a cycle and
the length λ of a cycle. This is for the linear case of a list.

In general the start and length are not unique, unless one requires
that the start is the minimal index and the length is minimal as well.
Here is an example of two different starts, illustrated

with a rational number:

10/81 = 0.(123456790) /* μ = 0, λ = 9 */

      = 0.12345679(012345679) /* μ = 8, λ = 9 */

Derived from the rational numbers 10/81 and 1/7, I found a
mishap in compare/3 for rational trees aka cyclic terms.
I am blaming some effect with Brent algorithm since is the

dominant algorithm in Prolog systems that became popular:

/* SWI-Prolog 9.1.7 and Scryer Prolog 0.9.1-207 */
?- X = X-0-9-7-6-5-4-3-2-1, Y = Y-7-5-8-2-4-1, X @< Y.
   true.

?- H = H-9-7-6-5-4-3-2-1-0, Z = H-9-7-6-5-4-3-2-1,
    Y = Y-7-5-8-2-4-1, Z @< Y.
   false.

Can this be fixed? It might also not directly related to the
Brent algorithm, and more the methods that are used inside
unify for example, with recording some visited terms in the functor.

This methods have a similar ambiguity.

Thanks for comment. I had noticed such inefficiency, which comes probably from
using ==/2 and using stack of pairs for history to detect looping. This is the reason why
I said it should be named useless_compare

However, mainly for curiosity, I am wondering how it improves if term_factorized/2 is extensional (i.e, maximaly structure shared one), though still it is not mature to say something positively.

Partial answer.

As terms are supposed to be ground, == is safely replaced with =.

X and Y in this are possible to be big cyclic terms, and
if “extensional” term_factorized are given for X and Y, it should be more easy
to test wheather X and Y are identical as expanded rational trees.

compare_with_stack(C, X, Y, P):-
	( 	X == Y -> C = (=)

Even for ground terms X and Y, there is cases in which X==Y termniates but X=Y does not terminate.
Thanks.

I rely on == in that it decides correctly whether two ground terms are equal in the sense of expanded rational trees. Do you doubt it ? Now I am loosing my confidence on == as for cyclic terms.

Unless I’m really wrong, ==/2 and =/2 are fully equivalent if both arguments are ground. In the current implementation they go through the same algorithm except that =/2 unifies if one side is variable and ==/2 fails.

Thanks for good comment. In fact, I have checked that with all == instances being replaced with =, the compare_with_stack works well as the original.

Maybe I read J.Burse’s case wrongly. I can’t reproduce the looping case now.

Strange. I can’t reproduce the ERROR.

For my compare_with_stack:

% ?- A=s(B,0), B=s(A,1), compare_with_stack(C, A, B).
%@ A = s(s(A, 1), 0),
%@ B = s(A, 1),
%@ C = (>).
``

The ERROR was for some code which was a modification of my original code,
and you explained why the modified code caused the ERROR. Is it right ?
If so, thank you for your kindness.

Anyway I have checked that the original codes compare_with_stack that I posted
is free from such ERROR.

It was said that complexty of naive unification is exponential at that time of Edinburgh Prolog.
I am curious about it nowadays.

I have tested using big binary trees of depth 30 as below.
It was suprising to see that it takes 0 second for unifying the two big trees. Something clever processing is done inside Prolog unification.

% ?- binary_tree(3, X).
%@ X = f(f(f(_A, _A), f(_A, _A)), f(f(_A, _A), f(_A, _A))).

% ?- binary_tree(30, X), binary_tree(30, Y), time(X=Y).
%@ % -1 inferences, 0.000 CPU in 0.000 seconds (53% CPU, -142857 Lips)

binary_tree(N, X):- length(Vs, N),
	binary_tree_(Vs, X).

binary_tree_([], _):-!.
binary_tree_([A|As], f(A, A)):-binary_tree_(As, A).

I’ve modified my logical version of compare to use this stack algorithm and it seems to work very well on everything I’ve tried (Kudos to @kuniaki.mukai) including one from @j4n_bur53 that was problematical for my previous attempts:

?- X=(X-b(X)), predsort(comparable,[X-b(X), X-a(X)], L).
X = X-b(X),
L = [X-a(X), X-b(X)].

Am I correct in assuming there are no outstanding “logical flaws” and the main issue of concern is performance?

Remember that we’re comparing a C implementation (compare/3) with a Prolog implementation, so that should be factored into the discussion. I’m seeing similar time ratio’s as @j4n_bur53:

?- X = X-0-9-5-4-2, Y = Y-7-5-2, time((between(1,10000,_), compare(_,X,Y), fail; true)).
% 19,998 inferences, 0.003 CPU in 0.003 seconds (100% CPU, 6586957 Lips)
X = X-0-9-5-4-2,
Y = Y-7-5-2.

?- X = X-0-9-5-4-2, Y = Y-7-5-2, time((between(1,10000,_), comparable(_,X,Y), fail; true)).
% 2,049,998 inferences, 0.485 CPU in 0.491 seconds (99% CPU, 4227576 Lips)
X = X-0-9-5-4-2,
Y = Y-7-5-2.

When I profile the Prolog version, it’s spending a little over half the time in memberchk/2 (actually $memberchk/3). 95% of the calls result in failures so it’s worst case (traversing the whole list). I believe this is also a C implementation so it’ll be hard to beat.

So I don’t think trying to compete with compare/3 is realistic. On the other hand the stack based Prolog implementation seems to be correct and it looks simple enough that it should point at ways to fix compare/3 which, I think we’re in agreement, is currently broken when comparing cyclic terms.

new `comparable/3` using match stack
:- module(comparableX, [comparable/3]).

comparable(Op,X,Y) :- comparable(Op,X,Y,[]).
	
comparable(Op,X,Y,_) :- Op == =, !,                   % Op = =, unify X and Y
	X = Y.
comparable(=,X,Y,_) :- X == Y, !.                     % X and Y identical, Op = =
comparable(=,X,Y,P) :- memberchk((X,Y),P), !.         % cyclic, Op = = to continue compare
comparable(Op,X,Y,P) :- compound(X), compound(Y), !,  % comparing two unequal compound terms
	functor(X,_,XA), functor(Y,_,YA),
	compare(FOp,XA,YA),           % compare arities, as per standard order
	( (FOp == =)
	 -> X =.. FX, Y =.. FY,       % arities equal, 	
	    comparableArgs(FX,FY,Op,[(X,Y)|P])  % lists are same length
	 ;  Op = FOp                  % arities unequal
	).
comparable(Op,X,Y,_) :- nonvar(X), nonvar(Y), !,      % comparing two other nonvars
	compare(Op,X,Y).              % standard compare
comparable(Op,X,Y,Refs) :-                                 % else X and/or Y are vars
	when((nonvar(X),nonvar(Y)), comparable(Op,X,Y,Refs)).  % defer
	
comparableArgs([X],[Y],Op,P) :- !,
	comparable(Op,X,Y,P).
comparableArgs([X|XArgs],[Y|YArgs],Op,P) :-
	comparable(IOp,X,Y,P),
	(nonvar(IOp)  % is arg compare Op defined?
	 -> comparableArg(IOp,Op,XArgs,YArgs,P)                     % yes, test
	 ;  when(nonvar(IOp), comparableArg(IOp,Op,XArgs,YArgs,P))  % no, defer
	).

% arg compare Op defined, continue if =, else exit
comparableArg(IOp,Op,XArgs,YArgs,P) :-
	(IOp = =) -> comparableArgs(XArgs,YArgs,Op,P) ; Op = IOp. 

It is good for me to hear that my idea on “useless_compare” is useful for you.

I would like to inform you of a drawback of using stack, aside stack search cost,
which I have no idea to fix for now. If your method is free from such drawback,
it would be nice.

Suppose comparing f(A, B) and f(C, D)
there is a case in which neither order between A and C, nor between B and D
are determined. In such case f(A, B) and f(C, D) is forced to be identified,
even when they are diffrent from each other by ==/2. There may be a way to manage such cases, but
I am not sure on necessary cost for that.

I can’t add more details because they are open problem for me. Also there seems other problems in being used as compare predicate argument of predsort.

I am sorry that I am not good at read codes written by others, often even my own old codes.
But I would like to help in what I could.

From previous thread:

I had a quick look at do_compare() but it’s beyond my capabilities to figure out what’s going on. Somebody with a better understanding of the C FLI might have a chance.

I have written codes semi_lex_sort/2 which sorts a list of ground possibly cyclic terms.
Due to Carson example, result can not be always lexcal ordered list.

The code has not been reviewed carefully, but runs for several test cases as expected.
To force ordering cyclic terms in compatible way in a sorting for a given list with
the standard lexical ordering, a global variable compare_history is used. Although
it may look ad hoc, this is best codes to show what I have vaguely in mind. I will be glad
if this codes is useful for you and someone else. Of course I am far from insisting that
this is a meaningful solution, but a hobby programming from curiosity inspired by the Calrson example.

% ?- semi_lex_sort([a,b,a], X).
% ?- A=f(A), B=f(B), semi_lex_sort([A, B], X).
% ?- A=f(A, 0), B=f(B,1), C=f(C, 2), semi_lex_sort([A, B, C], X).
% ?- A=f(A, 0), B=f(B,1), C=f(C, 2), semi_lex_sort([C, B, A, A, B, C], X).
%@ A = f(A, 0),
%@ B = f(B, 1),
%@ C = f(C, 2),
%@ X = [f(A, 0), f(B, 1), f(C, 2)] 

semi_lex_sort(X, Y):- b_setval(compare_history, []),
					  predsort(semi_lex_compare, X, Y).
%
semi_lex_compare(C, X, Y):- b_getval(compare_history, H),
					compare_with_stack(C, X, Y, H, H0),
					b_setval(compare_history, H0).

%
compare_with_stack(C, X, Y, H, H):- X == Y, !, C = (=).
compare_with_stack(C, X, Y, H, H0):- compare_with_stack(C0, X, Y),
	(	C0 = (=) -> force_order(C, X, Y, H, H0)
	;	C = C0, H0 = H
	).
%
force_order(C, X, Y, H, H0):- functor(X, F, N),
	(	select(F/N-G, H, H1) -> true
	;	G = [],
		H1 = H
	),
	force_order_in_arity_grouping(C, X, Y, G, G1),
	H0 = [F/N-G1|H1].
%
force_order_in_arity_grouping(C, X, Y, G, G0):-
	(	memberchk(X, G)->
		(	memberchk(Y, G) ->
			(	precede(X, Y, G) -> C = (<)
			;	C = (>)
			),
			G0 = G
		; 	C  = (>),
			G0 = [Y|G]
		)
	;	memberchk(Y, G)->
		C = (<),
		G0 = [X|G]
	;  	C = (<),
		G0 = [X, Y|G]
	).

% 
precede(X, _, [X|_]):-!.
precede(X, Y, [U|L]):- Y\==U, precede(X, Y, L).

%
compare_with_stack(C, X, Y):- compare_with_stack(C, X, Y, []).
%
compare_with_stack(C, X, Y, P):-
	( 	X = Y -> C = (=)
	;	(atomic(X); atomic(Y)) -> compare(C, X, Y)
	;	memberchk(X-Y, P) -> C = (=)
	;	functor(X, F, N),
		functor(Y, G, M),
		compare(D, N, M),
		(	D = (=) ->
			compare(E, F, G),
			(	E = (=) ->
				compare_args_with_stack(C, 1, X, Y, [X-Y|P])
			;	C = E
			)
		;	C = D
		)
	).
%
compare_args_with_stack(C, K, A, B, P):- arg(K, A, X), arg(K, B, Y), !,
	compare_with_stack(D, X, Y, P),
	(	D = (=) ->
		K0 is K+1,
		compare_args_with_stack(C, K0, A, B, P)
	;	C = D
	).
compare_args_with_stack(=, _, _, _).

Unless you’re talking about logic variables, I’m not sure what you mean. For a logical compare (my focus), order is determined or it’s deferred (as a when constraint) until it is. But this does mean memberchk/3 isn’t correct, a ‘==/2’ based member is required, so the cost is higher.

But now it handles cyclic terms:

?- comparable(Op,f(A,B),f(C,D)).
when((nonvar(A), nonvar(C)), comparable:comparable(_A, A, C, [(f(A, B), f(C, D))])),
when(nonvar(_A), comparable:comparableArg(_A, Op, [B], [D], [(f(A, B), f(C, D))])).

?- comparable(Op,f(A,B),f(C,D)), A=s(C,0), C=s(A,1).
Op = (>),
A = s(s(A, 1), 0),
C = s(A, 1).

?- comparable(Op,f(A,B),f(C,D)), B=s(D,0), D=s(B,1).
B = s(s(B, 1), 0),
D = s(B, 1),
when((nonvar(A), nonvar(C)), comparable:comparable(_A, A, C, [(f(A, s(s(B, 1), 0)), f(C, s(B, 1)))])),
when(nonvar(_A), comparable:comparableArg(_A, Op, [s(s(B, 1), 0)], [s(B, 1)], [(f(A, s(s(B, 1), 0)), f(C, s(B, 1)))])).

Performance isn’t my primary concern at this point, so this looks pretty good to me.

I don’t have much hope this will be resolved (does ISO say anything specific about comparing rational trees?) or if it is even resolvable (ref. Carllson).

Short of doing that, can we say anything about compare_with_stack? Specifically, if compare/3 was semantically equivalent (it currently isn’t in SWIP), would all terms, including cyclic terms, be ordered such that comparison operations are transitive, meaning dependant predicates like sort/2 and setof operate correctly? (From your “It only seems to be robust …” comment, maybe it does.)

Very good question, specifically for comparison. I don’t have any beyond the basic requirement that, generally speaking, they should behave much the same as acyclic terms when used with system predicates (compare, sort, setof, write, …) and, if they don’t, it should be clearly documented.

Regarding keysort, having pairs with keys which are lexically key-value pairs seems like a recipe for confusion (they certainly confuse me), let alone having keys which are cyclic terms. I really haven’t used keysort/pairs to any degree so this shouldn’t carry much weight, but some documented restrictions and/or best practices seem reasonable (IMO).

For example, consider ground terms A, B, C, D such that
A = s(B, 0), B = s(A, 1), C = s(A, 1), and D = s(B, 0).
Then by Carlson’s example, neither order between A and C, nor between B and D are determined.

A short answer. I abondon that compare_with_stack is transitive due to the Carlson’s example. However, using history of nodes of such “unstable” pairs during sorting of a given list, it can sort a list in a “semi-lexical ordering” The initial version of semi_lex_sort initialises the history for each sorting for simplicity reason. If such unstable pairs are rare, tabling the history might be interesting.

Not sure why you say this. Yes, in general, comparison of unequal cyclic terms is mathematically undecidable (ref. Carllson). But I’m looking for an “engineering” solution: a compareU(Op,X,Y) (U for useful) such that member(Op,[<, =, >]) is true specifying an order, and X and Y are any two Prolog terms (cyclic or acyclic) such that the following properties are preserved:

  1. Persistence: (compareU(Op,X,Y), compareU(Op,X,Y)) is always true.
  2. Transitivity: (compareU(Op,X,Y), compareU(Op,Y,Z)) -> compareU(Op,X,Z) ; true) is always true.

Does compare_with_stack satisfy these requirements? I think it might, but I’m not as creative at generating challenging test cases as @j4n_bur53 .

It follows from Persistency that compareU(Op, X, Y) is always true (for any Op, X, Y)
This is almost contradiction. Is this what you intend ?

As for Transitivity, I am not sure for now. For compare_with_stack = includes case in which
X and Y are incomparable like Carlson’s pair. So, the following might be true.
But I am not sure at this moment.

compareU(=, X,Y), compareU(=,Y,Z)) -> compareU(=,X,Z)