Foreach bug?

I am confused that your original version foreach is no more than that foreach = findall + foldl.

Sounds nice and great, and what I would like to know is whether the “standard” prolog coder could wrote something equivalent.

EDIT:

I think what I would be trying now might be to make the naive version of foreach much more elegant. Unfortunately I am not familiar to handle Prolog local stack, and without that there is no way for me. Aside possible performance issue, I should be satisfied with “foreach = findall + foldl” for its clean simplicity.

% ?- naive_foreach(V, between(1, 10, V), plus, R).
%@ R = 55.

naive_foreach(V, G, A, R):- retractall(acc(_)),
	assert(acc(0)),
	( 	call(G),
		retract(acc(X)),
		call(A, V, X, Y),
		assert(acc(Y)),
		fail
	;	retract(acc(R))
	).

EDIT: nb_setarg version instead of assert/retract

% ?- foreach1(V, between(1, 10, V), plus, R).
%@ R = 55.

foreach1(V, G, A, R):-
	Acc = acc(0),
	foreach1_(V, G, A, Acc),
	arg(1, Acc, R).

%
foreach1_(V, G, A, Acc):-
		call(G),
		arg(1, Acc, X),
		call(A, V, X, Y),
		nb_setarg(1, Acc, Y),
		fail.
foreach1_(_, _, _, _).

EDIT: Sample query with two dimensional array indexing.

% ?- N = 10000, time((foreach1([V,U], (between(1, N, U), between(1, N, V)),
%	test_action, R))).
%@ % 600,010,004 inferences, 33.882 CPU in 33.906 seconds (100% CPU, 17708672 Lips)
%@ N = 10000,
%@ R = 2500500025000000.

test_action([I,J], X, Y):- Y is I*J + X.

EDIT:
Being inspired by Jan’s explaining foreach, I have introduced a generic predicates fold/[5, 4, 3], which integrates foreach and foldl/4. The fold is what I wanted to have for a long time , and it is similar to for construct in C. The source expanding for fold follows.

As Jan gave me warning, I am afraid I am missing something.
For example, Jan’s codes of foreach uses term_variables but mine does not. I only hope no serious problems exists in my simple minded model. So far so good. Performance and usability is what I wanted.

EDIT: Source codes on fold expansion deleted because of revision.

EDIT: Speed of fold/3 has been improved due to new explicit syntax of port of consumer side. (Port^Consumer).
Need to check closures and global variables.

% ?- N is 10^8, time(fold(J, between(1, N, J), X^(X=X))).
%@ % 200,000,001 inferences, 4.925 CPU in 4.941 seconds (100% CPU, 40607323 Lips)
%@ N = 100000000,
%@ J = X.

% ?- N is 10^9, time(fold(J, between(1, N, J), X^(X=X))).
%@ % 2,000,000,001 inferences, 49.470 CPU in 49.519 seconds (100% CPU, 40428767 Lips)
%@ N = 1000000000,
%@ J = X.
1 Like

You do a foldl style operation using call/N, while the SICStus fased foreach/2 calls the same goal over and over again using call/1.

I see. Now I read
?- foreach(between(1, 5, M), member(M, L))
as
find L such that for all M satsifying between(1, 5, M)
member(M, L) holds.

EDIT:

Following your comment, I have tested a toy foreach.
I have seen too many possible issues on this toy approach using numbervars.

% ?- toy_foreach(between(1,5,J), member(J, L), [L], Ws).
%@ L = A,
%@ Ws = [[1, 2, 3, 4, 5|_]] .

toy_foreach(A, B, Vs, Ws):- numbervars(Vs),
	findall(B, A, Bs),
	hzip(Vs, Ws, Zip),
	subst_numbervars(Zip, Bs, Cs),
	maplist(call, Cs).

%
hzip([],[],[]).
hzip([X|Xs],[Y|Ys],[X-Y|Zs]):-hzip(Xs, Ys, Zs).
%
subst_numbervars(_, X, Y):- (var(X); atomic(X)), !, Y = X.
subst_numbervars(M, '$VAR'(I), A):-!, memberchk('$VAR'(I)-A, M).
subst_numbervars(M, X, Y):- functor(X, F, N), functor(Y, F, N),
	subst_numbervars(N, M, X, Y).
%
subst_numbervars(0, _, _, _):-!.
subst_numbervars(J, M, X, Y):- I is J - 1,
	arg(J, X, A),
	arg(J, Y, B),
	subst_numbervars(M, A, B),
	subst_numbervars(I, M, X, Y).

EDIT: A much shorter version using copy_term/2 instead of numbervars, which I think, almost the same as the current foreach, provided that “source” and “target” variables
are explicitly given. I am not sure that this assumption could be dropped.

% ?- toy_foreach1(between(1,5,J), member(J, L), [J], [L]).
%@ L = [1, 2, 3, 4, 5|_] .

toy_foreach1(A, B, Vs, Ws):-
	findall(Vs, A, Sol),
	copy_of_goal(Sol, Vs, Ws, B, Cs),
	maplist(call, Cs).
%
copy_of_goal([], _, _, _, []).
copy_of_goal([A|As], Vs, Ws, B, [G|Gs]):-
	copy_term(Vs+Ws+B, A+Ws+G),
	copy_of_goal(As, Vs, Ws, B, Gs).

EDIT. Using closures of package pac, copy_of_goal clauses avove are included in a maplist call:

% ?- yaforeach(between(1,5,J), member(J, L), [J], [L]).
%@ L = [1, 2, 3, 4, 5|_] .

yaforeach(A, B, Vs, Ws):-
	findall(Vs, A, Sols),
	maplist(pred([Vs, Ws, B],
				 ([A]:- copy_term(Vs+Ws+B, A+Ws+G), call(G))),
			 Sols).


% ?- listing(yaforeach).
%@ yaforeach(A, B, C, D) :-
%@     findall(C, A, E),
%@     '__aux_maplist/2_math:pac#96+3'(E, C, D, B).
%@ 
%@ true.

% ?- listing('__aux_maplist/2_math:pac#96+3').
%@ '__aux_maplist/2_math:pac#96+3'([], _, _, _).
%@ '__aux_maplist/2_math:pac#96+3'([A|B], C, D, E) :-
%@     'pac#96'(C, D, E, A),
%@     '__aux_maplist/2_math:pac#96+3'(B, C, D, E).
%@ 
%@ true.

% ?- listing('pac#96').
%@ 'pac#96'(A, B, C, D) :-
%@     copy_term(A+B+C, D+B+E),
%@     call(E).
%@ 
%@ true.

EDIT: foreach_by_assert/4 is a test version of foreach/2
without using findall/3 and copy_term/2, but using assert/1. It is not beautiful because of using assert, but I have not strong opinion against using assert. Anyway it is a test code.

% ?- foreach_by_assert(between(1,5,J), member(J, L), [J], [L]).
%@ L = [1, 2, 3, 4, 5|_].

foreach_by_assert(Gen, Con, Vs, Ws):-
	retractall('$consume'(_,_)),
	assert(( '$consume'(Vs, Ws):- Con)),
	Stash = '$STASH'(Ws),
	(	call(Gen),
		arg(1, Stash, U),
		once('$consume'(Vs, U)),
		nb_setarg(1, Stash, U),
		fail
	;   arg(1, Stash, Ws)
	).

EDIT: A benchmark test shows that foreach by copy_term is faster than that by assert. I expected the opposite.

% ?- N=100, K=1000, nopac(time(repeat(N, once(foreach_by_copy(between(1,K,J), member(J, L), [J], [L]))))).
%@ % 50,751,501 inferences, 2.570 CPU in 2.575 seconds (100% CPU, 19748888 Lips)
%@ N = 100,
%@ K = 1000.

% ?- N=100, K=1000, nopac(time(repeat(N, once(foreach_by_assert(between(1,K,J), member(J, L), [J], [L]))))).
%@ % 50,450,626 inferences, 3.569 CPU in 3.579 seconds (100% CPU, 14136215 Lips)
%@ N = 100,
%@ K = 1000.

EDIT: This is a complete final version of foreach/2 of mine, though I am not sure yet that I understand the foreach requirement. I understand simply foreach(P, Q) logically as
exist Ys forall Xs (P → Q), where Ys = (VQ + VP) \ VP (set subtraction), Xs = (VP+VQ)\Ys (+ set union), Xs and Ys are disjoint, where VP is the set of variables of P and VQ is the set of variables of Q. I am not sure this logical interpretation is correct, also I forget for now further questions, for instance, nested foreach.

?- foreach(between(1,5,N),  member(r(N),L)).
 L = [r(1), r(2), r(3), r(4), r(5)|_].

exists M exists L forall J  (between(1,5, J) -> (member(J, L), member(J, M))).
?- foreach(between(1,5, J), (member(J, L), member(J, M))).
 L = [1, 2, 3, 4, 5|_],
 M = [1, 2, 3, 4, 5|_].
exists L forall J  (between(1,5, J), between(3, 7, J)  -> member(J, L))
?- foreach((between(1,5, J), between(3, 7, J)), member(J, L)).
 L = [3, 4, 5|_].

exists L forall J  (between(1,5, J); between(3, 7, J))  -> member(J, L))
?- foreach((between(1,5, J); between(3, 7, J)), member(J, L)).
 L = [1, 2, 3, 4, 5, 6, 7|_].

%
foreach(Gen, Con):-	term_variables(Gen, Vs),
	sort(Vs, Vs0),
	term_variables(Con, Ws),
	sort(Ws, Ws0),
	ord_subtr_var(Ws0, Vs0, Ws1),
	once(foreach_by_copy(Gen, Con, Vs0, Ws1)).
%
foreach(Gen, Con, Vs, Ws):-	once(foreach_by_copy(Gen, Con, Vs, Ws)).

% ?- N=200, K = 5, time((repeat(N, once((foreach_by_copy(between(1,5, J), member(J, L), [J], [L]), writeln(L)))))).
foreach_by_copy(A, B, Vs, Ws):-
	findall(Vs, A, Sol),
	copy_of_goal(Sol, Vs, Ws, B, Cs),
	maplist(call, Cs).
%
copy_of_goal([], _, _, _, []).
copy_of_goal([A|As], Vs, Ws, B, [G|Gs]):-
	copy_term(Vs+Ws+B, A+Ws+G),
	copy_of_goal(As, Vs, Ws, B, Gs).
%
ord_subtr_var([], _, []):-!.
ord_subtr_var(X, [], X):-!.
ord_subtr_var([X|Xs], [Y|Ys], Zs):- X==Y, !,
	ord_subtr_var(Xs, Ys, Zs).
ord_subtr_var([X|Xs], [Y|Ys], [X|Zs]):- X@<Y, !,
	ord_subtr_var(Xs, [Y|Ys], Zs).
ord_subtr_var(Xs, [_|Ys], Zs):- ord_subtr_var(Xs, Ys, Zs).

Hi Jan,

As SWI’s foreach/2 was so fresh for me, I wrote myself my_foreach/2 but almost exactly after your tutorial codes of the foreach/2 using copy_term/2. I checked that both works for nested calls. However the current version my_foreach does not take care of scope variables seriously.

As for time test, foreach is faster than my_foreach. But, on testing, I noticed that once/1 is needed to call foreach. Otherwise I got “stack limit error.” Is it a bug or a limitation ? Unfortunately, logical specification of foreach/2 is unclear for me, but it must be clear e.g. viewing from the first-order logics. I would like to keep loose interests in foreach because it smells logic in other way than forall.

% ?- time(repeat(10^5, my_foreach(between(1, 5, J), (member(J, L),
%	my_foreach(between(6, 8, K), (member(K, L),
%	my_foreach(between(9, 10, M), member(M, L)))))))).
%@ % 99,800,001 inferences, 13.637 CPU in 14.883 seconds (92% CPU, 7318499 Lips)
%@ true.

% ?- time(repeat(10^5, once(foreach(between(1, 5, J), (member(J, L),
%	foreach(between(6, 8, K), (member(K, L),
%	foreach(between(9, 10, M), member(M, L))))))))).
%@ % 94,000,001 inferences, 9.676 CPU in 10.904 seconds (89% CPU, 9714508 Lips)
%@ true.

% ?- time(repeat(10, foreach(between(1, 5, J), member(J, L)))).
%@ % 153,355,477 inferences, 8.503 CPU in 8.897 seconds (96% CPU, 18035570 Lips)
%@ ERROR: Stack limit (1.0Gb) exceeded
%@ ERROR:   Stack sizes: local: 3Kb, global: 0.9Gb, trail: 0Kb
%@ ERROR:   Stack depth: 38,338,877, last-call: 100%, Choice points: 12
%@ ERROR:   In:
%@ ERROR:     [38,338,877] lists:member_(_870, 5, _874)
%@ ERROR:     [20] aggregate:prove_list('<garbage_collected>', <compound v/1>, <compound (:)/2>)
%@ ERROR:     [19] aggregate:prove_list('<garbage_collected>', <compound v/1>, <compound (:)/2>)
%@ ERROR:     [18] aggregate:prove_list('<garbage_collected>', <compound v/1>, <compound (:)/2>)
%@ ERROR:     [17] aggregate:prove_list('<garbage_collected>', <compound v/1>, <compound (:)/2>)
%@ ERROR:
%@ ERROR: Use the --stack_limit=size[KMG] command line option or
%@ ERROR: ?- set_prolog_flag(stack_limit, 2_147_483_648). to double the limit.
%@ ^  Exception: (12) setup_call_catcher_cleanup(system:true, prolog_statistics:catch(meta:(between(1, 10, _46), foreach(between(1, 5, _18), member(_18, _20)), false;true), _148,  (report(time{cpu:72.64252, inferences:1121869314, wall:1643463206.004699}, 10), throw(_148))), _5396, prolog_statistics:(_170=true)) ? creep
%@    Call: (14) _170=true ? abort
%@ % Execution Aborted

% ?- repeat(20, write(.)).
% ?- repeat(between(1, 20, I), writeln(I)).
:- meta_predicate repeat(?, 0).
repeat(N, G):- integer(N), !, (between(1, N, _), G, fail; true).
repeat(P, G):- (call(P), call(G), fail) ; true.

my_foreach(Gen, Con):-	term_variables(Gen, Vs),
	sort(Vs, Vs0),
	term_variables(Con, Ws),
	sort(Ws, Ws0),
	ord_subtr_var(Ws0, Vs0, Ws1),
	once(foreach_by_copy(Gen, Con, Vs0, Ws1)).
%
foreach(Gen, Con, Vs, Ws):-	once(foreach_by_copy(Gen, Con, Vs, Ws)).

% ?- N=200, K = 5, time((repeat(N, once((foreach_by_copy(between(1,5, J), member(J, L), [J], [L]), writeln(L)))))).
foreach_by_copy(A, B, Vs, Ws):-
	findall(Vs, A, Sol),
	copy_of_goal(Sol, Vs, Ws, B, Cs),
	maplist(call, Cs).
%
copy_of_goal([], _, _, _, []).
copy_of_goal([A|As], Vs, Ws, B, [G|Gs]):-
	copy_term(Vs+Ws+B, A+Ws+G),
	copy_of_goal(As, Vs, Ws, B, Gs).
%
ord_subtr_var([], _, []):-!.
ord_subtr_var(X, [], X):-!.
ord_subtr_var([X|Xs], [Y|Ys], Zs):- X==Y, !,
	ord_subtr_var(Xs, Ys, Zs).
ord_subtr_var([X|Xs], [Y|Ys], [X|Zs]):- X@<Y, !,
	ord_subtr_var(Xs, [Y|Ys], Zs).
ord_subtr_var(Xs, [_|Ys], Zs):- ord_subtr_var(Xs, Ys, Zs).

Not killing the choicepoint of the 2nd argument is part of the original SICStus spec. So, yes, if you backtrack into the member/2 call with a partial list as second argument you’ll run out of stack.