Dealing with state

This is unfortunately not done very nicely in the current implementation. As pred_info/3 is stored globally in the edcg module, it applies to all modules. That is probably not what we want. We should change this to be proper directives that manage the declarations inside the module being compiled. Given that, you can write an edcg based parser in a module, assuming all predicates that are part of it use the same set of accumulators.

Still, you probably want helpers that need no or fewer accumulators. Not sure how to deal with that.

There is also an unfortunate aspect of the SWI-Prolog VM that passes arguments over the stack rather than using registers. Adding more arguments is often more expensive than it is in register based VMs. As a result, it may quickly be the case that the dict based approach gets faster if the number of state variables grows.

Yet an alternative approach is not to get rid of the all extra variables, but just of the accumulator pairs and the nuisance of numbering the variables as you go through the clause. For example (disregard the syntax details):

sumlist(#List, #Sum) :-
    c(C, List),
    add(C, Sum),
    sumlist(List, Sum).

This implies that every head variable marked #Var is expanded to twp variables (Var0, Var) and every subsequent occurrence of this variable is translated into threading pair Var0, Var1, Var1, Var2, … except for the last which is translated into VarN, Var. So, the final clause is

sumlist(List0, List, Sum0, Sum) :-
    c(C, List0, List1),
    add(C, Sum0, Sum1),
    sumlist(List1, List, Sum1, Sum).

So the question that remains is:

What syntax should we use when using a compound state approach ?

We need syntax for getting/setting/updating/adding/deleting a specific part of the compound state using one/multiple key/keys

So, here is a proposition for this:

  • getting: state(o(Key, Value)), Key should already exist
  • setting: state(+(Key, Value)),
  • updating: state(-(Key, OldValue, NewValue)), Key should already exist
  • adding: state(add(Key, Value)), Key should not exist
  • deleting: state(del(Key, Value)), Key should already exist

Here are convenience predicate:

  • getting/setting/updating/adding/deleting multiple states: states([o(Key, Value), +(Key, Value), ...])
  • calling a non-dcg predicate with some states added as additional parameters: statep(Goal, [o(Key, Value)]). This is a shorthand for state(o(Key, Value)), call(Goal, Value)

Does this syntax appeal to anyone ?

We could provide multiple implementation, one with dict as a backend, one with rbtrees, one with records, etc ?

I believe this is Mercury solution to this problem.
Has anyone ever implemented this in pure prolog ?
Won’t this need a custom neck with term_expansion in order to implement ?

Thanks for the Mercury pointer. I know some people did something similar for their own use. And no, you do not need a custom neck if you reserve the special term. Now if I recall correctly, defining ! as a prefix operator is somewhat dubious in pure Prolog. Not sure how serious that is.

Having a reserved term is somewhat problematic if you happen to need that term. The way around is typically to use e.g., =../2, so code that requires a match against !(X) has to do Term =.. [!,X]. That currently also holds for SWI-Prolog’s ‘.’ for functional notation. In practice, this seems manageable as long as you use a term that is rarely seen in normal Prolog source code.

edit for reference: The Mercury Language Reference Manual: State variables

Did a little experiment. See attached file below. With this, we can write the code below to end up with the code you would write by hand as well. Now, this is also not a good example.

sv_sumlist(List, Sum) :-
    sv_sumlist(List, 0, Sum).

sv_sumlist([], !_).
sv_sumlist([H|T], !Sum) :-
    !:Sum is H+!.Sum,
    sv_sumlist(T, !Sum).

There surely are good examples as threading state variables in the body is a very common thing to do. Below is an example reading terms from a file and performing some selection and transformation on them. Note that we introduce the state variable in the head using !:S to get access to the final state. Next we we use !:S again to introduce a fresh variable (a successor without a current). From there we do the state transitions.

tf(File, !:S) :-
    read_file_to_terms(File, !:S, []),
    exclude(junk, !S),
    maplist(transform, !S).

Again, the question is how much sense does this make? The fact that Mercury went for it is an encouragement. To complete it we must notably add IDE support as the highlighting and GUI tracer get pretty confused. Program analysis runs on the transformed code, so that keeps working fine.

Not well tested …

state_vars.pl (8.2 KB)

I am not surprised that Jan W. can write such code in a blink.
But where is a person with age < 30 years that can do the same?

Long time friends, I guess… :joy: but I mean: everyone in here expects Jan to be the master, SWI-Prolog is in large part his creature (a little bit like Frankenstein’s), so I think it would be difficult to imagine a different outcome

It is hard to find people < 30 years old who have > 30 years of programming experience.

3 Likes

How old do you need to replicate this code here:

pred expand_bang_state_pairs_in_terms
https://\github.com/Mercury-Language/mercury/blob/92f02d474be2500031602bbf3bae9001e7e8a50f/compiler/state_var.m

But I don’t know whether Jan W. take does exactly the same as
Mercury. It says its Marcury state variables, it even adopted !.
and !: syntax. Lets make a test. This here:

foo(!IO) :-
    bar(!IO),
    baz(!IO).

Should do the same as this here, according to this tutorial:

foo(!.IO, !:IO) :-
    bar(!.IO, !:IO),
    baz(!.IO, !:IO).

Yep, it shows me in both cases:

?- listing(foo).
foo(A, B) :-
    bar(A, C),
    baz(C, B).

Edit 03.10.2023
The more difficult test case could be disjunction. What are
the expectations. What are the results in SWI-Prolog
and/or Mercury?

This translation is possibly wrong:

foo(!S1,!S2) :- bar(!S1); baz(!S2).

With SWI-Prolog prototype it gives me, it does alias variables:

foo(A, A, B, B) :-
    (   bar(A, A)
    ;   baz(B, B)
    ).

But expectation would be rather:

foo(A, B, C, D) :-
    (   bar(A, B), C = D
    ;   baz(C, D), A = B
    ).

I guess I should download Mercury, and see what it does.

Indeed looks wrong. It isn’t well tested. I only looked at disjunctions operating on the same state variable. Disjunctions is probably the hardest part of the code. Well, I haven’t looked into interaction with meta_predicate/1 declarations either.

The question is first of all whether it makes sense to support this part of Mercury? Maybe it is better to improve edcgs (e.g., module awareness and maybe some more to reduce the need for declarations)?

A middle ground instead of adopting Mercury, would be to adopt Picat,
since SWI-Prolog already adopts SSU (=>)/2. Picat has a couple of
papers that details how they translate their state handling.

It has less syntactic bagagge, its just the (:=)/2 and using
the same variable name again. But you have to use unification at
the end of a predicate to return a value:

foo(X, Y, R, S) =>
   (X := X+1; Y := Y+1), R = X, S = Y.

How does Picat translate the above?

Edit 03.10.2023
Or a hybrid that combines the best of both worlds Picat and
Mercury. Picat borrows from imperative languages that there exist
“assignment contexts” and “evaluation contexts”. The left hand

side of their (:=)/2 is an assignment context, so that there is
no need for the verbose Mercury (!:). And then right hand side
of (:=)/2 is evaluation context so there is no need for verbose

Mercury (!.). Even Haskell knows about these contexts, in their do
notation for monadic computations that is used to arrive at
pseudo-imperativity. Example from Haskell:

mothersPaternalGrandfather s = 
     do m <- mother s
        gf <- father m
        father gf

If you use the same name multiple times in (<-) what happens?
But I don’t know exactly how such a hybrid would look like.
Something is there in Picat that is missing in Marcury,

and something is missing in Mercury that is there in Picat.
What is missing in Picat would be an assignment context in
a predicate rule head or a predicate goal invokation,

which would then amount what we know from imperative
languages, an in out parameter.

What I didn’t test are Mercury states in the query. If
we have a query for example:

?- foo(!IO), bar(!IO), baz(!IO).

Are we supposed to show all 6 variables as answer
substitutions. We could also only show the last state.

Here is what a Picat & Mercury hybrid could do,
showing also what a query would do:

foo(!X, !Y) =>
   (X := X+1; Y := Y+1).

?- X = 0, Y = 0, foo(!X, !Y).
X = 1, Y = 0;
X = 0, Y = 1.

Interesting (note that the link is wrong). I guess it mostly works well for Python that, if I recall correctly, by default using functional notation, so

X := foo(1, X)

Translates to Mercury-style

foo(1, !.X, !:X) 

or simply

foo(1, !X).

Without using functional notation it gets less nice as we would get

foo(1, X, Y),
X := Y

right? Well, of course we could translate X := foo(1,X) into call(foo(1,!.X), !:X) and do something special for arithmetic? That leads to some ambiguity as e.g. the function sin(X) has in SWI-Prolog no relation to a possible predicate sin/2, so what would X := sin(X) do?

The sum_list example that I made had an evaluable function which is
predicate defined. But the foo/2 example with X := X+1 and Y := Y+1
doesn’t use some predicate defined evaluable function.

You have to check with Picat Manual. It has a section 1.3 Defining Functions,
But this was never my point, to introduce this feature of Picat. Because
you don’t need it to translate X := X+1. It is tempting to translate it into:

+(!.X, 1, !:X)

But you can also translate X := X+1 into:

!:X is !.X+1

Basically in Picat, every variable is already automatically a state
variable. There are papers of Picat that detail the translation
used in Picat. Basically Picat translates to BProlog. Example paper:

Canonicalizing High-Level Constructs in Picat
2.4 Assignments and While Loops
https://www.sci.brooklyn.cuny.edu/~zhou/papers/padl17.pdf

The approach is relatively straight forward, every variable is automatically
a state variable, but its not automatically a pair. It gets a pair during
assignment which is described by Neng-Fa Zhou and Jonathan Fruhman:

Canonicalizing High-Level Constructs in Picat
3.5 Transformation of Assignments
Picat creates a new variable, say X1, to hold the value of X after
the assignment X := X + 1. […] All occurrences of X after the assignment
are replaced by X1. […] When encountering X1 := X1 + 2, Picat creates
another new variable. Etc…
https://www.sci.brooklyn.cuny.edu/~zhou/papers/padl17.pdf

1 Like

I’m not sure that a logo like this might appeal to the mathematician or to the logician or to the academic. You should choose something that looks more intimidating and reserved exclusively to the very smart

You’re right. A mascot is not what I had in mind. I was thinking of logos more in general. And in that case I had in mind a thing (not a mascot) like a lambda symbol that you can find in many logos of Lisp-related languages for example

Here is my attempt at implementing the syntax I proposed above using the goal_expansion technique from Jan.
So, from this dcg predicate:

q -->
   state(o(key, value)).

this will expand to:

?- set_setting(backend, rbtrees).
true.

?- listing(q).
q(A, A) :-
    (   rb_lookup(key, value, A)
    ->  true
    ;   existence_error(key, key, A)
    ).

?- set_setting(backend, dict).

?- make.

?- listing(q).
q(A, A) :-
    (   get_dict(key, A, value)
    ->  true
    ;   existence_error(key, value, A)
    ).

?- set_setting(backend, record).

?- make.

?- listing(q).
q(A, A) :-
    state_data(key, A, value).

setting/updating one or multiple states and calling non-dcg predicates with appended states also works.

Also not very well tested :slight_smile:

state.pl (4.0 KB)

You can use the set or update syntax since when dealing with states instead of lists, push-back is just a way of stepping the state:

state(+(key, NewValue)).  % set syntax
state(-(key, OldValue, NewValue)).  % update syntax

You just need to make sure that the update is the last update in your predicate.

An alternative is to modify @jan’s goal expansion as follows – this gave me ~40% performance improvement for my “deadfish” code (see Autum Challenge: Short Deadfish Numbers). Possibly this wouldn’t be needed if get_dict/5 can be compiled to inline code (the overhead seems to be mainly in the call to the predicate; the code below gains most of its performance improvement by doing the “record” expansion inline, as if library(record) generated macros for expanding its predicates when used as goals).

Modified goal expansion
goal_expansion(<(On, Name,State0,State), Clause) :-
    expand_record(On, Name, State0, State, Clause).

% expand_record/2 uses record expansion instead of a dict, for faster
% performance. There must be a dcg_record_name/1 fact and an
% appropriate `:- record` directive.
expand_record(Literal,Name,State0,State, Clause), is_list(Literal) =>
    get_set_record(Name, State0, List, State, Tail),
    Clause = true,
    append(Literal, Tail, List).
expand_record(String,Name,State0,State, Clause), string(String) =>
    get_set_record(Name, State0, List, State, Tail),
    Clause = true,
    string_codes(String, Literal),
    append(Literal, Tail, List).
expand_record(Step,Name,State0,State, Clause), callable(Step) =>
    extend_goal(Step, [V0,V], StepEx),
    get_set_record(Name, State0, V0, State, V),
    Clause = ( % Clause0,
               StepEx
             ).

% get_set_record/5 is a bit more complicated because
% library(record) doesn't generate the equivalent of:
%   get_set_<name>_of_<constructor>(Rec, V, NewRec, NewV) :-
%       <constructor>_<name>(Rec, V),
%       set_<name>_of_<constructor>(NewV, Rec, NewRec).

get_set_record(Name, Rec, V0, NewRec, V) :-
    dcg_record_name(RecName),
    call_univ([RecName, '_', Name], [Rec, V0]),
    call_univ([set_, Name, '_of_', RecName], [V, Rec, NewRec]).

call_univ(PredParts, Univ) :-
    concat_atom(PredParts, Pred),
    Call =.. [Pred|Univ],
    call(Call).
Example of using record-expansion DCG
:- use_module(library(record)).

dcg_record_name(df).

:- record df( % must match dcg_record_name/1
              acc,              % accumulator
              ops,              % list of opcodes
              out,              % result of running the opcodes,
              num, % the number to be output (for limiting search space)
              nsq  % see deadfish/3 (for limiting search space)
            ).

deadfish(Num, OpsLen, Ops) :-
    number_digits(Num, Ds),
    NumSqLimit is floor(sqrt(Num + 1)) + 1, % probably can be less
    % State0 = #{acc:0, ops:Ops, out:Ds, num:Num, nsq:NumSqLimit},
    % State  = #{acc:_, ops:[],  out:[], num:Num, nsq:NumSqLimit},
    make_df([acc(0), ops(Ops), out(Ds), num(Num), nsq(NumSqLimit)], State0),
    make_df([acc(_), ops([]),  out([]), num(Num), nsq(NumSqLimit)], State),
    call_dcg(seq_of_len(OpsLen, deadfish_eval), State0, State).

seq_of_len(0, _) --> [].
seq_of_len(Len, P) -->
    { Len > 0 },
    call(P),
    { Len2 is Len - 1 },
    seq_of_len(Len2, P).

% deadfish_eval//2 computes a sequence of opcodes (in `ops`), the
% accumulator (in `apps`), and the resulting output (in `out`).
% For pruning the searchspace, there are `num` and `nsq`.

deadfish_eval -->
    [s]<ops,
    value(NumSqLimit)<nsq,
    square(NumSqLimit)<acc.
deadfish_eval -->
    [i]<ops,
    incr<acc.
deadfish_eval -->
    [d]<ops,
    decr<acc.
deadfish_eval -->
    [o]<ops,
    value(Acc)<acc,
    number_digits(Acc).

% The various predicates used by deadfish_eval//0:

value(V, V, V).

incr(X0, X) :- X is X0 + 1 .

decr(X0, X) :- X0 > 0, X is X0 - 1 .

square(X0, X) :- X0 > 1, X is X0*X0 .
square(NumSqLimit, X0, X) :- X0 > 1, X0 =< NumSqLimit, X is X0*X0.

number_digits(Number) -->
    { Number =< 9 }, !,
    [Number]<out.
number_digits(Number) -->
    { divmod(Number, 10, Number2, D) },
    number_digits(Number2),
    [D]<out.

output_digits([]) --> [].
output_digits([D|Ds]) -->
    [D]<out,
    output_digits(Ds).

% number_digits/2 is a convenience wrapper for number_digits//1.

number_digits(Number, Digits) :-
    make_df([out(Digits)], S0),
    make_df([out([])], S),
    % S0 = #{out:Digits},
    % S  = #{out:[]},
    call_dcg(number_digits(Number), S0, S).
1 Like