Some basic Messaging patterns, howto in Web-Prolog?

[j4n_bur53]

Is there some repository that shows how to implement these basic patterns in Web-Prolog by just using the Erlang idiom, and not more?

1. Competing Consumers:

2. Publish/Subscribe:

3. Routing:

4. Wire Tap (combinable with 1), 2) and 3) for debugging):

[torbjorn.lager]

No, there isn’t. The best (and most important) examples of what can be built on top of spawn/2-3, !/2, receive/1-2 etc. is the pengine, and rpc/2-3 on top of pengines.

Contributions are of course welcome!

[torbjorn.lager]

Yes, although I’m not going to spend time actually programming this myself (I have other aspects I want to concentrate on) I think that something like that might work.

If your queues are lists, for example, you should be able to call

?- spawn(pipe([]), Pid).
Pid = ....

and then start talking to your concurrent queue using either asynchronus communication primitives such as !/2 and receive/2, or choose a synchronous more high-level solution, such as your get/2 – which is defined in terms of !/2 and receive/2. Apart from no sign of respect for concrete syntax, you seem to get it.

@j4n_bur53

Can all this be demonstrated on xxx.prolog.computer?

No, you need a node that offers the ACTOR profile for that. If you haven’t already, you would need to download and install the PoC at https://github.com/Web-Prolog/swi-web-prolog and run the code there.

It’s a nice example, so I would perhaps like to add it to the many other examples in the tutorial that comes with the PoC, if that’s alright with you. If you don’t implement it, I might, but only later.

[torbjorn.lager]

Sure, my guess is that many examples in that book can fairly easily be translated into Web Prolog. After all, Akka is inspired by Erlang. (Jonas Bonér, who created Akka, worked for Ericsson too.)

It’s even easier to translate programs written in Erlang, though, or programs written in Elixir (since Elixir is basically Erlang with a different syntax).

Erlang has something called OTP (stands for Open Telecom Platform), which contains many interesting libraries for making writing fault-tolerant concurrent and distributed applications easier. Some of them might be possible to port to Web Prolog. For others, it’s not clear how useful they will be in a web programming context.

It’s more promising, I think, to first work in the direction of pengines and RPC. I’ve shown it before, but I think a program such as this says a lot about what can be done because receive/1-2 is semi-deterministic:

search(Query, Pid, Options) :-
    self(Self),
    spawn(query(Query, Pid, Self), Pid, [
        monitor(true),
        src_predicates([query/2])
        | Options
    ]).
    
query(Query, Self, Parent) :-
    call_cleanup(Query, Det=true),
    (   var(Det)
    ->  Parent ! success(Self, Query, true),
        receive({
            next -> fail;
            stop -> Parent ! stopped(Self)
        })
    ;   Parent ! success(Self, Query, false)
    ).

Again, we don’t have to build such things ourselves, since we have pengines.

[torbjorn.lager]

@j4n_bur53

It has much more resemblance with I/O select, call it therefore message select:

query(Query, Self, Parent) :-
    call_cleanup(Query, Det=true),
    (   var(Det)
    ->  Parent ! success(Self, Query, true),
        message_select([next,stop], I),   /* gives index of selected message, 
                              starting with 1, might block, 0 means 
                              timeout if timeout limit was specified */
         (I==1 -> fail;
          I==2 -> Parent ! stopped(Self))
    ;   Parent ! success(Self, Query, false)
    ).

First, let me point out that Erlang’s receive primitive is often referred to as a selective receive. (Googling for ’ “selective receive” Erlang’ finds almost 2000 hits.)

I see at least three issues with your proposal:

1. What about guards? Where do they come in? If you’re serious here, you owe me an implementation of important/1 using your message_select/2, i.e. what we discussed in this topic.

2. And in what way is your proposal any better than using thread_get_message/1, like so?:

    query(Query, Self, Parent) :-
        call_cleanup(Query, Det=true),
        (   var(Det)
        ->  Parent ! success(Self, Query, true),
            thread_get_message(Message),
            (Message==next -> fail;
             Message==stop -> Parent ! stopped(Self))
        ;   Parent ! success(Self, Query, false)
        ).

3. And what happens if someone sends a message foo to the process, does is crash? Or is the message deferred? Using thread_get_message/1 as above would not defer foo, but since the whole if-then fails, it would have the same effect as sending next. (There are ways around that but you have to change the form of your messages.) What about your solution?

[torbjorn.lager]

@j4n_bur53

I didn’t make any proposal concerning receive here, thats off topic

Since you plugged out my receive from the example and plugged in your own construct, I got the impression you were, but never mind.

@j4n_bur53

And it should not be so difficult to follow my argument by supplying a pairlist of patterns and guards to message_select/2 to see that it is deterministic and blocking even if there are when guards.

Ok, and if you also specify how it works with respect to the deferring of messages, you’d have something that might be equivalent to Erlang’s receive, right? But then why not go with something that basically is Erlang’s receive, except that it also has a couple of extra properties such as more expressive guards and a semi-deterministic behaviour?

A predicate is said to be semi-deterministic if it either fails, or succeeds exactly once. That’s true of read/1 for example:

?- read(X).
|: a.
X = a.

?- read(a).
|: b.
false.

Now, calling receive/1-2 either fails, or succeeds exactly once, so it is semi-deterministic. You can of course choose to implement something like this using receive/1:

message_select([Alt1, Alt2], I) :-
    receive({
        Alt1 -> I = 1;
        Alt2 -> I = 2
    }).

and then call it like so:

    ...
    message_select([next, stop], I),
    (I==1 -> fail;
     I==2 -> Parent ! stopped(Self)),
    ...

But why would you? Just to be able to say that your receive construct is deterministic?

And again, what if foo comes along? The receive in the definition of message_select/2 will defer it, but what should be the value of I then, and what happens in the if-then construct?

It seems to me that the if-then construct must somehow be moved into message_select/2, and yeah, wait for it … that’s what Erlang’s and Web Prolog’s receive does!

But, with the if-then construct integrated in that way, what should happen if the body of one of the receive clauses fails? We have to account for this possibility. Of course, you could imagine throwing an error at that point, but I think it’s much more useful if the whole call to receive/1-2 is allowed to fail. That’s the only way it can fail, and that’s why it must be deemed semi-deterministic.

@j4n_bur53

I want to use receive and actors for messaging patterns. I indicated that I want to use receive and actors unchanged, in that I said in my original post “by just using the Erlang idiom”

Good move!

@j4n_bur53

Anybody up to this challenge?

Sure, here’s an actor implementing one of the patterns you asked for - a simple publish-subscribe service:

pubsub_service(Subscribers0) :-
    receive({
        publish(Message) ->
            forall(member(Pid, Subscribers0), Pid ! msg(Message)),
            pubsub_service(Subscribers0);
        subscribe(Pid) ->
            pubsub_service([Pid|Subscribers0]);
        unsubscribe(Pid) ->
            (   select(Pid, Subscribers0, Subscribers)
            ->  pubsub_service(Subscribers)
            ;   pubsub_service(Subscribers0)
            ).    
    }).

It you are the owner of a node, you can spawn this actor, register it under a mnemonic name, and make it available on the Web:

:- spawn(pubsub_service([]), Pid),
   register(pubsub_service, Pid).

Here’s how to subscribe to the service, and invoke a repeat-fail loop waiting for messages to arrive from it:

?- self(Self),
   pubsub_service ! subscribe(Self),
   repeat,
   io:write("Waiting for a message ..."),
   receive({
       msg(Message) ->
           io:format("Received: ~p", [Message]),
           fail
   }).

Here’s how to publish a message:

?- pubsub_service ! publish(hello).

And no, it won’t work against an ISOBASE node - ACTOR capabilities are required.

This example is from the Web Prolog tutorial. If you haven’t already, you would need to download and install the PoC available at https://github.com/Web-Prolog/swi-web-prolog and run the code there.

[torbjorn.lager]

@j4n_bur53

Or is use_module/1 not part of ISOBASE?

Since an ISOBASE node is designed for stateless querying only, it’s very unlikely that it will offer use_module/1. An ISOTOPE node might offer it (or something comparable), and the same goes for an ACTOR node.

@j4n_bur53

How would you wire tap your pub sub?

What do you mean by “wire tap”? The clients talk to the pub-sub server over websockets, if that’s what you mean. The same way as a shell written in JavaScript talks to a pengine, and the same way as a pengine talks to another pengine running on a remote node. That’s why it needs an ACTOR node.

@j4n_bur53

An example product that is reusable is RabbitMQ, which was prototyped in Erlang.

You shouldn’t compare a simple demo in a tutorial with something like that.

[torbjorn.lager]

Great! Hope you can build something that can run the prio queue example, since this seems to be a nice acid test for what we may want to require of a construct like this.

[torbjorn.lager]

Looks good! Now if you would use the Erlang-ish syntax too, you would have the beginning of an implementation of Web Prolog.

[torbjorn.lager]

@j4n_bur53

But the adoption of spawn from Erlang/Elixir that returns a Pid should be scrutinized. It could be more efficient to have simply a run that doesn’t return a Pid. Similar idea has the “go” command in the go programming language.

Then it wouldn’t be Web Prolog. Doesn’t seem wise to miss the chance, in my opinion, to have a syntax and semantics which is close to Erlang – something that an Erlang programmer would understand right away. I’m not familiar with Go, but I doubt it has a syntax derived from Prolog.

[torbjorn.lager]

@j4n_bur53

Well its still Web Prolog, it has spawn, its only bootstrapped now:

Yes, that looks like something I might to in my implementation too.

@j4n_bur53

How do you bootstrap spawn from pengine_spawn?

You mean the other way around – how pengine_spawn/1-2 is defined in terms of spawn/2-3?

[torbjorn.lager]

In Web Prolog, pengine_spawn/1-2 is defined in terms of spawn/2-3. A pengine is seen as a special case of an actor – an actor that comes with a built-in protocol.

Have a look at this:

simple_pengine(Query, Pid, Options) :-
    self(Self),
    spawn(query(Query, Pid, Self), Pid, [
        monitor(true),
        src_predicates([query/2])
        | Options
    ]).
    
query(Query, Self, Parent) :-
    call_cleanup(Query, Det=true),
    (   var(Det)
    ->  Parent ! success(Self, Query, true),
        receive({
            next -> fail;
            stop -> Parent ! stopped(Self)
        })
    ;   Parent ! success(Self, Query, false)
    ).

Of course, a “real” pengine is more complicated.

[torbjorn.lager]

@j4n_bur53

Also rpc/2 can be bootstrapped, but you make it a primitive

Yes, since it can be shown that you can have rpc/2-3 in a profile as weak as ISOBASE, I don’t want to require that it be bootstrapped in that way. Then I also show that it can be defined in terms of a pengine, or in terms of an HTTP request to the stateless API. But an implementer of rpc/2-3 in an ISOBASE node could implement it anyway s/he want, as long as it behaves correctly.