New standard package: a redis client (Discussion)

Discussion
21

Yes, this is because when the message is put in the pending list, the information about the consumer it was delivered to is stored with it; but if the consumer dies, the pending list is still alive (and the messages in it still have the accounting information that it was delivered to the now dead consumer). From the documentation for XCLAIM (see point no.3)

1 There is a stream with an associated consumer group.
2 Some consumer A reads a message via XREADGROUP from a stream, in the context of that consumer group.
3 As a side effect a pending message entry is created in the pending entries list (PEL) of the consumer group: it means the message was delivered to a given consumer, but it was not yet acknowledged via XACK.
4 Then suddenly that consumer fails forever.
5 Other consumers may inspect the list of pending messages, that are stale for quite some time, using the XPENDING command. In order to continue processing such messages, they use XCLAIM to acquire the ownership of the message and continue.

22

By the way, you can easily test that the pending list belongs to the consumer group by doing the following:

  1. Start two sessions of redis-cli
  2. In session 2, create a consumer group, do an XADD
  3. In session 1 (representing the faulty consumer), do an XREADGROUP
  4. Kill session 1 with ^D (this represents a dead consumer)
  5. In session 2, run XPENDING and see the list is alive with one message (even though the original consumer is dead)
  6. In session 2, run XCLAIM and you will be able to get the whole message.
  7. In session 2, run XPENDING and now you will see that the pending consumer has changed from the initial one to the one you specified in XCLAIM

A message is assigned to a consumer either by XREADGROUP or XCLAIM.

EDIT: In the scope of a pending list, I think it is better to think of a consumer as a tag that is attached or changed for messages in the pending list.

EDIT2: I was playing a little bit with the library and found the following:

10 ?- redis(ping,R).
ERROR: redis_server `ping' does not exist
ERROR: In:
ERROR:   [13] throw(error(existence_error(redis_server,ping),_482))
ERROR:   [10] redis:redis(ping,'<garbage_collected>') at /home/u/tmp/swipl-devel/build.release/home/library/redis.pl:336
ERROR:    [9] toplevel_call('<garbage_collected>') at /home/u/tmp/swipl-devel/build.release/home/boot/toplevel.pl:1113
ERROR: 
ERROR: Note: some frames are missing due to last-call optimization.
ERROR: Re-run your program in debug mode (:- debug.) to get more detail.
11 ?- redis(ping(),R).
false.

12 ?- redis(default,ping,R).
R = status("PONG").

Should not redis(ping,R) work the same as redis(default,ping,R)?

23

I think this demo is quite useful! I really think most people trying to use consumer groups will find the same. Thanks for including it!

2 Likes
24

If you read the git log you find

commit b1f743c32a27655bf87226b770155d9f6da7b6af
Author: Jan Wielemaker <J.Wielemaker@vu.nl>
Date:   Mon Oct 12 15:21:57 2020 +0200

    MODIFIED: Renamed redis_cli/1 to redis/1 and changed redis/2 to mean
    redis(Connection, Request, _).  This avoids ambiguity of redis/2.

I found myself using redis/2 trying to give a server and expecting a result too often. It is about the same problem as format/1-3, which makes format/2 ambiguous and I often get it wrong. So, redis/2,3 now always take a server as first argument and redis/1 is a simple replacement for a redis client. Not sure how useful it is. I find myself using redis-cli for quick tests as this does completion and shows the expected arguments.

1 Like
25

Ah, sorry about that, I was reading the docs on the website which are not up to date yet.

26

No. That happens next devel release … Use ?- help(redis). gives the docs for the version you are running. First update to the current head or you will get an exception from help/1 on this request :frowning:

1 Like