English for Redis Streams

Redis Streams give you an append-only log with consumer groups similar to Kafka, but the terminology overlaps just enough with Redis’s other data structures and with Kafka’s vocabulary to cause real confusion in design discussions. This guide sorts out the terms.

Key Vocabulary

Stream — an append-only log data type in Redis (XADD, XREAD) where each entry gets a unique, ordered ID, distinct from a pub/sub channel because entries persist and can be replayed. “Switch this from pub/sub to a stream — pub/sub messages are lost if no subscriber is listening, but a stream keeps the entry around for consumers that connect later.”

Entry ID — the unique identifier Redis assigns to each stream entry, formatted as <timestamp>-<sequence>, used for ordering and for resuming consumption from a specific point. “We’re storing the last processed entry ID in our own checkpoint table so a restart can resume from XREAD at the correct position.”

Consumer group — a named group of consumers that share the work of reading from a stream, with Redis tracking which entries each consumer has been delivered via XREADGROUP. “Both worker instances belong to the same consumer group, so Redis splits the incoming entries between them instead of delivering each one twice.”

Pending entries list (PEL) — the per-consumer-group record of entries that were delivered but not yet acknowledged, used to detect and reprocess messages a consumer failed to complete. “The PEL is growing because our consumer is crashing before calling XACK — those entries are stuck as delivered-but-unacknowledged.”

XACK — the command a consumer calls after successfully processing an entry, removing it from the pending entries list for that consumer group. “Make sure XACK only runs after the database write actually commits — acking before that risks losing the entry if the write fails.”

XCLAIM / consumer failover — the mechanism for transferring ownership of a stale pending entry (one un-acked for too long) from a presumed-dead consumer to a healthy one. “We run a periodic XCLAIM sweep to pick up entries from consumers that crashed mid-processing, so nothing sits unacknowledged indefinitely.”

Common Phrases

• “Is this a stream or pub/sub — do we actually need the entries to persist for late consumers?”
• “What’s the current size of the pending entries list — are consumers falling behind on acking?”
• “Are we storing the entry ID as a checkpoint, or would a restart lose our place in the stream?”
• “Is this consumer acking before or after the actual side effect completes?”
• “Do we need an XCLAIM sweep for stale pending entries, or are consumers reliable enough that we haven’t needed one yet?”

Example Sentences

Explaining stream vs. pub/sub in a design doc: “We chose a stream over pub/sub specifically because we need at-least-once delivery — a consumer that’s briefly down shouldn’t lose the entries that arrived while it was offline.”

Reporting a stuck consumer group: “The pending entries list has grown to several thousand — it looks like the payment consumer stopped acking a few hours ago even though it’s still connected, which suggests a silent failure in the processing loop.”

Discussing failover design: “We should add an XCLAIM sweep with a reasonable idle-time threshold, so if a consumer dies mid-processing, another instance picks the entry back up instead of it sitting in the PEL forever.”

Professional Tips

• Say stream, not “channel,” when discussing Redis Streams specifically — “channel” implies pub/sub semantics, which don’t persist entries the way a stream does.
• Reference the pending entries list by name when diagnosing a backlog — it’s the concrete, inspectable state that explains whether consumers are keeping up.
• Be precise about when acking happens relative to the actual work — acking too early is a common source of silent data loss on consumer crash.
• Mention consumer group explicitly when describing load distribution, since two consumers reading the same stream without a shared group will each get every entry, not a split workload.

Practice Exercise

1. Explain in one sentence why a stream might be chosen over pub/sub.
2. Write a bug report describing a consumer group with a growing pending entries list.
3. Describe, in your own words, what XCLAIM is used for.