English for Redpanda Developers

Redpanda is Kafka-API-compatible, so much of the streaming vocabulary a team already knows from Kafka carries over directly — but its different architecture (no ZooKeeper, no JVM, a built-in Raft-based consensus) introduces its own terms that matter when discussing operational behavior. This guide covers the English used when discussing Redpanda with a team.

Key Vocabulary

Partition — an ordered, append-only log that a topic is split into, the unit of parallelism for both producing and consuming, and the basis for how Redpanda distributes load across a cluster. “We’re seeing hot-spotting on one broker because this topic only has three partitions but the producer key distribution is skewed — let’s repartition with a better key.”

Consumer group — a set of consumers cooperating to read a topic, with partitions divided among group members so each partition is processed by exactly one consumer within the group at a time. “Adding a fourth consumer to this group won’t help throughput — we only have three partitions, so the fourth consumer will sit idle with nothing assigned to it.”

Raft consensus — the algorithm Redpanda uses natively (instead of relying on an external coordination service like ZooKeeper) to replicate partition data and elect leaders across the cluster. “When the partition leader’s node went down, Raft consensus elected a new leader from the in-sync replicas automatically — no external coordinator was involved.”

Tiered storage — Redpanda’s feature for offloading older log segments to object storage (like S3), letting a topic retain far more historical data than local disk alone could hold, while still being queryable. “Instead of shrinking retention to fit local disk, let’s enable tiered storage — the older segments move to object storage and stay queryable without keeping everything on local NVMe.”

Consumer lag — the difference between the latest offset produced to a partition and the offset a consumer has processed, the primary metric for detecting a consumer falling behind. “Consumer lag on this topic has been climbing steadily for an hour — either the consumer is too slow for the incoming volume, or it’s stuck retrying a poison message.”

Producer acknowledgment (acks) — the setting controlling how many replicas must confirm receipt of a message before the producer considers the write successful, trading durability against latency. “We’re using acks=1 on this critical topic, which only waits for the leader — switching to acks=all means we wait for all in-sync replicas, at the cost of a bit more latency, but we won’t lose data on a leader failure.”

Common Phrases

• “Is this hot-spotting because of the partition count, or the producer’s key distribution?”
• “Are all the consumers in this group actually assigned partitions, or are some sitting idle?”
• “Is consumer lag climbing because of volume, or because the consumer is stuck?”
• “Should we enable tiered storage instead of shrinking retention to fit local disk?”
• “What acks setting is this producer using, and does that match the durability we need?”

Example Sentences

Reviewing a pull request: “This consumer group has five members but the topic only has two partitions — three of these consumers will never be assigned any work, so either add partitions or reduce the group size.”

Explaining a design decision: “We enabled tiered storage on the audit log topic so we can keep a year of history queryable without provisioning enough local NVMe to hold it all.”

Describing an incident: “Consumer lag spiked because a single malformed message kept crashing the consumer on retry — we needed a dead-letter mechanism, not just faster processing.”

Professional Tips

• Say “consumer lag” precisely as the offset gap, not just “the consumer is slow” — it’s the metric the whole team monitors and alerts on.
• When diagnosing hot-spotting, ask “is this a partition count problem or a key distribution problem?” — these have different fixes and are often confused.
• Use “acks” setting explicitly (acks=1 vs acks=all) when discussing durability tradeoffs — vague phrases like “we wait for confirmation” don’t specify which guarantee is actually in place.
• Distinguish “tiered storage” (offloading to object storage while remaining queryable) from simply “reducing retention” (deleting old data) when proposing a fix for disk pressure.

Practice Exercise

1. Explain in two sentences why adding more consumers to a group doesn’t help if there aren’t enough partitions.
2. Write a one-sentence recommendation for reducing hot-spotting caused by skewed producer keys.
3. Describe, in your own words, the tradeoff between acks=1 and acks=all.