What is the key difference between a topic and a queue in messaging?
In classic messaging terms: queues (point-to-point) deliver each message to one consumer; topics (publish-subscribe) deliver each message to all subscribers. Kafka uses 'topic' but consumer groups provide queue-like semantics within a group.
2 / 5
In Kafka, a consumer group works as follows:
Within a consumer group, each partition is consumed by exactly one consumer. This enables parallel processing — scale by adding consumers up to the partition count. Multiple groups can each receive all messages independently.
3 / 5
A message is sent to the dead-letter queue (DLQ) when:
DLQs capture 'poison messages' — messages that fail repeatedly (due to bad format, processing errors, etc.) so they don't block the queue. Ops teams can inspect, replay, or discard DLQ messages after root-cause analysis.
4 / 5
Your system uses at-least-once delivery. This means:
At-least-once guarantees no message loss but allows duplicates (e.g., if acknowledgment fails after processing). Consumers must be idempotent — processing the same message twice must produce the same result as processing it once.
5 / 5
In Kafka, the offset is:
Each message in a Kafka partition has a unique, sequential offset. Consumers commit their offset to record progress — if a consumer restarts, it resumes from the last committed offset, enabling at-least-once or exactly-once processing.