Why do at-least-once systems produce duplicate messages in the first place?

A duplicate happens whenever the sender cannot distinguish "the message was lost" from "the acknowledgement was lost." If a consumer processes a message but crashes before sending the acknowledgement, the producer times out and retries — delivering the same message again even though it was actually already handled once.

Is deduplication the same as exactly-once?

Deduplication is one common technique used to achieve exactly-once-like behaviour on top of an at-least-once transport. The consumer tracks message IDs it has already processed (often in a small window or a persistent store) and discards repeats. It is not automatic — someone has to design and implement the dedup logic; it doesn't come for free just because the transport retries.

Which delivery guarantee should webhooks use?

Almost all webhook systems (Stripe, GitHub, Slack) use at-least-once delivery and explicitly document that receivers must handle duplicates. This is why webhook payloads typically include a unique event ID — receivers are expected to check that ID against records they have already processed before acting on the event again.

Does "exactly-once" mean the message is processed only once, ever?

It means the observable effect of processing looks the same as if it happened exactly once, even if the underlying message was technically delivered more than once. For example, a payment charged $50 that gets "processed twice" due to a retry, but the idempotency key ensures the account is only charged once — the customer experiences exactly-once behaviour even though two delivery attempts happened at the transport layer.

Messaging delivery semantics comparison

At-least-once vs exactly-once delivery

Every message queue and event stream has to make a promise about what happens when a network hiccup, a crash, or a slow acknowledgement gets in the way. "Exactly-once" sounds like the obvious best choice — but it's the hardest, most misunderstood guarantee in distributed systems, and knowing what it actually means (and doesn't) prevents costly bugs.

TL;DR

At-least-once guarantees no message is ever lost, at the cost of possible duplicates — the sender keeps retrying until it gets confirmation.
Exactly-once is a stronger guarantee that the end effect of processing looks like it happened one time — achieved through idempotency or transactional consumption, not through the transport magically never duplicating.
Most production systems are "at-least-once + idempotent consumer," which behaves like exactly-once without needing perfect delivery.

Side-by-side comparison

Aspect	At-least-once	Exactly-once
Message loss	Never (sender retries until acknowledged)	Never (built on top of at-least-once)
Duplicates possible?	Yes — the defining trade-off	No observable duplicates, thanks to idempotency/transactions
Complexity to implement	Low — retry until ack	High — needs idempotency keys or transactional offsets
Consumer responsibility	Must tolerate/dedupe duplicates itself	Framework or app logic guarantees no double-processing
Performance cost	Low overhead	Higher — transactions, dedup lookups add latency
Common examples	Most webhooks (Stripe, GitHub), SQS standard queues, RabbitMQ default	Kafka transactions (idempotent producer + read-committed), Flink checkpointing
Failure mode if ignored	Duplicate side effects — double-charged payment, duplicate email	N/A if implemented correctly — but adds real engineering cost
Default choice for new systems	Yes — simpler, and idempotency is needed anyway for retries/timeouts	Only when duplicates are genuinely unacceptable and can't be handled downstream

Code side-by-side

Processing a "charge customer" event safely under each model:

At-least-once — naive (buggy)

// Consumer may receive this event TWICE
onMessage("ChargeCustomer", (event) => {
  paymentGateway.charge(
    event.customerId, event.amount
  );
  // No dedup -> customer charged twice
  // if the ack was lost and the
  // message was redelivered
});

At-least-once + idempotency = "effectively exactly-once"

onMessage("ChargeCustomer", (event) => {
  // event.id is a unique, stable key
  if (processedIds.has(event.id)) {
    return; // already handled -> skip
  }
  paymentGateway.charge(
    event.customerId, event.amount,
    { idempotencyKey: event.id } // gateway
    // also dedupes on its own side
  );
  processedIds.add(event.id);
});

When at-least-once is the right (and usual) choice

You are building or consuming webhooks. Every major webhook provider delivers at-least-once and expects receivers to dedupe on the event ID — fighting this is pointless; design for it.
Losing a message is far worse than processing it twice. Order confirmations, notifications, and audit events are usually safe to occasionally duplicate but never safe to drop.
You need low operational complexity and high throughput. At-least-once with a simple retry loop is cheap to build and reason about compared to distributed transactions.
Your downstream operations are naturally idempotent. If "set status to SHIPPED" runs twice, nothing breaks — you don't need exactly-once machinery to be safe.

When you need exactly-once semantics

Duplicate processing causes real financial or data-integrity damage. Incrementing a balance, decrementing inventory, or issuing a single physical action (print a check, ship a package) needs strict duplicate protection.
You're building a stream-processing pipeline with aggregations. Kafka Streams / Flink exactly-once semantics prevent double-counting in running totals, which naive at-least-once consumption would silently corrupt.
You can afford the operational and latency cost. Transactional consumption and offset commits add real overhead — worth it only when the alternative (custom idempotency logic everywhere) is riskier than the framework cost.
Downstream systems are not naturally idempotent and can't easily be made so. If dedup keys aren't available or the operation genuinely isn't naturally repeatable, framework-level exactly-once removes that burden from every consumer.

English phrases engineers use

At-least-once conversations

"This queue is at-least-once — always design consumers to be idempotent."
"We got a duplicate delivery after the consumer's ack timed out."
"The webhook retried three times before we returned a 200."
"Dedup on the event ID, not on message content."

Exactly-once conversations

"We enabled Kafka's exactly-once semantics for this consumer group."
"The idempotency key prevents the gateway from double-charging."
"This is effectively-once, not truly exactly-once at the network layer."
"We commit the offset in the same transaction as the write, so there's no gap."

Quick decision tree

Building or consuming webhooks → At-least-once + idempotency key dedup
Operation is naturally idempotent (set-status, upsert) → At-least-once is enough
Operation is a raw increment/decrement (balance, inventory count) → Need exactly-once processing or make it idempotent (upsert-style)
Stream aggregation / running totals → Exactly-once processing (Kafka transactions, Flink checkpoints)
Team is new to messaging and timeline is tight → Start at-least-once; add idempotency, not full exactly-once machinery
Regulatory requirement for "processed exactly once, provably" → Exactly-once with audited idempotency keys

Common Mistakes and Trade-offs

A remarkably common mistake developers make when adopting at-least-once delivery is conflating it with 'eventual consistency'. Teams often assume that because messages are eventually processed, the system is inherently robust and reliable. This overlooks the critical need for idempotency – ensuring a message can be safely processed multiple times without causing unintended side effects. Without explicit idempotency guarantees built into your consumers, you're just adding complexity on top of an already potentially complex delivery mechanism. Simply sending a message 'at-least-once' doesn't magically make your system consistent; it merely shifts the responsibility for ensuring consistency to the consumer logic itself. It's a crucial distinction that frequently leads to data corruption or duplicated operations when not carefully considered.

At scale, exactly-once delivery begins to reveal significant trade-offs related to distributed transaction management and rollback complexity. While conceptually simpler than traditional two-phase commit protocols, attempting 'exactly-once' processing across geographically dispersed services introduces a cascade of challenges. Each message might require its own idempotent operation on multiple systems – tracking state, compensating for failures, and managing potential retry loops. This quickly escalates operational overhead, increasing the cost not just in terms of engineering time but also in increased latency introduced by these complex coordination mechanisms. The performance impact of repeated attempts to reconcile inconsistencies becomes a significant bottleneck, particularly under heavy load.

There's a persistent misconception that 'exactly-once' delivery is always preferable for critical business processes – specifically financial transactions or order updates. This stems from the perception that it's inherently more reliable and provides stronger guarantees. However, this ignores the substantial operational overhead introduced by exactly-once systems, particularly in highly dynamic environments. At-least-once delivery, when combined with robust idempotency, can often provide sufficient reliability for many use cases – especially those where eventual consistency is acceptable or where the cost of achieving perfect accuracy outweighs the complexity of an exact-once implementation. The focus should be on understanding the business requirements and data sensitivity rather than blindly pursuing 'exactly-once' simply because it sounds more robust.

Migrating between at-least-once and exactly-once delivery isn't a straightforward switch; it often requires a layered approach. A common strategy is to initially adopt at-least-once for core functionality, building in strong idempotency layers within the consumer logic. Then, as specific requirements dictate, selectively introduce more sophisticated exact-once mechanisms – perhaps using transactional outbox patterns or dedicated choreography services – where true atomicity and guaranteed ordering are paramount. Furthermore, it's increasingly common to see hybrid approaches where some flows utilize at-least-once for asynchronous tasks while others leverage exactly-once for critical state changes, effectively combining the benefits of both models based on context and operational cost.

Frequently asked questions

Is "exactly-once" delivery actually possible?

True exactly-once network delivery is not possible in a distributed system with unreliable networks — you can never be 100% certain a message was received without an acknowledgement, and acknowledgements can themselves be lost, which forces a retry that may duplicate the message. What systems like Kafka actually provide is "exactly-once processing" or "effectively-once semantics": at-least-once delivery combined with idempotent writes or transactional offsets, so the end result looks exactly-once even though a message might physically be delivered more than once.

What is the difference between at-least-once and at-most-once?

At-least-once guarantees a message is never lost — the sender retries until it gets an acknowledgement, which can cause duplicates. At-most-once guarantees a message is never duplicated — the sender does not retry, which can cause message loss if the first attempt fails silently. At-least-once is the far more common choice because losing data is usually worse than having to deduplicate it.

How do you achieve exactly-once processing in practice?

The standard technique is idempotency: design the consumer so that processing the same message twice produces the same result as processing it once. Common patterns are using a unique message ID and an "already processed" table (idempotency key), using upserts instead of increments (SET balance = 100 instead of balance += 10), or using the messaging system's built-in transactional guarantees (Kafka's exactly-once semantics ties message consumption, processing, and offset commit into one atomic transaction).

Show more questions (4)