English for Apache Pulsar Messaging

Pulsar discussions often need to clarify one distinction early: unlike some messaging systems, a single topic in Pulsar can support several independent subscription patterns at once, so “who’s consuming this and how” is rarely a one-word answer.

Key Vocabulary

Subscription type (exclusive, shared, failover, key-shared) — the mode that determines how messages on a topic are distributed among the consumers attached to a subscription. “We switched this subscription from shared to key-shared, since we needed all messages for the same order ID to land on the same consumer, in order.”

Tiered storage — Pulsar’s ability to automatically offload older message segments from local disk to cheaper, long-term object storage, without consumers needing to know where the data physically lives. “We don’t need a separate archival pipeline — tiered storage moves the older segments to object storage automatically once they age out of the hot tier.”

Tenant / namespace — Pulsar’s built-in multi-tenancy model, where a tenant contains namespaces, and namespaces contain topics, letting many teams share one cluster with isolated configuration and quotas. “Instead of standing up a separate cluster for each team, we gave each one its own tenant, with its own retention and quota policies configured at the namespace level.”

BookKeeper (bookie) — the underlying distributed log storage layer that Pulsar brokers write to, responsible for durability and replication, separate from the brokers that handle client connections. “The brokers themselves were fine — the actual write latency spike traced back to a slow bookie in the BookKeeper cluster.”

Backlog — the accumulated set of unacknowledged messages waiting for a subscription to consume, a key metric for spotting a consumer that’s falling behind. “The alert wasn’t about the producer at all — it was the backlog on one subscription growing steadily, meaning that consumer group had stalled.”

Common Phrases

• “Which subscription type does this consumer group actually need — shared, or key-shared for ordering guarantees?”
• “Is this data still on local disk, or has it already moved to tiered storage?”
• “Should this be its own tenant, or can it share a namespace with the existing team’s topics?”
• “Is this a broker problem, or is the actual bottleneck down in BookKeeper?”
• “Is the backlog growing because the consumer is slow, or because it’s actually stopped processing?”

Example Sentences

Explaining a subscription choice in review: “We need per-key ordering for these events, so key-shared is the right subscription type here, not plain shared.”

Describing a cost optimization: “We enabled tiered storage on this namespace so months-old data moves to object storage automatically instead of eating expensive local disk.”

Diagnosing a latency issue: “The brokers reported healthy, so we checked BookKeeper directly and found one bookie with degraded disk I/O dragging down write latency.”

Professional Tips

• Name the exact subscription type when discussing message ordering or distribution — “consumers aren’t getting messages in order” almost always traces back to the wrong subscription mode.
• Mention tiered storage explicitly when discussing retention costs — it changes the cost conversation from “how long can we afford to keep this” to “how much hot-tier disk do we actually need.”
• Use tenant and namespace boundaries deliberately when onboarding a new team — it’s the mechanism for isolating quotas and configuration, not just an organizational label.
• Separate broker and BookKeeper issues explicitly when diagnosing latency — they’re different layers with different failure modes.

Practice Exercise

1. Explain the difference between a shared and a key-shared subscription.
2. Describe what tiered storage does and why it matters for cost.
3. Write a sentence explaining the difference between a broker and a bookie.

Navigating Nuance: Refining Your Pulsar Vocabulary for Effective Communication

Pulsar’s architecture is complex, and even seemingly simple terms can have layered meanings depending on the context. For non-native English speakers, this can be a significant source of misunderstanding during code reviews, discussions about system design, or when collaborating with international teams. It’s not just about knowing what a term means – it’s about understanding how to use it precisely and effectively within Pulsar’s ecosystem. Let’s consider some common scenarios where nuanced language becomes critical.

Imagine you’re reviewing a pull request that proposes increasing the retention period for messages in a specific topic. A simple statement like “increase retention” isn’t enough. A colleague might respond with, “That could impact tiered storage costs significantly. Can you elaborate on why you’re suggesting this change and what metrics you’re using to justify it?” The key difference is the addition of qualifying phrases – demonstrating a deeper understanding of the potential ramifications. Similarly, if you’re describing a new subscription pattern, saying “we need more subscriptions” lacks precision. Instead, consider: “We require a higher throughput subscription for this specific use case, enabling us to handle peak loads effectively and minimize latency.” The phrasing emphasizes the reason behind the request and its impact on performance – crucial elements of professional communication in a distributed system like Pulsar.

Another frequent area where vocabulary matters is when discussing multi-tenancy configurations. Simply stating “assign this tenant to the cluster” doesn’t convey the full picture. A more detailed explanation would be: “Let’s configure this tenant with resource quotas – specifically limiting CPU and memory usage – to ensure fair allocation of resources across our different teams and prevent any single tenant from negatively impacting overall system performance.” Precise terminology, like “resource quotas” and “fair allocation,” demonstrates a solid grasp of Pulsar’s multi-tenancy capabilities and highlights the importance of proactive resource management. This kind of careful wording helps avoid ambiguity and ensures everyone is on the same page.

Finally, during Slack conversations or PR descriptions, it’s vital to use the correct terminology when troubleshooting issues. Instead of saying “the thing isn’t working,” a more helpful approach would be: “I’m observing high latency in this topic; let’s investigate potential bottlenecks related to subscription throughput and consumer lag.” This demonstrates a systematic understanding of Pulsar’s components and helps guide the troubleshooting process effectively.

Here’s an example of how you might use pulsar-admin to create a new topic with specific configurations:

pulsar-admin topics create --name my-topic --type keyValue --partitions 10 --replication-factor 3 --retention-ms 86400000

This command, when communicated clearly – “We’re creating a new KeyValue topic named ‘my-topic’, configuring it with 10 partitions and a replication factor of 3 to ensure high availability and data redundancy. The retention period is set for 24 hours (86400000 milliseconds).” – demonstrates an understanding not just of the command itself but also of its implications within Pulsar’s architecture.