English for OpenTelemetry

“Where’s the latency coming from” is unanswerable without the right vocabulary in a distributed system, because the delay could be in any one of a dozen services — traces, spans, and instrumentation are the words that let a team point at the exact hop that’s slow.

Key Vocabulary

Trace — the complete record of a single request’s journey through a distributed system, made up of all the spans generated as it passes through each service. “Pull up the trace for that slow request — it’ll show us exactly which service in the chain added the extra 400 milliseconds.”

Span — a single unit of work within a trace, representing one operation (like a database query or an API call) with a start time, duration, and metadata, nested to show the call hierarchy. “The span for the database query is taking 300 of the trace’s 400 milliseconds — that’s where we should focus, not the API gateway.”

Instrumentation — the code (manual or automatic) that generates spans and traces as a service processes requests, the actual mechanism by which observability data gets collected in the first place. “We added instrumentation to the payment service last sprint — before that, it was a black box in every trace, just a gap with no visibility into what it was doing.”

Context propagation — passing trace identifiers between services as a request flows through them, so spans generated by different services can be correctly linked into a single trace instead of appearing as unrelated fragments. “The trace breaks at the queue boundary because context propagation isn’t wired up for async jobs — spans before and after the queue show up as two disconnected traces instead of one.”

Collector — the OpenTelemetry component that receives, processes, and exports telemetry data from instrumented services to a backend (like Jaeger or Datadog) for storage and visualization. “Traces stopped showing up in the dashboard because the collector’s been down for an hour — services are still generating spans, they’re just not reaching anywhere to be stored.”

Common Phrases

• “Can we pull up the trace for that request and see which span is slow?”
• “Is this service instrumented yet, or is it still a black box in the trace?”
• “Is context propagation actually wired up across this service boundary?”
• “Is the collector healthy, or is that why traces are missing?”
• “Which span is actually the bottleneck here?”

Example Sentences

Diagnosing latency with a trace: “Looking at the trace, the request spends 50ms in the API gateway and 20ms in auth, but then 800ms in a single span for the recommendation service — that’s clearly where the latency budget is going.”

Explaining a visibility gap: “We can’t see what’s happening in the legacy billing service during incidents because it’s not instrumented — every trace just shows a gap where that service should be, so we’re debugging blind for that hop.”

Describing a broken trace during a retro: “The trace was fragmenting into pieces because context propagation wasn’t implemented for the message queue — spans on either side of the queue looked like two unrelated requests instead of one continuous trace.”

Professional Tips

• Say span, not “step” or “part,” when pointing at a specific slow operation within a trace — it’s the exact unit observability tools measure, and using it precisely gets you a faster answer from whoever’s reading the trace.
• Flag missing instrumentation directly as a gap, not just “we don’t have visibility” — naming the specific uninstrumented service tells the team exactly what to fix.
• Check context propagation first when a trace looks fragmented across an async boundary (queues, background jobs) — it’s the most common reason traces break apart instead of linking correctly.
• Confirm the collector is healthy before assuming instrumentation itself is broken — a missing trace can mean nothing was ever generated, or that it was generated and lost in transit.

Practice Exercise

1. Write a sentence explaining the relationship between a trace and a span.
2. Explain what context propagation does and why it matters across service boundaries.
3. Describe how you’d diagnose whether missing trace data is an instrumentation problem or a collector problem.

Expanding Your Observability Vocabulary: A Focus on Nuance

The core concepts of OpenTelemetry – traces, spans, and instrumentation – are relatively straightforward to grasp. However, truly communicating effectively within a development team focused on observability requires more than just knowing the terms; it’s about using them precisely and with an understanding of the subtle differences in meaning that native English speakers often take for granted. For non-native developers, this can be a significant hurdle, leading to misunderstandings or inefficient discussions. Let’s look at how to refine your phrasing when describing these concepts – particularly focusing on clarity and precision.

One common issue arises during code reviews. Imagine receiving a comment like, “This span is missing instrumentation.” While technically accurate, it lacks context. A more helpful response might be, “Could you add an instrumented metric to this span that tracks the duration of the database query? This will help us understand if there are any performance bottlenecks.” Notice the added specificity – we’re not just pointing out a deficiency (“missing instrumentation”), but suggesting what should be added and why. Similarly, in Slack discussions about a new feature, stating “We need to instrument this service” is vague. A better approach would be: “Let’s add OpenTelemetry spans to this service to track request latency and error rates as we roll out the changes.” The second example provides immediate context and clarifies the purpose of the instrumentation. The key here is to move beyond simple definitions and demonstrate an understanding of how instrumentation contributes to a broader observability strategy.

Another area where nuance matters is in PR descriptions. A generic description like, “Added OpenTelemetry” simply doesn’t provide enough information for reviewers. Instead, aim for something like: “Implemented OpenTelemetry spans on the user_authentication service to capture request processing time and identify potential performance issues. This includes instrumentation for database query durations and API call latency. We’re using a sampling strategy to reduce overhead.” This detailed description allows reviewers to quickly assess the impact of the changes and understand how it aligns with the team’s observability goals. It also immediately signals that you’ve considered potential bottlenecks and proactively added relevant metrics.

Finally, be mindful of active vs. passive voice. While both are acceptable, using active voice generally leads to clearer and more concise communication. Instead of saying “Spans were created by this code,” say “This code creates spans.” It’s a small difference, but consistently applying this principle can significantly improve the flow and clarity of your technical writing.

# Example OpenTelemetry Python Instrumentation - Logging Span Attributes

import os
from opentelemetry import trace
from opentelemetry.propagate import tracecontext
import logging

tracer = trace.get_tracer(__name__)

def my_function(input_data):
    ctx = tracecontext.get(os.environ.get("OPENTELEMETRY_TRACEID", "default-traceid")) # Get trace ID from environment if available
    span = tracer.start_as_current("my_function")
    try:
        logging.info(f"Input data: {input_data}")
        result = input_data * 2
        logging.info(f"Result: {result}")
        span.set_attribute("input_value", input_data)
        span.set_attribute("output_value", result)
    except Exception as e:
        span.record_exception(e)
    finally:
        span.end()

if __name__ == "__main__":
    my_function(10)