Vocabulary for Observability and Monitoring in DevOps

Observability is the practice of understanding the internal state of a system by examining its outputs. As teams move toward distributed systems and microservices, the vocabulary of observability has expanded rapidly. Whether you are configuring dashboards, triaging an incident, or presenting reliability metrics to stakeholders, you need precise English vocabulary to communicate clearly.

The Three Pillars of Observability

The observability community commonly refers to three core data types that together give you visibility into a system’s behaviour.

1. Metrics

Metrics are numerical measurements collected over time. They are the backbone of monitoring dashboards.

• Time series — a sequence of data points indexed by time, e.g., request rate per second
• Gauge — a metric that represents a value at a specific moment, such as current memory usage
• Counter — a metric that only increases, such as total number of requests served
• Histogram — a metric that tracks the distribution of values, often used for latency
• Percentile (P50, P95, P99) — the value below which a percentage of observations fall; P99 latency means 99% of requests are faster than that value
• Cardinality — the number of unique label combinations in a metric; high cardinality can strain monitoring infrastructure
• Scrape interval — how frequently a metrics system polls a target for new data

2. Logs

Logs are time-stamped records of discrete events within a system.

• Structured logging — writing logs as machine-readable key-value pairs (e.g., JSON) rather than plain text strings
• Log level — the severity of a log entry: DEBUG, INFO, WARN, ERROR, FATAL
• Log aggregation — collecting logs from multiple sources into a central system (e.g., Elasticsearch, Loki)
• Log retention — how long log data is kept before deletion or archival
• Correlation ID — a unique identifier attached to a request as it flows through multiple services, enabling trace reconstruction from logs

3. Traces

Distributed tracing tracks a request as it travels across multiple services.

• Trace — the full record of a single request’s journey through the system
• Span — a single unit of work within a trace, representing one operation in one service
• Parent span / child span — the hierarchical relationship between operations in a trace
• Trace context propagation — passing trace identifiers between services so spans can be linked
• Sampling — recording only a fraction of traces to manage storage costs while maintaining statistical insight
• Flame graph — a visual representation of a trace showing which operations took the most time

Key Vocabulary: Reliability and SLOs

• SLI (Service Level Indicator) — a specific, measurable metric used to assess service performance, e.g., request success rate
• SLO (Service Level Objective) — the target value for an SLI, e.g., “99.9% of requests succeed”
• SLA (Service Level Agreement) — a contractual commitment to external customers based on SLOs
• Error budget — the allowable amount of downtime or errors within an SLO period; if the SLO is 99.9%, the error budget is 0.1%
• Burn rate — how quickly the error budget is being consumed; a high burn rate signals a service health problem
• MTTR (Mean Time to Recover) — the average time to restore service after an incident
• MTBF (Mean Time Between Failures) — the average time between incidents
• Availability — the percentage of time a service is operational and accessible

Key Vocabulary: Alerting and On-Call

• Alert — a notification triggered when a metric crosses a defined threshold
• Threshold — the value at which an alert fires, e.g., “alert if error rate exceeds 5%”
• Alert fatigue — the desensitisation that occurs when engineers receive too many low-quality alerts
• False positive — an alert that fires when there is no real problem
• False negative — a real problem that does not trigger an alert
• On-call rotation — a schedule assigning responsibility for responding to alerts to team members in turn
• Escalation policy — the defined process for notifying additional responders if an alert is not acknowledged within a time limit
• Runbook — a documented procedure for responding to a specific alert or incident type
• Silencing / muting — temporarily disabling an alert during planned maintenance

Talking About Observability in Team Conversations

Discussing Dashboard Health

• “The P99 latency on the checkout service has been trending upward for the past three hours.”
• “We’re seeing elevated error rates on the payment gateway — roughly 2.3%, which is above our SLO threshold of 1%.”
• “The trace data shows that 80% of the latency is coming from a single downstream call to the inventory service.”
• “Our error budget is at 23% remaining for this month. We need to be careful about what we ship this week.”

During an Incident

• “I’m seeing a spike in the 5xx rate starting at 14:32 UTC. All three instances in eu-west-1 are affected.”
• “The flame graph shows the slow span is in the database query layer — specifically the product search endpoint.”
• “Correlation ID trace shows the request is timing out waiting for the recommendations service.”
• “We’ve silenced the secondary alerts so we can focus on the primary incident. Escalation is active.”

In a Post-Incident Review

• “MTTR for this incident was 47 minutes — above our 30-minute target. Let’s look at the detection and response timeline.”
• “The alert fired correctly, but the runbook didn’t cover this failure mode, which added to the resolution time.”
• “We had two false positives this week that created noise during the incident. We’ll tune those thresholds.”

Tools Vocabulary

Observability conversations often reference specific tool categories:

• APM (Application Performance Monitoring) — tools like Datadog, New Relic, Dynatrace that provide end-to-end performance visibility
• TSDB (Time Series Database) — specialised databases for metrics, such as Prometheus or InfluxDB
• Observability platform — an integrated suite combining metrics, logs, and traces, e.g., Grafana Stack, Honeycomb, Datadog
• OpenTelemetry (OTel) — an open standard for collecting and exporting observability data across languages and vendors

Understanding and using this vocabulary precisely will make you a more effective participant in reliability discussions, incident response, and architectural reviews. Observability is increasingly a core engineering competency — and the language to discuss it is part of that competency.

In Practice: Navigating Nuances with Non-Native Speakers

Observability and monitoring terminology can be particularly challenging for developers whose first language isn’t English. It’s not just about knowing the definitions; it’s about understanding the subtle connotations and preferred phrasing that builds trust and clarity within a DevOps team. A simple translation often misses the mark, leading to misunderstandings during code reviews or when discussing performance issues. Let’s consider a common scenario: a junior developer receives feedback on a pull request.

Imagine Sarah, a developer recently moved from Poland to our company. She’d diligently followed the documentation and implemented the requested feature – a new API endpoint for user authentication. During code review, her teammate, Mark, comments directly on the PR description: “This needs more context; we need to understand why you chose this approach.” Sarah feels immediately defensive. The phrasing “needs more context” sounds accusatory, implying her work is somehow lacking in understanding or justification. A native speaker might interpret it as a request for clarification regarding the design decisions – perhaps a brief explanation of trade-offs she considered. For Sarah, it feels like criticism without offering constructive guidance.

Similarly, during an on-call shift, communicating effectively about an alert is crucial. Let’s say a monitoring system flags unusually high latency in our database service. Instead of simply stating “Database latency high,” a more helpful phrasing might be: “We’re seeing elevated latency in the database service, potentially impacting user transactions. Can you investigate and determine if this is related to recent schema changes or increased load?” The latter acknowledges the potential impact and invites collaboration – crucial when dealing with complex systems and time-sensitive issues. Using precise language reduces ambiguity and avoids assumptions. Focusing on impact rather than just stating a metric helps frame the problem in a way that’s easier for everyone to grasp, particularly those whose first language doesn’t naturally prioritize technical detail.

Finally, when documenting metrics or creating dashboards, it’s important to avoid jargon that might not be immediately familiar. Instead of “CPU Utilization,” consider “Percentage of CPU Time Used by the Application.” Small shifts in wording can significantly improve comprehension and ensure everyone is on the same page. The goal isn’t simply to translate technical terms; it’s about communicating effectively, fostering a collaborative environment, and ultimately, driving better operational outcomes.

# Example: Using Prometheus' `describe` command to investigate latency metrics
prom query -n my_namespace -g "up{job='my-api'}" -- noquery