Mesh Observability Language

Envoy operates at L7 (application layer) — it understands HTTP, gRPC, and headers — so it can generate rich metrics for every request without the application doing anything.

Compare this to L3/L4 monitoring (IP/port level — e.g., network flow logs): at L4, you can see "X bytes from service A to service B," but you can't see HTTP status codes or request paths. Envoy sees everything at L7: the HTTP method, URL path, response code, and response time for every request. It exposes these as Prometheus metrics (istio_requests_total, istio_request_duration_milliseconds) with labels for source service, destination service, and response code — enabling per-route latency and error dashboards with zero application instrumentation.

Key vocabulary:
• L7 observability — metrics at the HTTP/gRPC application layer (status codes, latency, routes)
• istio_requests_total — Prometheus counter: request count per source/destination/status_code
• zero instrumentation — mesh-generated metrics require no application code changes

The four golden signals — latency, traffic, errors, saturation — are the standard lens for service health, and a service mesh provides them for every service pair automatically.

Before a service mesh, each team had to instrument their service to expose these signals. With the mesh, Envoy generates them for every service-to-service call: Traffic = requests/second (istio_requests_total rate), Errors = error rate (HTTP 5xx / total), Latency = request duration histogram (istio_request_duration_milliseconds), Saturation = active connections / connection pool limit. The power: you can see the golden signals for a service your team doesn't own, or for a legacy service with no instrumentation.

Key vocabulary:
• traffic (golden signal) — requests per second; demand on the service
• errors (golden signal) — rate of failed requests; commonly HTTP 5xx / total
• saturation (golden signal) — how close a service is to its capacity limit

In Kiali, edge colour represents traffic health — green (healthy), yellow (degraded), red (high error rate). A red edge is an actionable signal pointing to a service communication problem.

Kiali's service graph visualises the live traffic topology: nodes are services, edges are active connections between them. Each edge shows: request rate (RPS label), error percentage (coloured by threshold), and response time. A red edge between checkout and inventory means a significant percentage of checkout-to-inventory requests are failing (returning 5xx or connection errors). Combined with Kiali's distributed trace links, you can drill from the red edge directly to failing trace samples to identify the root cause.

Key vocabulary:
• Kiali service graph — real-time service topology visualisation for Istio; nodes = services, edges = traffic flows
• edge colour — green (healthy), yellow (degraded), red (high error rate); thresholds configurable
• service graph — visual map of which services communicate with which; automatically generated from mesh telemetry

Envoy creates spans automatically, but it needs the trace context to know which trace they belong to — and only the application knows which incoming request caused which outgoing request.

Envoy generates a span for every request it handles (inbound and outbound). For outbound requests, Envoy includes the trace context headers automatically. The problem: when the application receives request A and makes three downstream requests (B, C, D), Envoy at the outbound sidecar needs to know "this outbound call is part of trace X, span Y." That information comes from the inbound request's trace context headers. The application must extract those headers from the incoming request and inject them into the outgoing requests. Without this, Envoy generates three isolated spans for B, C, D that can't be connected to the original trace for A.

Key vocabulary:
• trace context propagation — application forwards B3/W3C headers from incoming to outgoing requests
• traceparent — W3C trace context header: version-traceId-parentSpanId-flags
• span stitching — linking individual spans into a complete end-to-end trace using shared traceId

The mesh and application instrumentation are complementary — mesh metrics cover the network layer; application metrics cover business logic and internal state.

A concrete example: Envoy can tell you that 2% of requests to the payments service are returning 503. But it can't tell you: how many payment transactions are in a pending state, what the database connection pool utilisation is, or how many fraud checks are being bypassed by the fallback logic. Those require application instrumentation. The ideal observability stack combines: mesh metrics (automatic, per-service network health) + application custom metrics (business logic) + structured application logs (event details) + distributed traces (request flow). The mesh reduces the baseline instrumentation burden significantly — teams should focus their instrumentation effort on business-relevant signals.

Key vocabulary:
• mesh metrics — automatic HTTP/gRPC metrics generated by Envoy at the network layer
• application metrics — custom instrumentation for business logic, internal state, and domain events
• complementary observability — mesh + application metrics together provide full-stack visibility