Noisy Neighbour & Resource Fairness Vocabulary

The noisy neighbour problem is the central resource fairness challenge in every multi-tenant system that shares infrastructure.

How it manifests:

Resource	Noisy neighbour scenario	Impact on others
Database connection pool	Tenant X holds 90% of shared connections during a batch job	Other tenants time out waiting for a connection
CPU	Tenant Y runs a CPU-intensive report on a shared compute node	API latency spikes for all co-located tenants
Network I/O	Tenant Z triggers a large data export saturating the NIC	Increased latency for all tenants on the same host
Shared message queue	Tenant A floods a shared queue with millions of events	Other tenants' events are delayed — delayed webhooks, stale data

Why it is harder to detect than it sounds:
• Without per-tenant metrics, operators see aggregate resource pressure but cannot identify which tenant is responsible
• The noisy tenant often experiences no degradation themselves (they are consuming the resource, not waiting for it)
• The affected tenants may interpret the degradation as a product quality issue and churn

Key vocabulary:
• Noisy neighbour — a tenant whose high resource consumption negatively impacts co-tenant performance
• Shared infrastructure — compute, storage, network, or database resources consumed by multiple tenants simultaneously
• Resource contention — the condition where multiple tenants compete for the same finite resource
• SLA breach — a failure to meet a contracted service level agreement on latency, availability, or throughput

Per-tenant quotas are the primary preventive control against the noisy neighbour problem — they enforce bounded consumption at the platform level.

What quotas are applied to in a typical SaaS platform:

Resource	Quota example	Enforcement
API requests	1,000 req/min per tenant	API gateway rate limiter
DB connections	Max 10 concurrent connections per tenant	Connection pool manager (PgBouncer)
Storage	50 GB max per tenant on Free plan	Upload service quota check
Background jobs	Max 5 concurrent workers per tenant	Job queue worker pool with tenant partitioning
Outbound webhooks	100 webhook deliveries/min per tenant	Delivery rate limiter in webhook service

Quota enforcement strategies:
• Hard reject: return HTTP 429 (Too Many Requests) immediately when quota is hit — appropriate for API rate limits
• Throttle/queue: accept the request but delay execution — appropriate for background job queues where strict ordering matters
• Upsell prompt: allow soft-limit breach but surface an in-app upgrade prompt — appropriate for storage and seat limits

Key vocabulary:
• Per-tenant quota — a hard or soft limit on the amount of a given resource a single tenant may consume
• Resource governance — the set of policies and enforcement mechanisms that ensure fair resource distribution across tenants
• Throttle — to deliberately constrain a tenant's request rate or processing speed to enforce their quota
• HTTP 429 Too Many Requests — the standard response code indicating a rate limit has been exceeded

Connection pool starvation is one of the most common and severe noisy neighbour incidents in shared-database multi-tenant architectures.

Why it happens:
• A shared PostgreSQL connection pool has a hard ceiling (e.g. 100 connections set by max_connections)
• Without per-tenant limits, a single tenant running a slow data export or a misconfigured ORM (e.g. N+1 query loop) can exhaust the pool
• Other tenants' queries queue indefinitely waiting for a free connection — latency climbs from milliseconds to seconds

Immediate mitigation steps:
① Apply a per-tenant connection cap in PgBouncer: limit Tenant X to a maximum of N connections
② Tenant X's excess requests are queued or returned with a "too many connections" error
③ Available connections are immediately freed for other tenants
④ Latency for other tenants returns to normal within seconds

Permanent fix (after incident):
• Enforce per-tenant connection limits as a standing platform policy — not just during incidents
• Implement tenant-level observability (see next exercise) so operators detect connection pool pressure before it causes a SLA breach
• Investigate Tenant X's query patterns: N+1 queries and missing indexes are common root causes

PgBouncer per-tenant configuration example:
[tenant_x_pool] pool_size = 5 max_client_conn = 10
Key vocabulary:
• Connection pool starvation — a state where all connections in a shared pool are held, leaving other tenants unable to acquire a connection
• PgBouncer — a lightweight PostgreSQL connection pooler commonly used to enforce per-tenant connection limits
• Pool pressure — the degree to which a connection pool is approaching its maximum capacity
• N+1 query problem — an ORM anti-pattern that issues one query per record instead of a single JOIN, silently consuming many connections

Rate limiting and throttling are two distinct enforcement mechanisms — choosing the wrong one for a workload type creates poor user experience or wasted compute.

Property	Rate Limiting	Throttling
Behaviour when limit hit	Reject immediately (fail fast)	Queue/delay (slow down)
HTTP response	429 Too Many Requests + Retry-After header	200 OK eventually (after delay)
Best for	Interactive API calls, user-facing requests (fail fast is better UX than a 10 s wait)	Background jobs, bulk imports, batch exports (eventual completion is acceptable)
Client handling	Client must implement retry with exponential back-off	Client waits; no retry logic needed
Resource impact	Minimal — rejected requests consume no downstream resources	Higher — queued work still consumes memory and queue infrastructure

Common implementation patterns:
• Token bucket algorithm: tokens replenish at a fixed rate; each request consumes one token. Best for bursty traffic (allows short bursts, then rate-limits).
• Leaky bucket algorithm: requests enter a queue and drain at a fixed rate regardless of input rate — this is throttling, not rate limiting.
• Fixed window counter: simplest implementation; bucket resets every N seconds. Vulnerable to boundary bursts.
• Sliding window counter: more accurate; prevents boundary exploitation.

Key vocabulary:
• Rate limiting — rejecting requests above a defined per-tenant threshold; fast failure
• Throttling — accepting but slowing down requests above a threshold; eventual execution
• HTTP 429 — Too Many Requests; the standard rate limit response; should include a Retry-After header
• Token bucket — an algorithm that allows brief bursts while enforcing an average rate over time

Tenant-level observability is the difference between "the database is slow" and "Tenant X is responsible for 73% of current query load" — without it, noisy neighbour incidents are invisible until they cause SLA breaches.

What tenant-level observability tracks:

Metric	Grouped by tenant_id	Why it matters
API request rate	Requests/min per tenant	Identify tenants approaching or exceeding rate limits
DB query count	Queries/sec per tenant	Surface tenants with N+1 query anti-patterns
DB connection usage	Active connections per tenant	Detect connection pool starvation before it affects others
Storage I/O	Read/write MB per tenant	Catch large data exports or bulk import jobs
Job queue depth	Queued jobs per tenant	Identify tenants flooding background job queues

Implementation approaches:
• Tagged metrics: emit Prometheus/StatsD metrics with a tenant_id label on every instrumented operation
• Tenant dashboards in Grafana: pre-built panels showing top-N tenants by resource consumption, updated in real time
• Alerting rules: "Alert if any single tenant exceeds X% of total connection pool for more than 2 minutes"
• Structured logging: include tenant_id as a first-class field in all log lines — enables SQL queries in log analytics (e.g. CloudWatch Insights, Datadog)

Key vocabulary:
• Tenant-level observability — per-tenant metrics and logs that allow operators to isolate each tenant's resource consumption footprint
• Cardinality — the number of unique label values in a metric; high-cardinality tenant labels require careful handling in time-series databases
• Top-N analysis — identifying the highest-consuming tenants from aggregated metrics; essential for proactive noisy neighbour management
• P99 latency — the 99th percentile latency; a key SLA metric often degraded first during noisy neighbour events

The core of the 'noisy neighbour' problem lies in *unexpected* spikes in resource usage by one tenant that negatively impact others. Sarah's message highlights this – a sudden spike suggests an issue rather than a gradual accumulation. The term refers to a single tenant causing disproportionate problems for the overall system, not simply high CPU usage itself.

'Dynamic throttling' describes a system that doesn't rely on fixed thresholds. It actively monitors and adjusts resource allocation – like CPU or memory – in response to changing demand from individual tenants. This contrasts with static limits (option A) which are ineffective against fluctuating workloads and would inevitably lead to noisy neighbours. Options C and D represent entirely different operational activities.

Connection pooling with per-tenant limits is the most effective way to prevent a single tenant from exhausting shared resources. By assigning each tenant their own pool, contention is minimized. Increasing overall capacity (option A) simply exacerbates the problem if one tenant consumes everything. Options C and D are overly complex or ineffective solutions.

A sustained high CPU utilization (98%) indicates that Tenant Z is heavily utilizing resources, which directly correlates with the 'noisy neighbour' problem. This suggests they are exceeding their allocated capacity and negatively affecting other tenants sharing the same infrastructure. Options A, C, and D represent alternative interpretations of the data.

Rate limiting restricts the *rate* at which requests are processed from a tenant. This directly addresses the noisy neighbour problem by preventing one tenant from overwhelming the system and impacting the performance of others. It's about controlling the *volume* of requests, not scaling infrastructure or encrypting traffic.

This statement describes cross-tenant observability – understanding how tenants interact and share resources. Correlating usage patterns reveals potential abuse or imbalances that wouldn't be apparent by examining individual tenant metrics alone. This is crucial for proactively detecting and addressing noisy neighbour issues.