Intermediate Cloud-Native #immutable-infrastructure #circuit-breaker #sidecar #bulkhead #declarative

Cloud-Native Patterns

5 exercises — master the vocabulary of cloud-native design patterns: immutable infrastructure, circuit breaker states, sidecar architecture, bulkhead isolation, and declarative configuration.

0 / 5 completed

Cloud-native pattern vocabulary quick reference

Immutable infrastructure — replace rather than patch; every change produces a new versioned artifact
Circuit breaker — Closed (normal) → Open (tripped) → Half-open (probing) → Closed (recovered)
Sidecar — a co-located helper container handling cross-cutting concerns independently from the app
Bulkhead — isolated resource pools (threads, connections) so one slow consumer can't starve others
Declarative config — specify desired state; a reconciliation loop converges actual state toward it
Ambassador — a sidecar that proxies outbound calls (adds retry, circuit breaking, service discovery)
Pets vs. cattle — cattle servers are replaced when broken; pets are named and individually maintained

1 / 5

An operations engineer says: "We never SSH into a running server to apply a hotfix — we build a new image with the patch baked in, re-deploy, and terminate the old instance."

Which cloud-native principle does this practice reflect?

Immutable infrastructure treats compute instances the same way container images are treated — as read-only artifacts you replace, not modify.

Mutable vs. immutable operations:

Approach	How changes are applied	Risks
Mutable	SSH in, run `apt install`, edit config files, restart services	Configuration drift; unauditable; hard to reproduce; "works on this server but not that one"
Immutable	Build new AMI/container image → deploy → drain old instances → terminate	Slightly longer deploy cycle; every change requires a full image build

Why immutable infrastructure improves reliability:
• No configuration drift — every instance is identical to every other; every running server is the same as your last tested image
• Reproducibility — the image tag is the audit trail; you can deploy exactly the same artifact to any environment
• Fast rollback — deploy the previous image tag; no "undo" scripts needed
• Security compliance — no open SSH access to production; no manual changes outside the CI/CD pipeline

"Pets vs. cattle" explained (related concept):
• Pets — servers you name, nurture, and repair when sick (traditional ops model)
• Cattle — servers you number (web-01, web-02), and replace when sick; no individual attachment
Immutable infrastructure operationalises the "cattle" model at the infrastructure level.

Key vocabulary:
• Immutable infrastructure — a practice where deployed artifacts are never modified; changes require building and deploying a new version
• Configuration drift — divergence between the expected state of a server and its actual state after manual changes
• Golden image / AMI — a pre-baked, version-controlled machine image used as the base for all deployed instances
• Bake vs. fry — "baking" embeds config into the image at build time; "frying" applies config at boot time (Chef/Ansible) — immutable infrastructure prefers baking

2 / 5

A developer explains: "When our payment service's error rate exceeds 50% over a 10-second window, the circuit opens — we stop forwarding requests entirely and return a cached fallback instead of hammering a failing downstream service."

Which circuit breaker state transition is being described?

The circuit breaker pattern has three states that model the health of a downstream dependency.

State	Behaviour	Transition trigger
Closed (normal)	Requests flow normally; failures are counted	→ Open when error rate exceeds threshold
Open (tripped)	All requests fail immediately (fast-fail); fallback returned	→ Half-open after a configurable sleep window
Half-open (probing)	A limited number of probe requests are allowed through	→ Closed on success; → Open on failure

Why circuit breakers prevent cascading failure:
Without a circuit breaker, a slow payment service causes every caller to block until timeout — consuming threads, connections, and memory across the entire system. The circuit breaker provides fast-fail: failing requests in microseconds instead of seconds, freeing resources for other operations.

Fallback strategies when circuit is open:
• Return cached data (for read operations)
• Return a degraded response ("payment processing is temporarily unavailable")
• Queue the operation for retry later
• Redirect to an alternative service

Implementations: Hystrix (Netflix, deprecated), Resilience4j (Java), Polly (.NET), Envoy/Istio (service mesh layer — circuit breaking without code changes)

Key vocabulary:
• Circuit breaker — a resilience pattern that stops calls to a failing dependency when error rate exceeds a threshold
• Fast-fail — returning an error immediately without attempting the operation, preserving system resources
• Fallback — an alternative response provided when the primary path is unavailable
• Cascading failure — a downstream failure propagating upstream and causing the entire system to degrade

3 / 5

A platform team adds a Fluent Bit container as a sidecar in every application pod to collect, parse, and forward logs to the central log aggregator.

What is the primary architectural benefit of the sidecar pattern compared to embedding logging code directly in each application?

The sidecar pattern is the Kubernetes implementation of the single-responsibility principle applied to infrastructure concerns.

Sidecar pattern — what is separated from the application:

Sidecar type	Responsibility	Example
Logging agent	Tail logs from shared volume → parse → forward	Fluent Bit, Fluentd, Logstash
Service mesh proxy	mTLS, traffic shaping, retries, circuit breaking	Envoy (Istio/Linkerd)
Config sync	Fetch secrets from Vault → write to shared volume	Vault Agent, Secrets Store CSI driver
Metrics exporter	Scrape app metrics → expose Prometheus endpoint	Prometheus exporters

Operational benefits of the sidecar pattern:
• Independent versioning — upgrade Fluent Bit from 1.9 to 2.0 without touching application code or triggering an app test cycle
• Language agnosticism — the same Fluent Bit sidecar handles logs from Go, Python, and Java services identically
• Separation of concerns — application developers focus on business logic; platform engineers own the observability layer
• Consistent policy enforcement — every pod in the cluster gets the same logging config via a mutating admission webhook injecting the sidecar automatically

Related patterns:
• Ambassador — a sidecar that proxies network calls to external services (adds retry, circuit breaking, discovery)
• Adapter — a sidecar that transforms the app's output into a standard format (e.g., converts proprietary log format to JSON)

Key vocabulary:
• Sidecar pattern — co-locating a helper container in a pod to handle cross-cutting concerns independently from the main application
• Mutating admission webhook — automatically injects sidecar containers into new pods matching a label selector
• Cross-cutting concern — functionality (logging, security, observability) that spans multiple services and is better handled separately from business logic

4 / 5

After a major incident, a post-mortem reveals that slow database queries in one service caused thread starvation across the entire API gateway — all endpoints became unresponsive, even those with no database dependency.

An architect proposes partitioning the thread pool so that database-bound handlers cannot exhaust the threads reserved for fast API responses.

Which resilience pattern is this?

The bulkhead pattern is named after the watertight compartments in a ship's hull — flooding one compartment cannot sink the entire ship.

The incident without bulkhead isolation:

Shared thread pool: 200 threads
  Slow DB query: 198 threads blocked waiting for query result
  Fast API endpoints: 2 threads left → queue backs up → all requests time out

After applying the bulkhead pattern:

DB-bound pool:  50 threads  ← capped; slow queries only affect this pool
API fast pool: 150 threads  ← isolated; DB slowness has zero impact here

Bulkhead isolation dimensions:

Resource type	Bulkhead approach
Thread pools	Separate pool per downstream dependency (Hystrix/Resilience4j)
Connection pools	Separate DB connection pool per service consumer
Kubernetes	Separate node pools per criticality tier; PriorityClasses for scheduling
Semaphore bulkhead	Limit concurrent calls to a dependency without a dedicated thread pool

Bulkhead vs. circuit breaker — which pattern for which failure mode:
• Circuit breaker — protects against slow or failing downstream services by stopping requests after a threshold
• Bulkhead — protects against resource exhaustion spreading within your own service by isolating capacity allocations
In practice, both patterns are used together: bulkhead limits concurrent calls, circuit breaker stops calls when the service is failing.

Key vocabulary:
• Bulkhead pattern — isolating resource pools (threads, connections) so failure in one consumer cannot starve resources for others
• Thread starvation — a condition where no threads are available to process new requests because all threads are blocked waiting for slow operations
• Semaphore bulkhead — a lightweight bulkhead that limits concurrent executions using a counter rather than a dedicated thread pool

5 / 5

A platform engineer says: "Instead of scripting every step to deploy a service, we write a YAML manifest describing what the final state should look like — number of replicas, image version, environment variables — and the Kubernetes controller continuously ensures the cluster matches that description."

What configuration approach is this, and what operational benefit does it provide?

Declarative configuration is the foundational design principle of Kubernetes — the control loop pattern makes the cluster self-healing.

Imperative vs. declarative — the key distinction:

Property	Imperative	Declarative
You specify	How to achieve the goal ("create 3 pods", "scale to 5")	What the goal is ("replicas: 5")
Idempotent?	No — running twice may create duplicate resources	Yes — applying the same manifest twice is safe
Self-healing?	No — a crashed pod stays crashed until someone re-runs the command	Yes — the controller notices the pod is gone and creates a replacement
Audit trail	Script history; hard to review what changed	Git diff of YAML manifests; every change is a commit

How the Kubernetes reconciliation loop works:

loop forever:
  desired  = read manifest from etcd (e.g., replicas: 3)
  observed = count running pods matching the selector
  if desired != observed:
    take action (create/delete pods) to converge
  sleep(resync_period)

Practical benefit of declarative configuration:
• A pod crashes at 3am → the ReplicaSet controller notices and creates a replacement automatically — no on-call page for this
• A node fails → pods are rescheduled to healthy nodes within seconds
• A config change is a PR with a reviewer-approved YAML diff — fully auditable

Key vocabulary:
• Declarative configuration — specifying the desired end state rather than the steps to achieve it
• Control loop / reconciliation loop — a continuous process that compares desired and actual state and acts to close the gap
• Idempotent — an operation that produces the same result whether applied once or many times
• Self-healing — a system property where the runtime automatically corrects deviations from the desired state