Chaos Engineering English: Vocabulary for GameDays and Resilience Testing

Chaos engineering is the practice of deliberately introducing failures into a system to discover weaknesses before they cause real incidents. Teams that practise chaos engineering have a specific vocabulary — and writing a clear experiment design document requires knowing it. This guide covers the core terms and communication patterns used in chaos engineering.

Core Chaos Engineering Vocabulary

Term	Definition
Steady-state hypothesis	A description of the system’s normal behaviour that you expect to hold true during the experiment
Blast radius	The scope of potential impact if the experiment causes unexpected harm
Fault injection	Deliberately introducing failures (latency, errors, resource exhaustion) into a system
GameDay	A scheduled event where a team runs chaos experiments together, often in production
Abort condition	A pre-defined trigger that stops the experiment if things go wrong unexpectedly
Hypothesis	Your prediction of how the system will behave under the introduced fault
Observability	The ability to understand the internal state of a system from its external outputs
Resilience	The ability of a system to absorb failures and continue to function
Fallback	A backup behaviour that activates when the primary path fails

The Steady-State Hypothesis

The steady-state hypothesis is the most important concept in chaos engineering. Before running any experiment, you must define what “normal” looks like — so you can tell whether the experiment has caused a problem.

Format: The steady-state hypothesis should be measurable and observable.

Good examples:

• “The checkout API returns HTTP 200 for 99.9% of requests with a p99 latency below 200 ms.”
• “The recommendation service returns a non-empty list for at least 95% of requests.”

Weak examples:

• “The system works normally.” (not measurable)
• “Users can complete purchases.” (not directly observable without instrumentation)

Describing Blast Radius

The blast radius determines how cautious your experiment needs to be. Narrow the blast radius before expanding.

Blast radius	Meaning
Single instance	Only one service instance is affected
Availability zone	An entire data centre zone is simulated as failed
Production traffic	Real user traffic is affected
Read-only traffic	Only read operations are affected, writes are protected

Phrases for blast radius discussions:

• “We’ll start with a blast radius of a single instance and expand to an AZ once we’re confident in the steady-state hypothesis.”
• “The blast radius must be limited to internal traffic for the first run to protect paying customers.”

Writing an Experiment Design Document

A chaos experiment design document typically includes:

1. Objective — Why are you running this experiment?
2. Steady-state hypothesis — What does normal look like?
3. Fault to be injected — What failure will you introduce?
4. Blast radius — What is the maximum potential scope of impact?
5. Abort conditions — When will you stop the experiment?
6. Observations — What metrics will you monitor?
7. Expected outcome — What do you predict will happen?
8. Actual outcome — What actually happened? (Filled in after the experiment)

Sample experiment objective: “Validate that the payment service degrades gracefully when the downstream fraud-detection service becomes unavailable, by verifying that transactions fall back to a safe-hold queue rather than failing with a 500 error.”

GameDay Vocabulary

A GameDay is a collaborative exercise where a team runs chaos experiments together. The language used before, during, and after a GameDay follows specific patterns.

Pre-GameDay:

• “We’ll be conducting a GameDay on Thursday — the target is the recommendation pipeline.”
• “Please ensure your on-call phone is active and that you’ve reviewed the abort conditions.”

During the GameDay:

• “Fault injection has been initiated. Monitoring dashboards are live.”
• “We’ve hit an abort condition — stopping the experiment now.”
• “The system is recovering as expected — steady-state metrics are returning to baseline.”

Post-GameDay:

• “The hypothesis held — fallback behaviour activated correctly within 3 seconds.”
• “We identified a previously unknown failure mode: the circuit breaker was not configured on the cache layer.”

Example Sentences

1. “The steady-state hypothesis for this experiment is that the API gateway maintains a p99 latency below 400 ms and a success rate above 99% under normal load.”
2. “We will limit the blast radius to a single availability zone and exclude production checkout traffic for the first iteration of this experiment.”
3. “The abort condition is triggered if the error rate on any customer-facing service exceeds 5% for more than 30 seconds.”
4. “During the GameDay, fault injection revealed that the retry logic in the notification service was not respecting the circuit breaker state, resulting in a thundering herd during recovery.”
5. “The experiment confirmed our hypothesis: when the inventory service is unavailable, the product page degrades gracefully by showing cached stock information rather than returning an error page.”

Navigating Nuance: Addressing Common Communication Challenges

For developers learning professional English, especially within a field like Chaos Engineering where precision is paramount, understanding subtle phrasing can be as crucial as mastering the core concepts. It’s not just about knowing what to say; it’s about conveying that meaning clearly and effectively, minimizing ambiguity that could lead to misinterpretations or wasted effort. A common challenge for non-native speakers is grappling with the often indirect ways technical issues are discussed – particularly when describing problems and proposing solutions. The language of resilience isn’t always straightforward, relying heavily on hypotheticals and potential scenarios.

Consider a scenario: During a code review, a senior engineer leaves a comment on a pull request: “This function seems to be exhibiting unexpected behavior under high load. Investigate the concurrency issues – it might be experiencing race conditions.” A developer unfamiliar with the specific terminology could easily misinterpret this as a simple bug report. The phrasing “exhibiting unexpected behavior” is deliberately vague, setting up a hypothesis that needs testing. Similarly, “race conditions” isn’t immediately obvious to someone not deeply familiar with concurrent programming concepts. The key here is recognizing this deliberate opacity – it’s a tactic used by experienced chaos engineers to encourage deeper investigation and proactive thinking. It demands more than just fixing the immediate symptom; it requires understanding the underlying system vulnerabilities.

Another frequent situation arises in Slack channels, perhaps when reporting an issue discovered during a GameDay simulation: “We triggered a cascade failure related to database connection limits. The blast radius appears to be impacting API calls for user authentication.” Again, terms like “cascade failure” and “blast radius” are loaded with meaning beyond their literal definitions. “Blast radius” isn’t just about physical damage; it represents the scope of an impact – how far a problem can spread within the system. When describing this in a PR description, you might say, “The simulation revealed a vulnerability where exceeding 10 concurrent database connections led to authentication API timeouts, demonstrating a blast radius affecting approximately 30% of user sessions.” This level of detail is vital for conveying the severity and potential consequences.

Finally, remember that documentation – especially when describing the steady-state hypothesis – often leans heavily on conditional language. Phrases like “under normal circumstances” or “assuming no external interference” are frequently used to frame scenarios. These aren’t meant to be restrictive; they’re designed to establish a baseline against which deviations can be reliably measured and analyzed. It’s about setting the stage for controlled experimentation.

# Example: Simulating database connection overload using `stress` (a common tool)
stress -c 15 -t 60 --vm-bytes 2G --io-blocksize 4k --vm-max-threads 100 /path/to/application

This example, while simple, demonstrates the kind of monitoring and logging that’s critical when validating a steady-state hypothesis. Observing resource utilization during this simulated overload allows you to quantify the potential impact of the identified vulnerability.