Chaos Hypothesis — Language and Vocabulary

Chaos hypothesis format (Principles of Chaos Engineering): state what 'normal' looks like in measurable terms, then assert it will hold under a specific failure. Example: 'In steady state, the checkout service processes ≥ 95% of requests with latency < 500ms. We hypothesize this will hold even when the payment service returns 503 errors for 30% of requests — because the circuit breaker should open and serve cached responses.' This makes the experiment falsifiable.

Steady state is defined BEFORE the experiment, not assumed. 'The system is healthy' is not a steady state definition. 'p99 checkout latency < 300ms AND error rate < 0.5% AND order completion rate > 99%' is a steady state definition. You verify steady state before the experiment (baseline), during (does it hold?), and after (did it recover?). If you can't measure it, you can't run chaos experiments against it.

Control group vs. experiment group (borrowed from scientific method): the control group experiences no injected fault; the experiment group experiences the fault. By comparing their behavior, you can attribute differences to the fault rather than to coincidental changes (deployments, traffic spikes). In practice: route 5% of traffic through instances with the fault injected (experiment group), compare to the other 95% (control group).

Confidence level in chaos engineering: since systems have natural variability, you need enough sample size and duration to distinguish 'the fault caused degradation' from 'normal variance.' Run experiments long enough, use statistical significance tests, and define confidence thresholds before the experiment. Low confidence = more samples needed. Also used informally: 'our confidence in this system's resilience increased after the experiment confirmed the hypothesis.'

Falsifiability is a requirement for good chaos hypotheses (Karl Popper's scientific method applied to engineering). 'The system will be fine' is not falsifiable — 'fine' is undefined. 'p99 latency will remain below 400ms' is falsifiable — if it reaches 600ms, the hypothesis is falsified. A falsified hypothesis is not a failure; it reveals a real resilience gap that can be fixed. Unfalsifiable hypotheses give false confidence.