Learn the vocabulary of measuring whether a test suite actually catches an introduced bug, not just runs the code.
0 / 5 completed
1 / 5
At standup, a dev mentions a tool that deliberately introduces a small, artificial bug into the code and reruns the test suite to see whether any test actually catches it. What is this technique called?
Mutation testing deliberately introduces a small, artificial bug, called a mutant, into the code and reruns the test suite to see whether any test actually catches, or kills, that mutant. Code coverage measurement only tracks which lines of code a test suite executes, saying nothing about whether the assertions in that test would actually catch a real bug in that line. This deliberate-bug-and-rerun approach is what mutation testing uses to measure a test suite's real effectiveness, not just its reach.
2 / 5
During a design review, the team wants a metric that reflects the percentage of introduced mutants a test suite actually catches, rather than just the percentage of lines it happens to execute. Which capability supports this?
A mutation score reports the percentage of introduced mutants a test suite actually catches, giving a much more direct signal of test quality than a metric that only tracks which lines were executed. A line-coverage percentage can be high even if the executed lines have no meaningful assertion checking their actual behavior. This mutation score is what lets a team distinguish a test suite that runs code from one that genuinely verifies it.
3 / 5
In a code review, a dev notices a function has 100% line coverage yet a specific mutant introduced into one of its conditional branches still survives, meaning the test suite never noticed the branch's logic had been flipped. What does this represent?
A surviving mutant reveals a weak or missing assertion in the test suite, since the tests still passed even though the conditional branch's actual logic had been flipped, despite the function showing full line coverage. A killed mutant would mean the test suite correctly detected and failed on that same introduced change. This gap between full line coverage and a surviving mutant is exactly the blind spot mutation testing is designed to expose that coverage percentage alone can't.
4 / 5
An incident report shows a real regression shipped to production undetected in a function that had 100% line coverage, and a later mutation-testing run confirmed the exact same class of change would have survived as an unkilled mutant. What practice would prevent this recurring?
Running mutation testing regularly and strengthening any test whose corresponding mutant survives closes exactly the gap that let the real regression in this incident slip past a test suite with full line coverage. Continuing to rely solely on line-coverage percentage, with no mutation testing, leaves that same blind spot open indefinitely. This regular practice is what turns a coverage-focused test suite into one that genuinely catches a meaningful class of regression.
5 / 5
During a PR review, a teammate asks why the team runs mutation testing in addition to already tracking a high line-coverage percentage. What is the reasoning?
Line coverage only shows which lines were executed during a test run, saying nothing about whether the test's own assertions would actually catch a real bug introduced into that line. Mutation testing directly measures whether an introduced bug is caught, giving a far more direct signal of a test suite's real effectiveness. The tradeoff is the added computational cost of mutation testing, since it reruns the entire test suite once for every mutant introduced across the codebase.