English for Writing Mutation Test and Property-Based Test Descriptions

Mutation testing and property-based testing are less common than unit or integration testing, so when they come up in a PR description or a testing strategy discussion, the vocabulary needs to be precise — your reader may not already share the mental model. This guide covers the English for describing these techniques clearly, whether you’re introducing them to a team or reporting results.

Key Vocabulary

Mutation testing — a technique that introduces small deliberate bugs (“mutants”) into the code and checks whether the test suite catches them, used to measure how effective tests actually are, not just how much code they cover. “We ran mutation testing on the billing module and found that 30% of mutants survived, meaning our tests didn’t actually catch those introduced bugs despite showing 95% line coverage.”

Mutant — a single deliberately introduced code change (like flipping a comparison operator or changing a constant) used to test whether the test suite notices. “One surviving mutant changed >= to > in the discount threshold check, and no test failed — that’s a real gap in our coverage.”

Killed / survived — a mutant is “killed” if a test fails because of it (good — the test caught the bug); it “survives” if all tests still pass (bad — the test suite missed it). “87% of mutants were killed, which is a reasonable mutation score, but the 13% that survived are worth investigating individually.”

Property-based testing — a testing approach where you specify a general property that should always hold (an invariant) and the test framework generates many random inputs to try to find a counterexample, rather than testing a fixed set of examples. “Instead of writing ten example-based test cases for the sorting function, we wrote one property-based test asserting that the output is always sorted and contains the same elements as the input, and let the framework generate hundreds of random inputs.”

Invariant — a property that should always be true regardless of input, which property-based tests are built around. “The invariant we’re testing is that decoding an encoded value always returns the original value — that should hold for any input, not just the examples we thought of.”

Common Phrases

• “Our line coverage is high, but mutation testing revealed several gaps in what our tests actually verify.”
• “This mutant survived because no test asserts on the return value in this branch.”
• “We wrote a property-based test instead of enumerating edge cases by hand.”
• “The invariant here is that the function is idempotent — applying it twice gives the same result as applying it once.”
• “The framework found a counterexample we hadn’t considered: an empty string with only whitespace.”

Example Sentences

Reporting mutation testing results in a PR or ticket: “Ran mutation testing on the discount-calculation module: mutation score is 78%. The surviving mutants are concentrated in the edge cases around zero and negative quantities, which suggests our tests don’t actually exercise those paths despite being listed as covered.”

Explaining why property-based testing was chosen over example-based tests: “Rather than writing individual test cases for each input format, we defined the invariant that parsing and then re-serializing should always produce an equivalent value, and let the property-based framework generate a wide range of inputs to test that invariant directly.”

Describing a bug the technique found: “Property-based testing found a counterexample we’d never have written by hand: a deeply nested object with a circular reference caused the serializer to recurse infinitely. We’ve added a fix and a regression test for this specific case.”

Introducing the technique to a team unfamiliar with it: “Mutation testing works differently from coverage — instead of asking ‘did any test run this line,’ it asks ‘would any test actually notice if this line were wrong.’ That’s a much stronger signal for whether our tests are doing their job.”

Professional Tips

• When reporting mutation testing results, report the mutation score as a number, and describe where surviving mutants are concentrated — a raw percentage alone isn’t actionable.
• Explain property-based testing by contrasting it directly with example-based testing — most readers already understand example-based tests, so the contrast builds understanding fast.
• Use the word “invariant” deliberately — it signals “a property that always holds,” which is more precise than “a rule” or “a condition.”
• When property-based testing finds a bug, call the failing input a “counterexample” — it’s the standard term and immediately communicates that the framework disproved your assumption.
• If introducing either technique to a team for the first time, lead with a concrete example of a bug it caught that existing tests missed — abstract explanations of the technique are much less persuasive than a real result.

Practice Exercise

1. Write a sentence reporting a mutation score and where the surviving mutants are concentrated.
2. Write a sentence stating an invariant for a hypothetical function, in the form “X should always be true regardless of input.”
3. Write two sentences introducing property-based testing to a teammate by contrasting it with example-based testing.