Learn test quality metrics vocabulary: mutation testing, mutation score, killed mutants, branch coverage vs. line coverage, test coverage as vanity metric — the language of meaningful test quality measurement.
0 / 18 completed
1 / 18
The lead developer says: 'Test coverage is a vanity metric — 90% line coverage tells us nothing about test quality.' Why might this be true?
Line coverage (or statement coverage) measures what percentage of code lines are executed by the test suite. A test can execute a line without asserting the correct outcome. Therefore, high line coverage can give a false sense of security — the tests 'ran the code' but would not catch many real bugs. This is why experienced engineers call line coverage a vanity metric when used in isolation.
2 / 18
Your colleague explains: 'We use mutation testing to get a better measure of test quality.' What is mutation testing?
Mutation testing tools (e.g., Pitest for Java, Stryker for JavaScript/TypeScript) automatically create slightly modified versions of the source code (mutants) — for example, changing a '+' to '-', or flipping a boolean condition. They then run the test suite against each mutant. If the tests fail, the mutant is 'killed' (good). If the tests pass, the mutant 'survived' — indicating a gap in test quality.
3 / 18
The quality report shows: 'Mutation score: 78%. Killed mutant rate: 82%.' What does the mutation score represent?
The mutation score (killed mutant rate) is the percentage of generated mutants that were detected by the test suite (i.e., caused at least one test to fail). A score of 78% means 78% of the artificial code mutations were caught by the tests. The remaining 22% survived — meaning those code changes were not detected, indicating gaps in test assertions. A higher mutation score indicates a more effective test suite.
4 / 18
The tech lead says: 'We track branch coverage, not just line coverage.' What is branch coverage and why is it more meaningful?
Branch coverage (also called decision coverage) measures whether every possible branch from each conditional statement (if/else, switch/case, ternary) has been executed by at least one test. This is more meaningful than line coverage because a line containing an if/else only needs to be executed once to count toward line coverage, but branch coverage requires both the true and false paths to be tested.
5 / 18
During a code review, the developer says: 'The killed mutant rate is 82% — the remaining 18% are equivalent mutants or unkilled.' What is an 'equivalent mutant'?
An equivalent mutant is a code mutation that is syntactically different from the original but semantically identical — it does not change the observable behaviour of the program. For example, changing 'i = i + 1' to 'i += 1' produces an equivalent mutant. No test can kill an equivalent mutant because the behaviour is unchanged. Equivalent mutants are a known limitation of mutation testing and inflate the apparent 'unkilled' percentage.
6 / 18
PR Description
Subject: Refactor User Profile Service - Increased Test Coverage
Body:
"Implemented a comprehensive refactor of the user profile service. Increased test coverage to 95% (line coverage) and achieved an 80% killed mutant rate. This significantly improves reliability!"
This question assesses understanding of how line coverage and killed mutant rate relate. While high line coverage is desirable, it doesn't guarantee effective testing. The killed mutant rate – which measures how many mutants (modified versions of the code) were successfully detected by the tests – provides a more accurate reflection of test quality. A high killed mutant rate suggests the tests are actively finding and exposing flaws, while simply achieving high line coverage might mean the tests just cover a lot of code without actually testing its behavior effectively.
7 / 18
Sarah: 'Okay, the killed mutant rate is 80%, which is great! But I'm a little worried about the remaining 20% of mutants that weren't killed. What does that really *mean*?'
In a code review comment, what should David respond to address Sarah's concern?
Sarah is pointing out a potential issue with the interpretation of the killed mutant rate. A high 'killed mutant rate' (e.g., >90%) generally indicates good test quality because it means the tests are effectively finding flaws in the code. However, a remaining 20% suggests that there are still mutants – slightly modified versions of the code – that the current set of tests haven't been able to detect, highlighting areas where more focused testing is needed.
8 / 18
PR Description
Subject: Refactor User Profile Service - Increased Test Coverage
Body:
"Implemented a comprehensive refactor of the user profile service. Increased test coverage to 95% (line coverage) and achieved an 80% killed mutant rate. This significantly improves reliability!"
This question assesses understanding of how line coverage and killed mutant rate relate. While high line coverage is desirable, it doesn't guarantee effective testing. The killed mutant rate – which measures how many mutants (modified versions of the code) were successfully detected by the tests – provides a more accurate reflection of test quality. A high killed mutant rate suggests the tests are actively finding and exposing flaws, while simply achieving high line coverage might mean the tests just cover a lot of code without actually testing its behavior effectively.
9 / 18
Sarah: 'Okay, the killed mutant rate is 80%, which is great! But I'm a little worried about the remaining 20% of mutants that weren't killed. What does that really *mean*?'
In a code review comment, what should David respond to address Sarah's concern?
Sarah is pointing out a potential issue with the interpretation of the killed mutant rate. A high 'killed mutant rate' (e.g., >90%) generally indicates good test quality because it means the tests are effectively finding flaws in the code. However, a remaining 20% suggests that there are still mutants – slightly modified versions of the code – that the current set of tests haven't been able to detect, highlighting areas where more focused testing is needed.
10 / 18
PR Description
Subject: Refactor User Profile Service - Increased Test Coverage
Body:
"Implemented a comprehensive refactor of the user profile service. Increased test coverage to 95% (line coverage) and achieved an 80% killed mutant rate. This significantly improves reliability!"
This question assesses understanding of how line coverage and killed mutant rate relate. While high line coverage is desirable, it doesn't guarantee effective testing. The killed mutant rate – which measures how many mutants (modified versions of the code) were successfully detected by the tests – provides a more accurate reflection of test quality. A high killed mutant rate suggests the tests are actively finding and exposing flaws, while simply achieving high line coverage might mean the tests just cover a lot of code without actually testing its behavior effectively.
11 / 18
Sarah: 'Okay, the killed mutant rate is 80%, which is great! But I'm a little worried about the remaining 20% of mutants that weren't killed. What does that really *mean*?'
In a code review comment, what should David respond to address Sarah's concern?
Sarah is pointing out a potential issue with the interpretation of the killed mutant rate. A high 'killed mutant rate' (e.g., >90%) generally indicates good test quality because it means the tests are effectively finding flaws in the code. However, a remaining 20% suggests that there are still mutants – slightly modified versions of the code – that the current set of tests haven't been able to detect, highlighting areas where more focused testing is needed.
12 / 18
PR Description
Subject: Refactor User Profile Service - Increased Test Coverage
Body:
"Implemented a comprehensive refactor of the user profile service. Increased test coverage to 95% (line coverage) and achieved an 80% killed mutant rate. This significantly improves reliability!"
This question assesses understanding of how line coverage and killed mutant rate relate. While high line coverage is desirable, it doesn't guarantee effective testing. The killed mutant rate – which measures how many mutants (modified versions of the code) were successfully detected by the tests – provides a more accurate reflection of test quality. A high killed mutant rate suggests the tests are actively finding and exposing flaws, while simply achieving high line coverage might mean the tests just cover a lot of code without actually testing its behavior effectively.
13 / 18
Sarah: 'Okay, the killed mutant rate is 80%, which is great! But I'm a little worried about the remaining 20% of mutants that weren't killed. What does that really *mean*?'
In a code review comment, what should David respond to address Sarah's concern?
Sarah is pointing out a potential issue with the interpretation of the killed mutant rate. A high 'killed mutant rate' (e.g., >90%) generally indicates good test quality because it means the tests are effectively finding flaws in the code. However, a remaining 20% suggests that there are still mutants – slightly modified versions of the code – that the current set of tests haven't been able to detect, highlighting areas where more focused testing is needed.
14 / 18
During a standup meeting, Maria says: 'Our branch coverage is currently at 70%, which we're aiming for 90% by the end of the sprint. How does this relate to test quality?'
Branch coverage measures the proportion of branches in code that are exercised by tests. While high branch coverage is generally desirable, it doesn't guarantee good test quality – tests could still be superficial or not adequately cover critical scenarios. It's a proxy for thoroughness, but not a direct measure of test effectiveness.
15 / 18
You're reviewing a PR and see the following comment: 'The killed mutant rate is low (5%). This suggests our tests are effectively finding weaknesses in the code.' Is this statement accurate?
A low killed mutant rate doesn't necessarily mean good test quality. It indicates that the mutants used were *easy* for the tests to detect – these are often trivial changes. High-quality tests should identify more challenging and complex mutants, indicating a deeper understanding of potential vulnerabilities.
16 / 18
In a Slack message to the team, Ben writes: 'We've increased our test coverage to 98% using line coverage. Great job everyone!' What potential problem does this miss?
High line coverage alone doesn't guarantee good testing. It simply means that a large percentage of lines of code have been included in tests. However, tests might be superficial or not adequately cover critical scenarios – the *quality* of those tests is what matters most, something Ben's message ignores.
17 / 18
The API response from a test execution shows: 'Test quality score: 65. Metrics: Coverage (85%), Pass Rate (92%).' What does this suggest?
A low test quality score, despite high coverage and a pass rate, indicates that not all tests are effective. It suggests there might be issues with the design or execution of those tests, meaning they aren't actually uncovering meaningful defects.
18 / 18
A developer provides this PR description: 'Implemented new feature X. Increased test coverage to 90%.' What is a key element missing from this description that would improve its quality?
Details on the specific tests added and their purpose is a crucial element missing. A good PR description should explain *what* tests were added, *why* they were added (to cover which scenarios), and how those tests relate to the new feature. This provides reviewers with the information needed to assess the test quality.
What does the "Test Quality Metrics Vocabulary" exercise practise?
Learn test quality metrics vocabulary: mutation testing, mutation score, killed mutants, branch coverage vs. line coverage, test coverage as vanity metric — the language of meaningful test quality measurement.
How many questions are in this exercise?
This exercise has 18 questions, each multiple-choice with a full explanation shown after you answer.
What English level is this exercise for?
This exercise is tagged Intermediate. If the vocabulary feels difficult, browse the Code Quality & Metrics category page for an easier module to start with.
Is this exercise free to use?
Yes. Every exercise on CoderSlingo, including this one, is free with no account, sign-up, or paywall.
Do I get feedback if I answer incorrectly?
Yes — whichever option you choose, right or wrong, you'll immediately see an explanation clarifying the correct term and why the other options don't fit.
Can I retry this exercise?
Yes — once you finish all the questions, a "Try again" button on the results screen resets the exercise so you can practise as many times as you like.
Do I need an account to track my progress?
No account is required. Your progress bar and score for this session are tracked in the browser as you go, but nothing is saved once you leave the page.
Is "Test Quality Metrics Vocabulary" part of a larger series?
Yes — it's one exercise in the Code Quality & Metrics category on CoderSlingo. See the category page for the full list of related exercises on similar terminology.
Can I link directly to this exercise?
Yes — this exercise has its own permanent URL, so you can bookmark it or share the link directly with a colleague or study partner.
Where can I find more exercises like this one?
See the Code Quality & Metrics category page for related exercises, or browse the main Exercises hub for other IT English topics.