Code Review Metrics Vocabulary — Code Quality Exercises

1 / 5

An engineering manager presents data to the team: "Our average PR cycle time is currently 4.2 days. That means a developer opens a pull request and it takes over four days on average to get merged. We're targeting a cycle time of under 24 hours." What does PR cycle time measure, and why does it matter as a team metric?

PR cycle time is one of the most actionable team-level engineering metrics because it reveals exactly where the delivery pipeline is slow — and it is directly in the team's control to improve.

PR cycle time components (where the time is spent):

Stage	What it measures	Common cause of delay
Pickup time	PR open → first review comment	Reviewers overloaded; no SLA
Review time	First comment → "approved"	Large PR size; unclear scope
Rework time	Approval requests → re-submission	Unclear review comments; tech debt
Merge time	Final approval → merge	Manual merge gates; branch conflicts

Why sub-24-hour PR cycle time matters:
• Reduces context-switching cost — the author still has the code fresh in their mind
• Enables higher deployment frequency — faster merges mean more frequent releases
• Reduces merge conflicts — shorter-lived branches have less divergence from main
• Improves team flow — unblocked reviewers and authors maintain momentum

Communication language:
• "Our 4.2-day cycle time suggests we have a pickup time problem — PRs are sitting unreviewed."
• "Let's establish a 24-hour SLA for first review response to drive cycle time down."
• "Smaller PRs are the single biggest lever for reducing cycle time — aim for under 200 lines."

Key vocabulary:
• PR cycle time — total elapsed time from pull request opened to pull request merged
• Pickup time — the wait between PR opening and first reviewer engagement
• Cycle time target — the team's agreed-upon maximum acceptable PR cycle time
• Delivery pipeline — the series of stages code passes through from development to production

2 / 5

In a team retrospective, the tech lead introduces a new policy: "Starting next sprint, we're implementing a 24-hour SLA for code reviews. When a PR is opened and you are tagged as a reviewer, you must provide your first review within 24 hours — not necessarily approval, but substantive feedback." What does "review SLA" mean in this context, and why is it important to specify "substantive feedback"?

3 / 5

An engineering analytics platform surfaces this metric: "Review turnaround time: median 18 hours, 90th percentile 72 hours." The team lead says: "The median is acceptable, but the 90th percentile tells us there's a long tail of PRs that are getting stuck. We need to find what's causing that." Why are both the median and the 90th percentile important to look at?

4 / 5

A team introduces a new metric to their engineering dashboard: "First-time-right rate: 42% of PRs are merged without any change requests." The engineering manager says: "A 42% first-time-right rate tells us that well over half our PRs need at least one round of revision. This might indicate unclear requirements, insufficient self-review, or overly strict review standards — we need to investigate before concluding." What does first-time-right rate measure, and why does the manager caution against jumping to conclusions?

First-time-right rate is a nuanced metric — interpreting it correctly requires understanding that it can be driven by very different underlying causes with very different remediation strategies.

The four root causes of a low first-time-right rate:

Root cause	Signal	Intervention
Unclear requirements	Change requests are about scope, not code quality	Improve specification and definition of done
Insufficient self-review	Review catches obvious issues (typos, missing tests)	Author checklist before opening PR
Large PR size	Change requests correlate with PR line count	Enforce PR size limit (e.g. <300 lines)
Rigorous review standards	Change requests are substantive and improve code	No change needed — this is healthy

How to investigate which cause dominates:
• Categorise a sample of change requests: scope changes vs. quality improvements vs. style issues
• Correlate first-time-right rate with PR size — does it drop for large PRs?
• Ask authors: were the change requests expected or surprising?

Communication language:
• "Our first-time-right rate of 42% isn't inherently bad — we need to understand whether those revisions are adding value."
• "If reviewers are catching genuine bugs, a lower first-time-right rate reflects rigorous review, not poor quality."
• "Let's tag change requests by category this sprint to understand what's driving the revisions."

Key vocabulary:
• First-time-right rate — the percentage of PRs approved and merged without any change request round
• Change request — a reviewer's request for the author to modify code before approval
• Root cause analysis — investigating the underlying reason for a metric value before intervening
• Review quality — how thoroughly and constructively reviewers engage with submitted code

5 / 5

A senior engineer presents the team's quarterly review metrics report: "PR cycle time is down to 22 hours (target: 24 hours — achieved). Review turnaround SLA compliance is 91%. But our first-time-right rate dropped from 55% to 38%. This combination suggests we may be trading review quality for review speed." What does "trading review quality for review speed" mean, and how should the team respond?

"Trading review quality for review speed" describes a metric anti-pattern where optimising for a speed metric (cycle time, SLA compliance) inadvertently reduces review thoroughness — a classic Goodhart's Law failure.

Goodhart's Law in engineering metrics:
"When a measure becomes a target, it ceases to be a good measure." If reviewers know they are tracked on 24-hour response time, they may respond within 24 hours with superficial reviews to meet the SLA — while the actual review quality degrades.

The diagnostic pattern in this scenario:
• PR cycle time ↓ (faster) — reviewers are responding quickly ✓
• SLA compliance ↑ (91%) — reviewers are meeting the time target ✓
• First-time-right rate ↓ (55% → 38%) — more PRs are requiring rework after "approval" ✗
This combination strongly suggests reviewers are approving code without catching issues that are then discovered post-merge.

Corrective actions:
① Audit post-merge defects: are bug fixes concentrating in recently merged PRs?
② Review calibration: run a session where reviewers all review the same PR and compare findings
③ Author checklist: require authors to self-review before opening a PR (reduces avoidable revision rounds without degrading review quality)
④ Balance the dashboard: display first-time-right rate alongside SLA compliance so reviewers see both metrics

Communication language:
• "We've achieved our speed target but it looks like review depth may have suffered as a result."
• "Metrics need to be balanced — optimising speed alone can erode quality in ways that show up later as bugs."
• "Let's look at which PRs had post-merge issues this quarter to validate the hypothesis."

Key vocabulary:
• Trading review quality for review speed — a pattern where speed metrics improve at the cost of review thoroughness
• Goodhart's Law — the principle that a measure loses validity when it becomes a target
• Review quality signal — a metric or process indicator reflecting how thoroughly PRs are reviewed
• Post-merge defect — a bug found in production that was introduced in a merged PR; a lagging indicator of review quality