Documentation Testing Vocabulary

Running docs code samples in CI is the highest-leverage practice for keeping documentation accurate — it converts documentation maintenance from a human memory task to an automated quality gate.

Implementation approaches: (1) Doctest (Python) — executable examples embedded in docstrings, run by pytest; (2) mdBook / runbook tools — markdown code blocks marked as "runnable" are extracted and executed; (3) Custom test harnesses — scripts that extract code blocks from markdown by language/tag and execute them against a test API environment; (4) Jupyter notebooks — entire documentation notebooks executed as part of CI. The key insight: outdated code samples are the most-complained-about documentation problem because they directly waste developer time. If the first code sample a developer tries doesn't work, trust in the documentation collapses. Automation converts this from "we try to remember to update docs when the API changes" to "the build fails if docs aren't updated."

Key vocabulary:
• docs-as-code — treating documentation with the same tooling and practices as software code (version control, CI, tests, code review)
• runnable code sample — a documentation code example that is also executed as part of CI to verify its correctness
• documentation CI pipeline — automated checks (link checking, code execution, spell checking) run on documentation changes

A broken link is documentation's equivalent of a 404 in production — it breaks the user's flow and signals that the documentation is not actively maintained, eroding trust in the accuracy of everything else.

Link checking tools: htmltest (Go, fast), lychee (Rust, supports parallel checking), Broken Link Checker (Node.js), GitHub Actions has built-in link checking. Best practice: run on every PR for fast feedback, and also on a daily schedule to catch externally-broken links that aren't caused by your own changes. Configuration considerations: exclude known external sites that block crawlers (some documentation sites return 403 to bots), configure appropriate timeout thresholds, and distinguish errors (true broken links) from warnings (redirect chains, slow responses). The "zero 404s" target mentioned in the exercise question is a maintainability standard — it means the team has committed to treating broken links as build failures, not just warnings.

Key vocabulary:
• broken link checker — an automated tool that verifies all hyperlinks in a documentation site return valid responses
• 404 error — an HTTP response indicating a requested page does not exist; in docs, signals unmaintained or relocated content
• anchor link — a link to a specific section within a page (using a # fragment); breaks when the section heading changes

Auto-generating screenshots converts documentation image maintenance from a human-memory task to an automated process — every UI change that breaks a screenshot is either fixed automatically or surfaced as a CI failure.

Implementation approaches: (1) Playwright/Puppeteer scripts that navigate to specific UI states and capture screenshots — run in CI on each UI deployment; (2) Percy or Chromatic visual testing — captures screenshots and diffs against baselines, highlighting changes; (3) Storybook documentation — component screenshots auto-generated from the component library. The most sophisticated implementations parameterise screenshots: the same script captures the same UI state across multiple locales, screen sizes, and themes — generating the full matrix of documentation images automatically. The key technical challenge is making the UI state reproducible: the script must log in, navigate to the correct state, populate the correct data, and capture consistently.

Key vocabulary:
• auto-generated screenshot — a documentation image captured programmatically by a script rather than manually by a human
• visual testing — automated comparison of UI screenshots against approved baselines to detect unintended visual changes
• UI state reproducibility — the ability to consistently navigate a UI to a specific state for automated screenshot capture

Readability testing gives technical writers objective feedback on prose clarity — it is not about dumbing down technical content but about expressing technical ideas in the clearest, most efficient language possible.

The Hemingway Editor is popular for technical writing review: it highlights passive voice (often clearer as active), adverbs that can be cut, complex words with simpler alternatives, and sentences that are "hard to read" or "very hard to read." Google's developer documentation style guide specifically advises: use active voice, use second person ("you"), write short sentences, and avoid unnecessary hedging words. Readability targets for developer documentation typically allow somewhat lower scores than consumer content — technical precision sometimes requires longer sentences. But consistently "very hard to read" passages in procedural documentation indicate a writing quality problem that will cause developers to abandon the docs and resort to trial and error.

Key vocabulary:
• readability score — a metric quantifying how easy prose is to understand, based on sentence and word length measures
• Flesch-Kincaid grade level — a readability formula that estimates the US school grade level needed to understand the text
• active voice — sentence construction where the subject performs the action ("click the button") rather than passive ("the button should be clicked")

Zero 404s is a documentation quality commitment, not a one-time achievement — it requires automated monitoring, a redirect strategy for moved content, and a team culture where broken links are treated as build failures.

The redirect strategy is the most important element for established documentation sites: when a page is renamed or restructured, the old URL must redirect to the new location. Without this, every external link and search engine bookmark pointing to the old URL becomes a 404. Most documentation frameworks support redirects in configuration (Docusaurus, MkDocs, Hugo, Jekyll all have redirect plugins). A stable URL structure policy — agreed patterns for how pages are named that don't need to change — reduces the volume of redirects needed. For very large documentation sites, a "404 monitoring" pipeline that checks the site map against a crawl-based link inventory helps catch gaps that CI doesn't cover (e.g., links in content templates or dynamically generated pages).

Key vocabulary:
• redirect — an HTTP response that sends the browser from an old URL to a new location; preserves incoming links when content moves
• stable URL structure — a documentation URL naming convention designed to not require changes, reducing redirect maintenance
• sitemap — an index of all pages in a documentation site, used by search engines and link checkers to discover all URLs