Runbook Writing — Vocabulary and Operational Language

A runbook is an operational recipe: step-by-step procedures for specific scenarios (deploy, rollback, incident response, maintenance). Good runbooks are written so a new on-call engineer unfamiliar with the system can follow them successfully at 3am under pressure. They are living documents updated after every use.

Pre-conditions for a runbook: 'System is in healthy state except for X. You have SSH access to production servers. Deployment pipeline is green. You have reviewed and understood the full procedure before beginning.' Pre-conditions prevent operators from starting a procedure they cannot safely complete.

Every runbook for changes (deployments, config changes, migrations) needs a rollback procedure: 'If Step 5 fails or the system behaves unexpectedly after Step 7, execute the following to restore the previous state: [steps].' Rollback procedures must be tested. A runbook without a rollback is dangerous for production operations.

Verification steps prevent operator errors from compounding: 'VERIFY: The deployment has completed. Run `kubectl rollout status deployment/api` — you should see: successfully rolled out. Do NOT proceed to the next step until this verification passes.' Explicit verification gates turn a procedure into a safe, auditable process.

Escalation path: 'If the procedure does not resolve the issue within 15 minutes, escalate to the on-call backend lead via PagerDuty. Include: timeline of actions taken, current system state, and relevant log snippets. Do not continue attempting this procedure without guidance.' Runbooks should explicitly tell operators when to stop and get help.