Testing: flaky test, test quarantine, flakiness rate, test sharding
0 / 5 completed
1 / 5
A DevSecOps engineer explains pipeline security stages: "We run three scanners on every pull request. The first analyses source code without executing it — catching SQL injection patterns, hardcoded secrets, and insecure crypto. The second hits the running app with real HTTP requests to find runtime vulnerabilities. The third checks our package.json against CVE databases for vulnerable dependencies." Which scanner is described by "analyses source code without executing it"?
SAST (Static Application Security Testing): analyses source code, bytecode, or binaries without executing the application. Runs early in the pipeline (on push/PR). Catches: SQL injection patterns, XSS vulnerabilities, hardcoded credentials, insecure crypto usage. Tools: Semgrep, SonarQube, Checkmarx, Bandit (Python). DAST (Dynamic Application Security Testing): tests the running application by sending real HTTP requests and observing responses. Finds runtime issues SAST cannot detect: authentication bypasses, server-side request forgery. Tools: OWASP ZAP, Burp Suite. SCA (Software Composition Analysis): scans third-party dependencies (npm, Maven, pip) against CVE databases. Tools: Snyk, OWASP Dependency-Check, Trivy. Quality gate: a pipeline stage that fails the build if security or quality metrics breach a defined threshold — e.g., "zero critical CVEs" or "coverage > 80%". In conversation: "SAST caught the SQL injection pattern statically; DAST later confirmed it was exploitable against the running app — both tools together give full coverage."
2 / 5
Complete the platform engineer's explanation using the correct term:
"We store all Docker images in our private ___. We enforce immutable tags — once myapp:v1.2.3 is pushed it cannot be overwritten. We pin Kubernetes manifests to the image digest (SHA-256 hash of the image content), so even if a tag is moved our pods keep running the exact image we tested."
Which term fills the blank?
Artifact registry: a versioned, access-controlled repository for build outputs — container images (Docker/OCI), language packages (npm, Maven), binaries, and Helm charts. Examples: Docker Hub, AWS ECR, Google Artifact Registry, GitHub Container Registry, JFrog Artifactory, Harbor (open-source). Immutable tag: a registry policy preventing an existing tag from being overwritten. Prevents silent supply chain attacks where a compromised pipeline replaces a known-good image with the same tag name. Image digest: the SHA-256 hash of an image manifest. Pinning to a digest (e.g. myapp@sha256:abc123...) guarantees the exact image content regardless of tag changes — even if a tag is later moved to a different image, digest-pinned deployments are unaffected. Deployment gate: a pipeline checkpoint requiring a manual approval or automated quality check before promoting a build to the next environment (e.g. staging → production). Different from a quality gate — a deployment gate is environment-level, a quality gate is metric-level. In conversation: "After a supply chain incident we enforced digest pinning everywhere — tags are human-friendly aliases, but only the digest guarantees immutability."
3 / 5
Match the correct definition to the term rollback trigger:
An engineering manager describes the release process: "We promote our canary to 100% automatically unless a rollback trigger fires. We define thresholds: if error rate exceeds 1%, or p99 latency rises above 500 ms, the system automatically reverts to the previous version without human intervention."
Rollback trigger: an automated condition — typically a metric threshold — that initiates reversion to the previous deployment when breached. Part of progressive delivery pipelines. Common triggers: error rate spike, latency p99 increase, success rate drop, custom business metric anomaly. Tools: Argo Rollouts analysis templates, Flagger metric checks, Spinnaker automated judgements. Related vocabulary: Percentage rollout — gradually shifting traffic from 0% to 100% for a new version, with rollback triggers guarding each step. Kill switch — a feature flag specifically designed for emergency disablement; distinct from a rollback trigger because it operates at the feature level (not the deployment level) and takes effect without a new deployment. Feature toggle / flag debt: stale flags never cleaned up after a rollout completes; become a maintenance burden and source of hidden complexity. Every flag should have a planned removal date. In conversation: "The rollback trigger fired at 2% traffic when error rate hit 0.8% — the system rolled back automatically before any customer noticed."
4 / 5
A tech lead explains the branching strategy: "We practise trunk-based development. Everyone integrates to main at least once a day. Feature branches live no longer than two days. Long-running work is hidden behind feature toggles so the trunk is always releasable. We also use a merge queue — it serialises merges and runs CI on each in order, so a passing branch that conflicts with a concurrent merge can never silently break main." What problem does a merge queue solve?
Merge queue: a mechanism that serialises merges to the main branch. When a PR is added to the queue: (1) it is rebased on the current main plus any queued PRs ahead of it, (2) CI runs on this combined state, (3) only if CI passes does the merge complete. This prevents the "works on my branch" problem where two passing branches conflict when merged concurrently. Supported natively by GitHub (merge queue feature), GitLab (merge trains), and Aviator/Mergify. CODEOWNERS: a file (`.github/CODEOWNERS` on GitHub) mapping repository paths to teams or individuals who must approve changes. Example: `src/billing/` owned by `@payments-team`. Enforces ownership without manual tracking. Trunk-based development (TBD): a branching model requiring all developers to integrate to a single shared trunk (main/master) at least daily, using feature flags to hide incomplete work. Enables continuous integration and DORA Elite performance. In conversation: "Before the merge queue, two teams could both see green CI and merge simultaneously — and break main. The queue catches that."
5 / 5
A senior engineer discusses release management hygiene: "We have a flag debt problem. Eighteen months ago we launched a percentage rollout for the new checkout flow. It reached 100% and was never cleaned up. Now we have 47 dead feature flags cluttering the codebase. The code paths for the old checkout still exist, still tested, still confusing new developers. Meanwhile, our flaky test count is at 89 — engineers routinely re-run CI without investigating failures, which means real bugs are getting ignored." What is flag debt and what is the risk it creates?
Flag debt: the accumulation of feature flags (toggles) that have outlived their purpose — fully rolled out but never removed. Causes: dead code branches that still require maintenance, confusing conditional logic that new developers must understand, test matrix explosion (which code paths are active?), latent bugs in the "off" path that nobody tests. Best practice: every flag has a removal ticket created at creation time with a target date. Flaky test: a non-deterministic test that sometimes passes and sometimes fails without any code change. Causes: shared mutable state between tests, timing dependencies (sleep() instead of polling), unmocked external calls, random data without fixed seeds. Harm: trust erosion — engineers learn to re-run CI without investigating, which means real failures get treated as flakiness and ignored. Mitigation: quarantine flaky tests (still run, do not block merges), track flakiness rate as a quality metric, treat each flaky test as a P2 bug. In conversation: "We treat flag debt the same as technical debt — it appears in our quarterly tech health report alongside flaky test count."