Software Testing Vocabulary

42 testing terms in plain English — what each one means, a code example, and the gotcha worth knowing. From "what's the difference between a mock and a stub" to reading a coverage report.

Last reviewed:

Sections

Core concepts

unit test

Tests a single function or method in isolation from the rest of the system — every dependency is faked or removed.

test('add sums two numbers', () => {
  expect(add(2, 3)).toBe(5);
});

💡 Fast and cheap — should make up the bulk of your test suite (see "test pyramid").

integration test

Verifies that two or more real units — a service and a real database, two microservices — work together correctly.

test('creating a user persists to the real DB', async () => {
  await createUser({ email: 'a@b.com' });
  expect(await db.users.findOne({ email: 'a@b.com' })).toBeTruthy();
});

💡 Slower and more brittle than unit tests, but catches bugs unit tests structurally cannot.

end-to-end (e2e) test

Exercises the whole system the way a real user would — clicking through the UI, hitting the real backend, the real database.

await page.goto('/login');
await page.fill('#email', 'a@b.com');
await page.click('button[type=submit]');
await expect(page).toHaveURL('/dashboard');

💡 Slowest and most valuable-per-test, but also the most expensive to write and maintain.

test suite

A collection of related test cases — usually every test in one file, or every test for one feature.

describe('UserService', () => {
  test('creates a user', () => { /* ... */ });
  test('rejects a duplicate email', () => { /* ... */ });
});

💡 "The suite is red" means at least one test in it is failing.

test case

A single scenario with a specific input and an expected outcome — the smallest unit a test runner reports on.

test('returns 0 for an empty array', () => {
  expect(sum([])).toBe(0);
});

💡 A good test case name describes the behaviour, not the implementation.

assertion

A statement that checks an actual value against an expected one and fails the test if they don't match.

expect(response.status).toBe(200);
assert.strictEqual(user.name, 'Alice');

💡 One logical assertion per test keeps failures easy to diagnose.

fixture

A fixed, known piece of test data or environment state that is set up before a test runs.

beforeEach(() => {
  testUser = { id: 1, name: 'Alice', role: 'admin' };
});

💡 Shared fixtures save typing but can hide what a test actually depends on — keep them small.

setup / teardown

Code that runs before (setup) and after (teardown) each test to prepare and then clean up state.

beforeEach(() => db.connect());
afterEach(() => db.clear());

💡 Teardown that never runs (a test that crashes before it) is a classic cause of flaky tests.

Test doubles

test double

The umbrella term for any object that stands in for a real dependency during a test — mocks, stubs, spies, fakes, and dummies are all test doubles.

// "test double" is the category; the specific kind matters for what you can assert

💡 Borrowed from film — a stunt double stands in for the real actor.

mock

A test double that records how it was called, so you can assert the interaction itself, not just the return value.

const sendEmail = jest.fn();
await notifyUser(sendEmail);
expect(sendEmail).toHaveBeenCalledWith('a@b.com', 'Welcome!');

💡 Overusing mocks couples your tests to implementation details — assert behaviour, not every call.

stub

A test double that returns hardcoded, canned responses without any real logic behind them.

const getUser = () => ({ id: 1, name: 'Alice' }); // stub — no real DB call

💡 A stub answers questions; a mock also remembers what you asked it.

spy

Wraps a real function or object and records calls to it while (usually) still calling through to the real implementation.

const spy = jest.spyOn(analytics, 'track');
checkout();
expect(spy).toHaveBeenCalledWith('purchase_completed');

💡 Use a spy when you want the real side effect to happen but also need to verify it happened.

fake

A working but simplified implementation of a dependency — e.g. an in-memory database instead of a real Postgres instance.

class InMemoryUserRepo {
  #users = new Map();
  save(u) { this.#users.set(u.id, u); }
  find(id) { return this.#users.get(id); }
}

💡 Fakes are more work to build than mocks/stubs but give you far more realistic tests.

dummy

A placeholder object passed only to satisfy a function's signature — the test never actually uses it.

createInvoice(order, /* logger */ null); // dummy — this test doesn't check logging

💡 The simplest test double: it exists to fill a parameter slot, nothing more.

Test types & strategy

smoke test

A quick, shallow check that the most critical paths work at all — run before investing time in the full suite.

test('app boots and the homepage returns 200', async () => {
  const res = await fetch('http://localhost:3000/');
  expect(res.status).toBe(200);
});

💡 Named after hardware testing: power it on and check for smoke before deeper diagnostics.

regression test

A test written specifically to make sure a previously fixed bug never comes back.

// Bug #482: negative quantities crashed checkout
test('rejects a negative quantity', () => {
  expect(() => addToCart({ qty: -1 })).toThrow();
});

💡 Best practice: write the failing regression test first, confirm it fails, then fix the bug.

snapshot test

Captures the current output — often a rendered component — and fails on any future run that produces a different output.

expect(render(<Button label="Save" />)).toMatchSnapshot();

💡 Cheap to write but easy to rubber-stamp ("update snapshot") without reading the diff — use sparingly.

mutation testing

Deliberately introduces small bugs ("mutants") into the code — flips a `<` to `<=`, removes a line — to check whether the test suite actually catches them.

# Stryker / mutmut reports a "mutation score":
# 87% of mutants were killed by the test suite

💡 Answers "is my coverage number actually meaningful?" — high line coverage can still miss real bugs.

flaky test

A test that passes and fails intermittently with no code changes — usually caused by timing, execution order, or shared state.

// Flaky: assumes the animation finished in exactly 300ms
await sleep(300);
expect(el).toBeVisible();
// Fix: wait for the actual condition, not a fixed delay
await waitFor(() => expect(el).toBeVisible());

💡 A quarantined/retried flaky test is technical debt — it hides real failures behind "just re-run it".

test pyramid

The guideline that a healthy suite has many fast unit tests, fewer integration tests, and very few slow e2e tests.

      /\   e2e (few)
     /--\  integration (some)
    /----\ unit (many)

💡 An "ice cream cone" anti-pattern — mostly e2e, almost no unit tests — is slow and flaky by nature.

black-box testing

Testing based only on inputs and outputs, with no knowledge of (or dependency on) the internal implementation.

test('POST /users returns 201 with a Location header', async () => {
  // no knowledge of the handler's internals required
});

💡 Survives refactors that don't change behaviour — the classic argument for testing "what", not "how".

white-box testing

Testing that uses knowledge of the internal code structure — branches, loops, edge cases — to design test cases.

// Knows the function has an early-return for a null input
test('returns null immediately without hitting the DB', () => { /* ... */ });

💡 Good for hitting hard-to-reach branches; brittle if it tests internals instead of behaviour.

acceptance test

Confirms a feature meets the actual business requirement, often written from the user's perspective.

Given a logged-in user with items in their cart
When they complete checkout
Then they should see an order confirmation

💡 The Given/When/Then format (Gherkin) is common in BDD tools like Cucumber.

contract test

Verifies that a service's API matches what its consumers actually expect, without spinning up the whole downstream system.

// Pact: consumer defines the expected request/response,
// provider verifies it can satisfy that contract in CI

💡 Catches breaking API changes early in microservice architectures, cheaper than full e2e.

Assertions & matchers

matcher

A method that expresses an expectation about a value — toBe, toEqual, toContain, and dozens more.

expect(items).toContain('apple');
expect(response.body).toMatchObject({ ok: true });

💡 A good matcher choice produces a readable failure message — prefer toEqual over manual comparisons.

toBe / toEqual

toBe checks reference/primitive equality (Object.is); toEqual checks deep structural equality, recursing into objects and arrays.

expect({ a: 1 }).not.toBe({ a: 1 });   // different references
expect({ a: 1 }).toEqual({ a: 1 });     // same shape — passes

💡 The #1 beginner bug in Jest tests: using toBe on an object and getting a confusing failure.

truthy / falsy

Matchers that check whether a value is truthy or falsy, rather than matching a specific value.

expect(user).toBeTruthy();
expect(errors.length).toBeFalsy();

💡 Looser than toBe(true) — 1, "x", and {} are all truthy; 0, "", null, undefined are falsy.

toThrow

Asserts that a function throws an error when called — the function itself, not its result, must be passed in.

expect(() => parseConfig('{bad json')).toThrow(SyntaxError);

💡 Common mistake: expect(parseConfig(x)).toThrow() — this calls the function immediately, before the assertion runs.

deep equality

Comparing two objects or arrays by their contents rather than by memory reference.

deepEqual({ a: [1, 2] }, { a: [1, 2] }); // true — same content, different objects

💡 Every mainstream assertion library has a deep-equal matcher; reaching for JSON.stringify() comparisons is a common workaround anti-pattern.

golden file

A saved reference output file that test output is diffed against — similar to a snapshot, but stored as a real file on disk.

const actual = renderReport(data);
const expected = fs.readFileSync('testdata/report.golden.txt', 'utf8');
expect(actual).toBe(expected);

💡 Common in Go and CLI-tool testing; update goldens deliberately, not on every red run.

Coverage & metrics

code coverage

The percentage of code actually executed by the test suite — a proxy for how much of the codebase is tested at all.

# jest --coverage
Statements   : 87.3% ( 1042/1194 )

💡 100% coverage does not mean bug-free — it only means every line ran, not that every case was checked.

line coverage

The percentage of source lines executed at least once by the test suite.

Lines        : 91.2% ( 512/561 )

💡 The most common (and most misleading) coverage number — easy to game with shallow tests.

branch coverage

The percentage of possible branches — every if/else path, every ternary — actually exercised by the tests.

function discount(user) {
  if (user.isPremium) return 0.2;  // branch A
  return 0;                          // branch B — is this ever tested?
}

💡 A much stricter, more useful signal than line coverage alone.

coverage threshold

A minimum coverage percentage that a CI pipeline requires before it allows a merge.

// jest.config.js
coverageThreshold: { global: { branches: 80, lines: 85 } }

💡 Set thresholds per-project, not per-PR — ratcheting up gradually beats a sudden strict gate.

uncovered code

Code never exercised by any test — a candidate for either a missing test or safe deletion (dead code).

# coverage report highlights lines 40-45 in red — never executed

💡 Coverage tools flag this automatically; it's the fastest way to find untested edge cases.

Tooling & CI

test runner

The tool that discovers, executes, and reports the results of your tests — Jest, pytest, Mocha, JUnit, Vitest.

npx jest --watch
pytest -v tests/

💡 Often bundled with an assertion library and a mocking API, but they're conceptually separate pieces.

assertion library

A library providing the matcher/expect syntax, sometimes shipped separately from the test runner itself.

import { expect } from 'chai';
expect(sum(1, 2)).to.equal(3);

💡 Chai + Mocha is the classic "runner + assertions are separate packages" combo, unlike Jest which bundles both.

watch mode

Runs tests automatically whenever a file changes — the fast feedback loop for local development.

jest --watch
vitest --watch

💡 Most watch modes are smart enough to re-run only the tests affected by the changed file.

parallel test execution

Running independent test files or suites concurrently across multiple workers to cut total CI time.

jest --maxWorkers=4
pytest -n auto  # pytest-xdist

💡 Requires test isolation — parallel tests that share state (a DB, a port) will flake.

test isolation

Ensuring one test's state or side effects can never leak into another test, regardless of execution order.

beforeEach(() => db.beginTransaction());
afterEach(() => db.rollback()); // every test starts from a clean slate

💡 The #1 root cause of "tests pass alone but fail in the full suite."

headless browser testing

Running browser-based e2e tests without an actual visible UI window — the standard way to run them in CI.

npx playwright test
npx cypress run --headless

💡 "Headless" describes the browser, not the test — the same test can run headed locally, headless in CI.

CI test gate

A required, passing check that blocks a pull request from merging until the pipeline is green.

# GitHub branch protection: "Require status checks to pass"
# → the "test" job must succeed before Merge is enabled

💡 Distinct from a coverage threshold — a gate can require tests to pass at 0% new coverage or vice versa.

English phrases engineers use

  • "That's not a real bug — the test is just flaky, re-run it."
  • "We stubbed out the payment gateway so the test doesn't hit the real API."
  • "Coverage is at 60%, but we should focus on branch coverage, not just lines."
  • "Can you add a regression test so this bug doesn't come back?"
  • "This PR is blocked — the CI test gate is red."
  • "Don't mock the database in an integration test — that defeats the point."
  • "Let's write a quick smoke test before we dig into the full suite."