Esc
↑↓ navigate   Enter open   Esc close

Software Testing Vocabulary

42 testing terms in plain English — what each one means, a code example, and the gotcha worth knowing. From "what's the difference between a mock and a stub" to reading a coverage report.

Last reviewed:

Sections

Core concepts

unit test

Tests a single function or method in isolation from the rest of the system — every dependency is faked or removed.

test('add sums two numbers', () => {
  expect(add(2, 3)).toBe(5);
});

💡 Fast and cheap — should make up the bulk of your test suite (see "test pyramid").

integration test

Verifies that two or more real units — a service and a real database, two microservices — work together correctly.

test('creating a user persists to the real DB', async () => {
  await createUser({ email: 'a@b.com' });
  expect(await db.users.findOne({ email: 'a@b.com' })).toBeTruthy();
});

💡 Slower and more brittle than unit tests, but catches bugs unit tests structurally cannot.

end-to-end (e2e) test

Exercises the whole system the way a real user would — clicking through the UI, hitting the real backend, the real database.

await page.goto('/login');
await page.fill('#email', 'a@b.com');
await page.click('button[type=submit]');
await expect(page).toHaveURL('/dashboard');

💡 Slowest and most valuable-per-test, but also the most expensive to write and maintain.

test suite

A collection of related test cases — usually every test in one file, or every test for one feature.

describe('UserService', () => {
  test('creates a user', () => { /* ... */ });
  test('rejects a duplicate email', () => { /* ... */ });
});

💡 "The suite is red" means at least one test in it is failing.

test case

A single scenario with a specific input and an expected outcome — the smallest unit a test runner reports on.

test('returns 0 for an empty array', () => {
  expect(sum([])).toBe(0);
});

💡 A good test case name describes the behaviour, not the implementation.

assertion

A statement that checks an actual value against an expected one and fails the test if they don't match.

expect(response.status).toBe(200);
assert.strictEqual(user.name, 'Alice');

💡 One logical assertion per test keeps failures easy to diagnose.

fixture

A fixed, known piece of test data or environment state that is set up before a test runs.

beforeEach(() => {
  testUser = { id: 1, name: 'Alice', role: 'admin' };
});

💡 Shared fixtures save typing but can hide what a test actually depends on — keep them small.

setup / teardown

Code that runs before (setup) and after (teardown) each test to prepare and then clean up state.

beforeEach(() => db.connect());
afterEach(() => db.clear());

💡 Teardown that never runs (a test that crashes before it) is a classic cause of flaky tests.

Test doubles

test double

The umbrella term for any object that stands in for a real dependency during a test — mocks, stubs, spies, fakes, and dummies are all test doubles.

// "test double" is the category; the specific kind matters for what you can assert

💡 Borrowed from film — a stunt double stands in for the real actor.

mock

A test double that records how it was called, so you can assert the interaction itself, not just the return value.

const sendEmail = jest.fn();
await notifyUser(sendEmail);
expect(sendEmail).toHaveBeenCalledWith('a@b.com', 'Welcome!');

💡 Overusing mocks couples your tests to implementation details — assert behaviour, not every call.

stub

A test double that returns hardcoded, canned responses without any real logic behind them.

const getUser = () => ({ id: 1, name: 'Alice' }); // stub — no real DB call

💡 A stub answers questions; a mock also remembers what you asked it.

spy

Wraps a real function or object and records calls to it while (usually) still calling through to the real implementation.

const spy = jest.spyOn(analytics, 'track');
checkout();
expect(spy).toHaveBeenCalledWith('purchase_completed');

💡 Use a spy when you want the real side effect to happen but also need to verify it happened.

fake

A working but simplified implementation of a dependency — e.g. an in-memory database instead of a real Postgres instance.

class InMemoryUserRepo {
  #users = new Map();
  save(u) { this.#users.set(u.id, u); }
  find(id) { return this.#users.get(id); }
}

💡 Fakes are more work to build than mocks/stubs but give you far more realistic tests.

dummy

A placeholder object passed only to satisfy a function's signature — the test never actually uses it.

createInvoice(order, /* logger */ null); // dummy — this test doesn't check logging

💡 The simplest test double: it exists to fill a parameter slot, nothing more.

Test types & strategy

smoke test

A quick, shallow check that the most critical paths work at all — run before investing time in the full suite.

test('app boots and the homepage returns 200', async () => {
  const res = await fetch('http://localhost:3000/');
  expect(res.status).toBe(200);
});

💡 Named after hardware testing: power it on and check for smoke before deeper diagnostics.

regression test

A test written specifically to make sure a previously fixed bug never comes back.

// Bug #482: negative quantities crashed checkout
test('rejects a negative quantity', () => {
  expect(() => addToCart({ qty: -1 })).toThrow();
});

💡 Best practice: write the failing regression test first, confirm it fails, then fix the bug.

snapshot test

Captures the current output — often a rendered component — and fails on any future run that produces a different output.

expect(render(<Button label="Save" />)).toMatchSnapshot();

💡 Cheap to write but easy to rubber-stamp ("update snapshot") without reading the diff — use sparingly.

mutation testing

Deliberately introduces small bugs ("mutants") into the code — flips a `<` to `<=`, removes a line — to check whether the test suite actually catches them.

# Stryker / mutmut reports a "mutation score":
# 87% of mutants were killed by the test suite

💡 Answers "is my coverage number actually meaningful?" — high line coverage can still miss real bugs.

flaky test

A test that passes and fails intermittently with no code changes — usually caused by timing, execution order, or shared state.

// Flaky: assumes the animation finished in exactly 300ms
await sleep(300);
expect(el).toBeVisible();
// Fix: wait for the actual condition, not a fixed delay
await waitFor(() => expect(el).toBeVisible());

💡 A quarantined/retried flaky test is technical debt — it hides real failures behind "just re-run it".

test pyramid

The guideline that a healthy suite has many fast unit tests, fewer integration tests, and very few slow e2e tests.

      /\   e2e (few)
     /--\  integration (some)
    /----\ unit (many)

💡 An "ice cream cone" anti-pattern — mostly e2e, almost no unit tests — is slow and flaky by nature.

black-box testing

Testing based only on inputs and outputs, with no knowledge of (or dependency on) the internal implementation.

test('POST /users returns 201 with a Location header', async () => {
  // no knowledge of the handler's internals required
});

💡 Survives refactors that don't change behaviour — the classic argument for testing "what", not "how".

white-box testing

Testing that uses knowledge of the internal code structure — branches, loops, edge cases — to design test cases.

// Knows the function has an early-return for a null input
test('returns null immediately without hitting the DB', () => { /* ... */ });

💡 Good for hitting hard-to-reach branches; brittle if it tests internals instead of behaviour.

acceptance test

Confirms a feature meets the actual business requirement, often written from the user's perspective.

Given a logged-in user with items in their cart
When they complete checkout
Then they should see an order confirmation

💡 The Given/When/Then format (Gherkin) is common in BDD tools like Cucumber.

contract test

Verifies that a service's API matches what its consumers actually expect, without spinning up the whole downstream system.

// Pact: consumer defines the expected request/response,
// provider verifies it can satisfy that contract in CI

💡 Catches breaking API changes early in microservice architectures, cheaper than full e2e.

Assertions & matchers

matcher

A method that expresses an expectation about a value — toBe, toEqual, toContain, and dozens more.

expect(items).toContain('apple');
expect(response.body).toMatchObject({ ok: true });

💡 A good matcher choice produces a readable failure message — prefer toEqual over manual comparisons.

toBe / toEqual

toBe checks reference/primitive equality (Object.is); toEqual checks deep structural equality, recursing into objects and arrays.

expect({ a: 1 }).not.toBe({ a: 1 });   // different references
expect({ a: 1 }).toEqual({ a: 1 });     // same shape — passes

💡 The #1 beginner bug in Jest tests: using toBe on an object and getting a confusing failure.

truthy / falsy

Matchers that check whether a value is truthy or falsy, rather than matching a specific value.

expect(user).toBeTruthy();
expect(errors.length).toBeFalsy();

💡 Looser than toBe(true) — 1, "x", and {} are all truthy; 0, "", null, undefined are falsy.

toThrow

Asserts that a function throws an error when called — the function itself, not its result, must be passed in.

expect(() => parseConfig('{bad json')).toThrow(SyntaxError);

💡 Common mistake: expect(parseConfig(x)).toThrow() — this calls the function immediately, before the assertion runs.

deep equality

Comparing two objects or arrays by their contents rather than by memory reference.

deepEqual({ a: [1, 2] }, { a: [1, 2] }); // true — same content, different objects

💡 Every mainstream assertion library has a deep-equal matcher; reaching for JSON.stringify() comparisons is a common workaround anti-pattern.

golden file

A saved reference output file that test output is diffed against — similar to a snapshot, but stored as a real file on disk.

const actual = renderReport(data);
const expected = fs.readFileSync('testdata/report.golden.txt', 'utf8');
expect(actual).toBe(expected);

💡 Common in Go and CLI-tool testing; update goldens deliberately, not on every red run.

Coverage & metrics

code coverage

The percentage of code actually executed by the test suite — a proxy for how much of the codebase is tested at all.

# jest --coverage
Statements   : 87.3% ( 1042/1194 )

💡 100% coverage does not mean bug-free — it only means every line ran, not that every case was checked.

line coverage

The percentage of source lines executed at least once by the test suite.

Lines        : 91.2% ( 512/561 )

💡 The most common (and most misleading) coverage number — easy to game with shallow tests.

branch coverage

The percentage of possible branches — every if/else path, every ternary — actually exercised by the tests.

function discount(user) {
  if (user.isPremium) return 0.2;  // branch A
  return 0;                          // branch B — is this ever tested?
}

💡 A much stricter, more useful signal than line coverage alone.

coverage threshold

A minimum coverage percentage that a CI pipeline requires before it allows a merge.

// jest.config.js
coverageThreshold: { global: { branches: 80, lines: 85 } }

💡 Set thresholds per-project, not per-PR — ratcheting up gradually beats a sudden strict gate.

uncovered code

Code never exercised by any test — a candidate for either a missing test or safe deletion (dead code).

# coverage report highlights lines 40-45 in red — never executed

💡 Coverage tools flag this automatically; it's the fastest way to find untested edge cases.

Tooling & CI

test runner

The tool that discovers, executes, and reports the results of your tests — Jest, pytest, Mocha, JUnit, Vitest.

npx jest --watch
pytest -v tests/

💡 Often bundled with an assertion library and a mocking API, but they're conceptually separate pieces.

assertion library

A library providing the matcher/expect syntax, sometimes shipped separately from the test runner itself.

import { expect } from 'chai';
expect(sum(1, 2)).to.equal(3);

💡 Chai + Mocha is the classic "runner + assertions are separate packages" combo, unlike Jest which bundles both.

watch mode

Runs tests automatically whenever a file changes — the fast feedback loop for local development.

jest --watch
vitest --watch

💡 Most watch modes are smart enough to re-run only the tests affected by the changed file.

parallel test execution

Running independent test files or suites concurrently across multiple workers to cut total CI time.

jest --maxWorkers=4
pytest -n auto  # pytest-xdist

💡 Requires test isolation — parallel tests that share state (a DB, a port) will flake.

test isolation

Ensuring one test's state or side effects can never leak into another test, regardless of execution order.

beforeEach(() => db.beginTransaction());
afterEach(() => db.rollback()); // every test starts from a clean slate

💡 The #1 root cause of "tests pass alone but fail in the full suite."

headless browser testing

Running browser-based e2e tests without an actual visible UI window — the standard way to run them in CI.

npx playwright test
npx cypress run --headless

💡 "Headless" describes the browser, not the test — the same test can run headed locally, headless in CI.

CI test gate

A required, passing check that blocks a pull request from merging until the pipeline is green.

# GitHub branch protection: "Require status checks to pass"
# → the "test" job must succeed before Merge is enabled

💡 Distinct from a coverage threshold — a gate can require tests to pass at 0% new coverage or vice versa.

English phrases engineers use

  • "That's not a real bug — the test is just flaky, re-run it."
  • "We stubbed out the payment gateway so the test doesn't hit the real API."
  • "Coverage is at 60%, but we should focus on branch coverage, not just lines."
  • "Can you add a regression test so this bug doesn't come back?"
  • "This PR is blocked — the CI test gate is red."
  • "Don't mock the database in an integration test — that defeats the point."
  • "Let's write a quick smoke test before we dig into the full suite."

Frequently Asked Questions

What's the difference between a mock and a stub?

A stub is a test double that returns hardcoded, canned responses with no real logic behind them — it just answers questions. A mock does the same but additionally records how it was called, so the test can assert on the interaction itself (e.g. "was sendEmail called with this address?"), not just on a return value. Overusing mocks tends to couple tests to implementation details, so it's worth reaching for a stub when you only need a canned answer.

What's the difference between toBe and toEqual in Jest?

toBe checks reference/primitive equality using Object.is, so two objects with identical contents but different references will fail a toBe comparison. toEqual checks deep structural equality, recursing into objects and arrays to compare their contents rather than their identity — using toBe on an object when you meant toEqual is one of the most common beginner mistakes in Jest tests.

Why does the test pyramid recommend many unit tests and few end-to-end tests?

Unit tests are fast and cheap because they isolate a single function from the rest of the system, so a healthy suite can run thousands of them in seconds. End-to-end tests exercise the whole system through the real UI and backend, making them slow, expensive to maintain, and more prone to flakiness — the pyramid shape (many unit, some integration, few e2e) balances speed and confidence, while the inverted "ice cream cone" anti-pattern is both slow and flaky.

Related references