Testing Patterns & TDD Vocabulary

Test doubles vocabulary (Gerard Meszaros's taxonomy):

Dummy
Passed as an argument but never actually used in the test. Just fills a required parameter.
Example: new UserService(logger = NullLogger()) — the test doesn't care about logging.

Stub
Returns hard-coded or pre-configured answers. Does NOT verify how it was called.
Example: userRepo.findById.returns(testUser) — control input to the unit under test.
Use when: you want to control indirect inputs (return values, exceptions thrown).

Mock
Pre-programmed with expectations. Verifies that the expected calls were made with the expected arguments.
Example: emailService.expects("sendWelcomeEmail").with(user).once()
Use when: the test is verifying that the system under test called a collaborator correctly.
Failure mode: test fails at mock.verify() if expectations were not met.

Spy
Like a mock but set up after the call. Records what happened without pre-setting expectations.
Example: const spy = jest.spyOn(emailService, "send"); // assert after
Use when: you want to observe calls to a real object without completely replacing it.

Fake
A working, simplified implementation of a real dependency.
Examples: in-memory SQLite database (instead of PostgreSQL), in-memory message broker, test SMTP server.
Use when: integration-level tests need a fast real-like implementation.

Rule of thumb:
• "Don't mock what you don't own" — only mock collaborators you control
• Over-mocking leads to tests that pass but don't test real behavior
• Prefer stubs/fakes for infrastructure; prefer mocks for verifying key interactions

Vocabulary:
• test double — umbrella term for any stand-in for a production object
• system under test (SUT) — the class or function being tested
• collaborator — objects or services the SUT interacts with
• indirect input — data supplied to the SUT through a collaborator
• indirect output — side effects or calls the SUT makes to a collaborator

TDD (Test-Driven Development) cycle vocabulary:

Red phase — Write a failing test
Write a test that describes the desired behavior before writing production code.
• The test fails because the code doesn't exist yet (or the feature is not implemented)
• This confirms the test is testing something real and can detect failure
• "If your test doesn't fail, it's not testing anything"
• Keep the test small — one behavior at a time

Green phase — Make it pass (minimally)
Write the simplest possible code that makes the failing test pass.
• Resist adding extra logic — only what the test requires
• Deliberately writing minimum code prevents over-engineering
• It's acceptable to hardcode values temporarily — refactor fixes this

Refactor phase — Improve without breaking
Clean up both production code and test code while keeping all tests green.
• Remove duplication, rename variables, extract methods
• Apply design patterns where they reduce complexity
• Never change behavior — tests stay green throughout
• If a test breaks during refactoring, undo and try differently

Why the order matters:
1. Writing the test first forces you to define the expected behavior before implementation — this is the design discipline
2. Minimum green code prevents feature creep
3. Refactoring with a safety net (existing tests) makes structural improvements safe

Vocabulary:
• test-first — writing tests before production code (broader term; TDD is a specific test-first practice)
• BDD (Behaviour-Driven Development) — extends TDD with natural language test specifications (Given/When/Then)
• failing test — a test that does not pass; in TDD this is intentional at the Red phase
• regression — a previously passing test that now fails after a code change
• coverage — percentage of production code exercised by tests; TDD naturally produces high coverage
• baby steps — very small TDD iterations; advocates writing one assertion at a time

Property-based testing vocabulary:

Example-based testing (the default)
You write specific input-output pairs:

expect(add(2, 3)).toBe(5)
expect(add(0, 0)).toBe(0)
expect(add(-1, 1)).toBe(0)

You choose the examples. Good for known edge cases, but limited by the tester's imagination.

Property-based testing
You describe invariants (properties) that must hold for all valid inputs, and the framework generates many random inputs to try to falsify them.

Example property for a JSON serialization function:
• Property: "For any valid object, JSON.parse(JSON.stringify(obj)) deeply equals obj"
• The framework generates hundreds of objects automatically
• If any input violates the property, it reports the minimal failing example (shrinking)

Frameworks:
• Hypothesis — Python; most mature property-based testing library
• fast-check — TypeScript/JavaScript
• jqwik — Java
• QuickCheck — Haskell (the original); ported to many languages
• FsCheck — F# / .NET

Strengths:
• Finds edge cases that humans miss: empty collections, negative numbers, unicode, max int overflow
• Especially powerful for: parsers, serialization, sorting algorithms, math functions, invertible operations
• Shrinking: when a failure is found, the framework minimizes the failing input to the simplest case

Weaknesses:
• Harder to define good properties than to write examples
• Not suitable for: side-effectful systems, user interfaces, complex workflows
• Can be slow for large input spaces

Vocabulary:
• property — an invariant that must hold for all valid inputs (e.g., symmetry, idempotency)
• generator — a framework component that produces random test data of a given type
• shrinking — automatically reducing a failing input to its simplest form
• fuzzing — related technique; generates random byte/string inputs to find crashes in parsers or security vulnerabilities
• oracle problem — it can be hard to know the correct output for arbitrary inputs (property-based testing sidesteps this with invariants rather than exact outputs)

Flaky test vocabulary and causes:

Definition
A flaky test is one that produces different results (pass or fail) when run on the same code without any changes. It is non-deterministic.

Why flaky tests are dangerous:
• Developers lose trust in the CI/CD signal — they assume failures are noise and start merging broken code
• Masking real failures: a real regression may look like "just another flaky test"
• CI costs: retrying flaky tests wastes build minutes
• Debugging: intermittent failures are extremely hard to reproduce and fix

Common causes and fixes:

1. Time-dependent code
Cause: new Date(), Date.now(), sleep/delay without mocking
Fix: inject a clock abstraction; mock time with jest.useFakeTimers()

2. Shared or leaked state
Cause: a previous test modifies a global variable, singleton, or database record that the next test depends on
Fix: reset state in beforeEach/afterEach; use transactions that roll back; isolate each test

3. Real network calls
Cause: test calls a real API endpoint that is slow or unavailable
Fix: mock HTTP calls (nock, MSW, WireMock); use hermetic test environment

4. Test ordering dependency
Cause: test B relies on side effects left by test A; fails when run in isolation or different order
Fix: every test must be independent; run with random ordering to detect these

5. Race conditions and async issues
Cause: improper async/await usage, event-driven code not properly awaited
Fix: use proper async utilities; avoid arbitrary sleep/setTimeout; use polling utilities (waitFor in Testing Library)

Vocabulary:
• hermetic test — a test completely isolated from external systems; always produces the same result
• test isolation — each test has no shared state with other tests
• test quarantine — temporarily disabling known flaky tests to prevent CI noise while they are being fixed
• retry logic — auto-retrying failed tests in CI (hides flakiness; never a substitute for fixing)
• determinism — a test is deterministic if the same code always produces the same pass/fail result

Contract testing and Pact vocabulary:

The problem with integration tests across microservices:
• End-to-end (E2E) integration tests require both services to be deployed and running simultaneously
• Slow: environment setup, network calls, database state
• Fragile: flaky because of real infrastructure
• Scale poorly: 10 services = potentially 10! combinations of integration tests

Consumer-driven contract testing solution:

Core idea:
The consumer (the service that calls the API) defines what it needs from the provider (the service that serves the API). The provider only needs to satisfy what consumers actually use — not the entire possible API surface.

Pact workflow:
1. Consumer writes a Pact contract — defines the interactions (requests + expected responses) it relies on
2. Pact generates a JSON contract file (the "pact")
3. Contract is published to a Pact Broker — a shared registry of all contracts
4. Provider verifies the contract — runs a Pact test suite against just the contract, without deploying the consumer
5. If all consumer interactions pass on the provider → "can I deploy?" check returns green

Benefits:
• Services can be tested in isolation — no joint environments needed
• Breaking API changes are detected instantly — the provider's CI fails before deployment
• "Consumer-driven" = providers only need to implement what consumers use

Vocabulary:
• consumer — the service that makes requests (calls the API)
• provider — the service that receives requests (serves the API)
• pact — the contract document: a list of request/response interactions the consumer expects
• Pact Broker — a server that stores and versions contracts and tracks can-i-deploy status
• can-i-deploy — CLI command that checks whether a version is safe to deploy based on verified contracts
• consumer-driven — the consumer writes the contract; the provider verifies it (vs. provider-driven)
• schema contract testing — a lighter approach using OpenAPI/JSON Schema to verify compatibility without the full Pact workflow

Alex is highlighting the need for robustness in testing. Negative tests – specifically targeting invalid inputs – are crucial for verifying how the system handles unexpected data. The other options represent overly narrow or ineffective test strategies that don't address the core issue of anticipating and handling unexpected scenarios. Focusing on generic assertions can be part of a good strategy, but doesn't replace targeted negative testing.

The Red-Green-Refactor cycle is central to TDD. The 'Refactor' phase isn't about fixing bugs; it's about improving the code itself. This involves restructuring, simplifying, or removing unnecessary complexity – all while maintaining the original behavior and ensuring that the tests continue to pass. Deploying changes or debugging are not part of this cycle.

A flaky test is one that produces inconsistent results – passing some runs and failing others without any obvious code changes. This can be caused by timing issues, external dependencies, or other unpredictable factors. The CI report highlights this instability as a major concern because it undermines confidence in the test suite's reliability. Fixing flaky tests is critical for maintaining a trustworthy automated testing environment.

A 502 Bad Gateway error indicates a problem with communication between two services. David needs to investigate the network connection – this could involve checking DNS resolution, firewall rules, or other infrastructure-level issues that might be preventing Service A from reaching Service B correctly. Examining the code of either service would only be a step after confirming basic connectivity is working.

Consumer-driven contract testing is a shift in focus. Instead of services defining their own contracts based on their internal logic, *consumers* (other services) define the expected behavior they require from a provider service. This ensures that the provider service meets the actual needs of its consumers and provides a clear definition of the API contract.