Chapter 27: Contract Testing

---

Learning Objectives

By the end of this chapter, you will be able to:

• Define contract testing and explain how it validates API behavior against a formal specification
• Distinguish consumer-driven contracts from provider-driven contracts
• Apply OpenAPI contract testing: validate responses against schema, detect breaking changes, generate mock servers
• Use tools such as Pact, Specmatic, Prism, and Dredd for contract testing
• Apply contract testing in microservices to ensure service compatibility
• Set up contract testing for a running project through a hands-on tutorial
• Integrate contract testing into CI/CD pipelines
• Understand contract evolution: versioning and backwards compatibility
• Execute the resolution workflow when contracts break

---

What Is Contract Testing?

Contract testing validates that an API's actual behavior matches a formal specification—the contract. The contract defines request and response formats, status codes, headers, and schemas. Contract tests answer: "Does this API conform to what it promises?"

Unlike integration tests (which verify business logic and end-to-end flows), contract tests focus on structure and compatibility. They ensure that:

1. Responses match the schema — The JSON (or other format) returned has the expected shape, types, and required fields.
2. Breaking changes are detected — If a provider removes a field or changes a type, contract tests fail immediately.
3. Consumers and providers stay compatible — In microservices, each consumer's expectations are validated against the provider's implementation.

Contract testing is especially valuable when:

• Multiple services or clients depend on an API
• API evolution must not break consumers
• You want to mock APIs from a specification (contract as mock source)
• You need to validate API responses without running full integration tests

---

Consumer-Driven vs. Provider-Driven Contracts

Provider-Driven Contracts

The provider (the API implementation) publishes the contract. Consumers use it to build their integrations.

Flow:

1. Provider defines OpenAPI/AsyncAPI spec
2. Provider implements API
3. Contract tests validate: "Does our implementation match our spec?"
4. Consumers read the spec and integrate

Pros: Single source of truth; provider controls evolution. Cons: Provider may not know all consumer needs; consumers can be surprised by changes.

Consumer-Driven Contracts

Consumers define their expectations. The provider must satisfy all consumer contracts.

Flow:

1. Each consumer defines: "I need GET /users to return { id, email }"
2. Consumer contract tests run against provider (or mock)
3. Provider must satisfy all consumer contracts to deploy
4. Pact and similar tools support this model

Pros: Consumers explicitly state needs; provider knows impact of changes. Cons: Multiple contracts to satisfy; coordination overhead.

When to Use Which

Scenario	Approach
Single provider, few consumers	Provider-driven (OpenAPI)
Many consumers, provider serves all	Consumer-driven (Pact)
Public API	Provider-driven; versioning critical
Internal microservices	Either; consumer-driven helps with compatibility

In SDD, the specification often is the contract. Your OpenAPI spec from the plan phase becomes the contract. Contract tests validate implementation against that spec.

---

OpenAPI Contract Testing

OpenAPI (Swagger) is the dominant format for REST API contracts. Contract testing with OpenAPI typically involves:

1. Schema validation — Assert that response bodies conform to the schema
2. Status code validation — Assert that documented status codes are returned for documented scenarios
3. Header validation — Assert required headers are present
4. Breaking change detection — Compare new spec to old; flag incompatible changes

Validating Responses Against Schema

Given an OpenAPI spec:

openapi: 3.0.3
info:
  title: Bookmarks API
  version: 1.0.0

paths:
  /bookmarks:
    get:
      responses:
        '200':
          description: List of bookmarks
          content:
            application/json:
              schema:
                type: object
                required: [items, total]
                properties:
                  items:
                    type: array
                    items:
                      $ref: '#/components/schemas/Bookmark'
                  total:
                    type: integer
    post:
      requestBody:
        content:
          application/json:
            schema:
              type: object
              required: [url]
              properties:
                url: { type: string, format: uri }
                title: { type: string, maxLength: 200 }

components:
  schemas:
    Bookmark:
      type: object
      required: [id, url, createdAt]
      properties:
        id: { type: string, format: uuid }
        url: { type: string, format: uri }
        title: { type: string, nullable: true }
        createdAt: { type: string, format: date-time }

A contract test would:

1. Call GET /bookmarks
2. Parse the response
3. Validate the response body against the schema (items array of Bookmark, total integer)
4. Fail if any field is missing, wrong type, or violates constraints

Detecting Breaking Changes

Breaking changes include:

• Removing a field — Consumer expects email; provider removes it
• Changing type — count was integer; now string
• Adding required field — New required field; old clients don't send it
• Changing enum values — Status was "active"|"inactive"; now "ACTIVE"|"INACTIVE"
• Changing URL or method — Endpoint removed or renamed

Contract testing tools can compare spec versions and flag breaking changes before deployment.

Generating Mock Servers from Contracts

Tools like Prism can generate a mock server from an OpenAPI spec. The mock returns valid (but fake) data that conforms to the schema. This allows:

• Consumer development — Build and test consumers before the provider exists
• Parallel work — Frontend and backend teams work independently
• E2E testing — Use mocks when real services are unavailable

---

Tools: Pact, Specmatic, Prism, Dredd

Pact

• Model: Consumer-driven
• Flow: Consumer defines expectations; Pact generates a contract; provider verifies against it
• Languages: JS, Python, Java, .NET, Go, Ruby
• Use when: Multiple consumers; you want consumers to drive compatibility

Example (Pact JS, consumer side):

const { Pact } = require('@pact-foundation/pact');

const provider = new Pact({
  consumer: 'BookmarksFrontend',
  provider: 'BookmarksAPI',
});

describe('Bookmarks API contract', () => {
  beforeAll(() => provider.setup());
  afterEach(() => provider.verify());
  afterAll(() => provider.finalize());

  it('returns bookmarks list', async () => {
    await provider.addInteraction({
      state: 'user has bookmarks',
      uponReceiving: 'a request for bookmarks',
      withRequest: { method: 'GET', path: '/bookmarks' },
      willRespondWith: {
        status: 200,
        body: { items: [{ id: 'uuid', url: 'https://example.com', createdAt: '2026-01-01T00:00:00Z' }], total: 1 },
      },
    });

    const res = await fetch('http://localhost:8080/bookmarks');
    const data = await res.json();
    expect(data.items).toBeDefined();
    expect(data.total).toBe(1);
  });
});

Specmatic

• Model: Contract-first; spec is source of truth
• Features: Contract as stub, contract testing, schema validation
• Use when: You want a single spec to drive stubs and tests

Prism

• Model: Mock server from OpenAPI
• Features: Generate mock server, validate requests against spec
• Use when: You need a mock API for development or testing

Example:

npx prism mock openapi.yaml
# Server runs at http://localhost:4010
# Returns example responses that match the schema

Dredd

• Model: Provider-driven; validates HTTP transactions against OpenAPI
• Features: Run requests, validate responses against schema
• Use when: You have OpenAPI and want to validate a live or mock API

Example:

dredd openapi.yaml http://localhost:3000
# For each path in the spec, Dredd sends a request and validates the response

---

Contract Testing in Microservices

In a microservices architecture, services communicate via APIs. Contract testing ensures that when Service A calls Service B, the contract between them is satisfied.

The Problem

• Service B (provider) changes a response format
• Service A (consumer) expects the old format
• Integration tests might not catch this if they use mocks or if the change is subtle
• Production: Service A breaks when calling Service B

The Solution

1. Define the contract — OpenAPI or Pact contract
2. Consumer tests — Consumer runs contract tests against provider (or provider's stub)
3. Provider tests — Provider runs contract tests to verify it satisfies the contract
4. CI/CD — Both consumer and provider pipelines run contract tests; breaking changes fail the build

Contract Testing Workflow

Consumer defines expectations (Pact) or Provider publishes spec (OpenAPI)
         ↓
Contract stored (e.g., Pact Broker, Git)
         ↓
Provider verification: Does our implementation satisfy the contract?
         ↓
Consumer verification: Does the provider (or stub) satisfy our expectations?
         ↓
Both pass → Safe to deploy
Either fails → Fix before deploy

---

Tutorial: Set Up Contract Testing for the Running Project

This tutorial walks you through setting up contract testing for the Bookmarks API. You will write an OpenAPI specification, generate contract tests, run against the live API, detect a breaking change, and fix the violation.

Step 0: Prerequisites

• A running project with a Bookmarks API (or similar)
• Node.js and npm (or your stack's package manager)
• The API running locally (e.g., npm run dev)

Step 1: Write OpenAPI Specification

Create specs/005-bookmarks/contracts/openapi.yaml:

openapi: 3.0.3
info:
  title: Bookmarks API
  version: 1.0.0

servers:
  - url: http://localhost:3000/api

paths:
  /bookmarks:
    get:
      summary: List bookmarks
      parameters:
        - name: page
          in: query
          schema: { type: integer, default: 1 }
        - name: limit
          in: query
          schema: { type: integer, default: 20 }
      responses:
        '200':
          description: Success
          content:
            application/json:
              schema:
                type: object
                required: [items, total]
                properties:
                  items:
                    type: array
                    items:
                      $ref: '#/components/schemas/Bookmark'
                  total:
                    type: integer
    post:
      summary: Create bookmark
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required: [url]
              properties:
                url: { type: string, format: uri }
                title: { type: string, maxLength: 200 }
      responses:
        '201':
          description: Created
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Bookmark'
        '400':
          description: Validation error

  /bookmarks/{id}:
    delete:
      summary: Delete bookmark
      parameters:
        - name: id
          in: path
          required: true
          schema: { type: string, format: uuid }
      responses:
        '204':
          description: Deleted
        '404':
          description: Not found

components:
  schemas:
    Bookmark:
      type: object
      required: [id, url, createdAt]
      properties:
        id: { type: string, format: uuid }
        url: { type: string, format: uri }
        title: { type: string, nullable: true }
        createdAt: { type: string, format: date-time }

Step 2: Install Contract Testing Tools

Using Dredd (OpenAPI validation):

npm install -D dredd

Or using a custom Vitest + OpenAPI validation approach:

npm install -D ajv ajv-formats

Step 3: Create Contract Test Suite

Create tests/contract/bookmarks.contract.test.ts:

import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import Ajv from 'ajv';
import addFormats from 'ajv-formats';
import openapi from '../../specs/005-bookmarks/contracts/openapi.yaml';

// Load schemas from OpenAPI
const bookmarkSchema = openapi.components.schemas.Bookmark;
const listSchema = {
  type: 'object',
  required: ['items', 'total'],
  properties: {
    items: { type: 'array', items: bookmarkSchema },
    total: { type: 'integer' },
  },
};

const ajv = new Ajv();
addFormats(ajv);
const validateBookmark = ajv.compile(bookmarkSchema);
const validateList = ajv.compile(listSchema);

const API_BASE = 'http://localhost:3000/api';

describe('Bookmarks API Contract', () => {
  let authHeader: string;

  beforeAll(async () => {
    // Get auth token (adjust for your auth flow)
    const loginRes = await fetch(`${API_BASE}/auth/login`, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ email: 'contract-test@example.com', password: 'test123' }),
    });
    const { token } = await loginRes.json();
    authHeader = `Bearer ${token}`;
  });

  it('GET /bookmarks returns valid schema', async () => {
    const res = await fetch(`${API_BASE}/bookmarks`, {
      headers: { Authorization: authHeader },
    });
    expect(res.status).toBe(200);
    const data = await res.json();
    const valid = validateList(data);
    expect(valid).toBe(true);
    if (!valid) console.error(validateList.errors);
  });

  it('POST /bookmarks returns valid Bookmark schema', async () => {
    const res = await fetch(`${API_BASE}/bookmarks`, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json', Authorization: authHeader },
      body: JSON.stringify({ url: 'https://contract-test.example.com' }),
    });
    expect(res.status).toBe(201);
    const data = await res.json();
    const valid = validateBookmark(data);
    expect(valid).toBe(true);
    if (!valid) console.error(validateBookmark.errors);
  });

  it('POST /bookmarks with invalid URL returns 400', async () => {
    const res = await fetch(`${API_BASE}/bookmarks`, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json', Authorization: authHeader },
      body: JSON.stringify({ url: 'not-a-valid-url' }),
    });
    expect(res.status).toBe(400);
  });
});

Step 4: Run Contract Tests

Ensure the API is running, then:

npm run test:contract

If the API conforms, all tests pass. If not, you get schema validation errors.

Step 5: Simulate a Breaking Change

Introduce a breaking change in the API: change the createdAt field to created_at (snake_case) in the response. Run the contract tests again. They fail:

Expected: schema requires 'createdAt'
Actual: response has 'created_at'

This is contract testing in action: the implementation drifted from the spec, and the contract test caught it.

Step 6: Fix the Contract Violation

Two options:

Option A: Fix the implementation to match the spec (revert to createdAt).

Option B: Update the spec if the change was intentional (e.g., team standard is snake_case). Then update all consumers. Contract tests ensure you don't forget.

In SDD, the spec is the source of truth. If the spec says createdAt, the implementation must use createdAt. Fix the implementation.

Step 7: Add to CI/CD

Add a contract test step to your pipeline:

# GitHub Actions example
- name: Contract tests
  run: npm run test:contract
  env:
    API_URL: http://localhost:3000

Ensure the API is started before the step (e.g., via a service container or npm run dev in background).

---

Contract Testing in CI/CD Pipelines

Where Contract Tests Run

Stage	What Runs
Provider CI	Provider runs contract tests against its own implementation ("Do we satisfy the contract?")
Consumer CI	Consumer runs contract tests against provider stub or real provider ("Does provider satisfy us?")
Pre-merge	Both; PR fails if contract tests fail
Pre-deploy	Contract tests as gate; no deploy if broken

Pipeline Example

Provider PR:
  - Unit tests
  - Contract tests (provider verification)
  - Integration tests
  → Merge if all pass

Consumer PR:
  - Unit tests
  - Contract tests (consumer verification against Pact stub or provider)
  → Merge if all pass

Deploy:
  - Run contract tests against staging
  - Deploy if pass

---

Contract Evolution: Versioning and Backwards Compatibility

Contracts evolve. New fields, new endpoints, deprecated fields. The key is backwards compatibility.

Backwards-Compatible Changes

• Adding optional fields — New field; old clients ignore it
• Adding new endpoints — No impact on existing consumers
• Adding new enum values — Old clients don't use them
• Deprecating (without removing) — Mark deprecated; remove in next major version

Breaking Changes

• Removing a field — Consumers may depend on it
• Changing type — count: number → count: string
• Adding required field — Old clients don't send it
• Removing endpoint — Consumers break

Versioning Strategy

URL versioning: /v1/bookmarks, /v2/bookmarks

• Pros: Clear, easy to route
• Cons: Multiple versions to maintain

Header versioning: Accept: application/vnd.api+json;version=1

• Pros: Same URL
• Cons: Less visible, tooling support varies

Schema versioning: Contract includes version; consumers check compatibility

• Pros: Flexible
• Cons: More complex

Recommendation: Use URL versioning for major breaks (/v1/, /v2/). Use backwards-compatible changes within a version. When you must break, release a new version.

Resolution Workflow When Contracts Break

1. Detect — Contract tests fail in CI
2. Identify — Which consumer(s) or provider(s) are affected?
3. Communicate — Notify dependent teams
4. Decide:
   • Fix provider — Revert breaking change or add compatibility layer
   • Fix consumer — Update consumer to new contract
   • Coordinate — Deploy provider and consumer together (big bang; avoid if possible)
5. Prevent — Add contract tests to catch this class of break in the future
6. Document — ADR or changelog: what broke, why, how it was resolved

---

Common Pitfalls in Contract Testing

Pitfall 1: Testing Only Happy Paths

Wrong: Contract tests only validate 200/201 responses.

Right: Include 400, 404, 401, 500 in your contract. Document error response schemas and validate them.

Pitfall 2: Over-Specifying in Consumer Contracts

Wrong: Consumer expects exact field values (e.g., id: "123").

Right: Consumer expects structure and types. Use matchers (e.g., "any string") for values that vary.

Pitfall 3: Stale Contracts

Wrong: Implementation changes, contract tests are updated to match, but the spec is not.

Right: The spec is the source of truth. Update the spec first, then implementation, then run contract tests. Never "fix" contract tests by relaxing assertions without updating the spec.

Pitfall 4: Ignoring Authentication in Contract Tests

Wrong: Contract tests hit endpoints without auth; production requires auth.

Right: Contract tests must use the same auth mechanism (tokens, API keys) as production. Otherwise you may miss auth-related contract violations.

Pitfall 5: Running Contract Tests Against Wrong Environment

Wrong: Contract tests run against production.

Right: Run against staging or local. Production is for validation, not for failing builds.

---

Try With AI

Prompt 1: OpenAPI from Spec

"I have a feature specification at specs/005-bookmarks/spec.md with API requirements. Generate an OpenAPI 3.0 specification that captures the endpoints, request/response schemas, and error responses. Save to specs/005-bookmarks/contracts/openapi.yaml."

Prompt 2: Contract Test Generation

"Given the OpenAPI spec at specs/005-bookmarks/contracts/openapi.yaml, generate contract tests using Vitest and Ajv. The tests should validate GET /bookmarks and POST /bookmarks responses against the schema. Include setup for authentication. Use a configurable API_BASE URL."

Prompt 3: Breaking Change Analysis

"I'm considering changing the Bookmarks API: (1) Rename 'createdAt' to 'created_at', (2) Add a required 'userId' field to the POST request. For each change, explain whether it's breaking or backwards-compatible. If breaking, suggest a migration path (e.g., versioning, deprecation period)."

Prompt 4: Pact Consumer Contract

"Convert our OpenAPI-based contract tests to a Pact consumer contract. The consumer is 'BookmarksFrontend', the provider is 'BookmarksAPI'. Define an interaction for GET /bookmarks that expects 200 with items and total. Show the Pact setup and verification code."

---

Practice Exercises

Exercise 1: Create OpenAPI and Contract Tests

Take an API you have (or the Bookmarks API from the tutorial). Write a minimal OpenAPI spec for one endpoint. Implement contract tests that validate the response against the schema. Run them against the live API. Intentionally break the API (e.g., change a field name) and confirm the contract tests fail.

Expected outcome: Passing contract tests, then failing tests after the intentional break, with a clear error message.

Exercise 2: Breaking Change Detection

Create two versions of an OpenAPI spec: v1 and v2. In v2, make a breaking change (e.g., remove a field, change a type). Write a script or use a tool to compare the specs and list the breaking changes. Document the migration path for consumers.

Expected outcome: A report listing breaking changes and a migration guide.

Exercise 3: CI Integration

Add contract tests to your project's CI pipeline (GitHub Actions, GitLab CI, or similar). Ensure the API is started before contract tests run. Verify that a failing contract test fails the pipeline. Document the setup in a README or ADR.

Expected outcome: CI runs contract tests; pipeline fails when contract tests fail.

---

Key Takeaways

1. Contract testing validates API behavior against a formal specification. It ensures responses match schemas and detects breaking changes before they reach production.

2. Consumer-driven (Pact) vs. provider-driven (OpenAPI): Consumers define expectations vs. provider publishes spec. Choose based on your architecture and team structure.

3. OpenAPI contract testing involves schema validation, status code checks, and breaking change detection. Tools: Dredd, Prism, Ajv, Specmatic.

4. In microservices, contract testing ensures service compatibility. Both consumer and provider run contract tests; breaking changes fail the build.

5. Contract evolution requires backwards compatibility. Add optional fields, new endpoints; avoid removing fields or changing types without versioning. When contracts break, follow the resolution workflow: detect, identify, decide, fix, prevent, document.

6. CI/CD integration runs contract tests as a gate. No deploy with broken contracts.

---

Chapter Quiz

1. What is contract testing, and how does it differ from integration testing?

2. Explain the difference between consumer-driven and provider-driven contracts. When would you use each?

3. What are three types of breaking changes in an API contract? Give an example of each.

4. How can you generate a mock server from an OpenAPI spec? Name a tool and describe one use case.

5. In a microservices setup, where do contract tests run (consumer CI, provider CI, or both)? What does each verify?

6. What is backwards compatibility? Give two examples of backwards-compatible changes and two of breaking changes.

7. When a contract test fails in CI, what steps should you follow to resolve it? (Outline the resolution workflow.)

8. How does contract testing fit into Spec-Driven Development? Where does the contract come from in the SDD pipeline?