Skip to main content

📘 CASE STUDY — Part III: Designing a Rate Limiter (Edge + Service + DB)

SECTION 0 — SCENARIO

Your public API is being abused.

  • spikes cause DB saturation

  • attackers rotate IPs

  • legitimate users get degraded

You need a rate limiter that is:

  • fair

  • hard to bypass

  • observable

  • cheap to operate

SECTION 1 — CONSTRAINTS

  • multiple regions

  • at-least-once retries exist

  • traffic is bursty

SECTION 2 — DESIGN (MULTI-LAYER)

Use layers:

  1. Edge/CDN/WAF: coarse IP-based rules (cheap)

  2. API Gateway: per API key / per user

  3. Service-level: per tenant, per route, per expensive operation

Rule:

One limiter is never enough. You need the right limiter at the right boundary.

SECTION 3 — ALGORITHM CHOICE

  • token bucket for bursts

  • sliding window for fairness

Implementation detail:

  • centralized store (Redis) for counters/tokens

  • key schema includes: tenantId + route + time bucket

SECTION 4 — FAILURE MODES

  • Redis down → fail open or fail closed?

    • for auth endpoints: fail closed

    • for low-risk reads: fail open with fallback limits

  • stampede on shared keys → shard keys + local caching

SECTION 5 — CONTRACT

Return:

  • 429 with typed error RATE_LIMITED

  • Retry-After header

UI behavior:

  • show clear message

  • backoff and retry

SECTION 6 — OBSERVABILITY

  • rate-limited counts by key dimension (tenant, route)

  • top offenders

  • false positives (support signals)

SECTION 7 — EXERCISE

Design your limiter policy table:

  • endpoint

  • limit

  • scope (user/tenant/ip)

  • fail open/closed

🏁 END — PART III CASE STUDY