Clarify the requirementsCore idea
Before picking an algorithm, pin down what is being limited and how finely.
Functional
Limit by client, API key, user ID or IP. Reject extra requests with a clear 429 error. Configurable rules (for example 100 requests/min on the free tier). Different kinds of limit: requests/sec, bytes/sec, concurrent connections.
Non-functional
Under 1ms added per check. No bursts that slip past the limit. If the limiter fails, it must not take the whole API down. The same limits on every API server.
The four classic algorithmsCompare
Fixed window countersimple
Count requests per fixed bucket, like per minute. Simple, but allows a double burst at the boundary: 60 requests in the last second of one minute plus 60 in the first second of the next is 120 in two seconds.
Sliding window logprecise
Store a timestamp for every request and drop the old ones on each check. Exact, but uses O(n) memory per key.
Sliding window countersmooth
Blend the previous window’s count with the current one: weighted = prev × overlap + current. Smooth, without storing every request.
Token bucketfavourite
A bucket holds up to capacity tokens and refills at a steady rate. Each request spends one token; an empty bucket means reject. Allows short bursts but enforces the long-run average.
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","actorBkg":"#e7f5ff","actorBorder":"#1e1e1e","noteBkgColor":"#fff9db","noteBorderColor":"#1e1e1e"}}}%%
graph LR
subgraph W1["minute 1 · limit 60"]
a["…quiet…"] --> b["60 requests<br/>in the last second"]
end
subgraph W2["minute 2 · limit 60"]
c["60 requests<br/>in the first second"] --> d["…quiet…"]
end
b --> c
class b,c red
class a,d grey
classDef orange fill:#ffd8a8,stroke:#1e1e1e,color:#1e1e1e
classDef blue fill:#a5d8ff,stroke:#1e1e1e,color:#1e1e1e
classDef green fill:#b2f2bb,stroke:#1e1e1e,color:#1e1e1e
classDef red fill:#ffc9c9,stroke:#1e1e1e,color:#1e1e1e
classDef violet fill:#d0bfff,stroke:#1e1e1e,color:#1e1e1e
classDef yellow fill:#ffec99,stroke:#1e1e1e,color:#1e1e1e
classDef grey fill:#e9ecef,stroke:#1e1e1e,color:#1e1e1eToken bucketPattern
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","actorBkg":"#e7f5ff","actorBorder":"#1e1e1e","noteBkgColor":"#fff9db","noteBorderColor":"#1e1e1e"}}}%%
graph LR
R["refill:<br/>10 tokens/sec"] --> B["bucket<br/>capacity 20"]
Req["request"] --> B
B -->|"token available"| OK["allow,<br/>spend 1 token"]
B -->|"bucket empty"| No["reject: 429"]
class R blue
class B yellow
class OK green
class No red
classDef orange fill:#ffd8a8,stroke:#1e1e1e,color:#1e1e1e
classDef blue fill:#a5d8ff,stroke:#1e1e1e,color:#1e1e1e
classDef green fill:#b2f2bb,stroke:#1e1e1e,color:#1e1e1e
classDef red fill:#ffc9c9,stroke:#1e1e1e,color:#1e1e1e
classDef violet fill:#d0bfff,stroke:#1e1e1e,color:#1e1e1e
classDef yellow fill:#ffec99,stroke:#1e1e1e,color:#1e1e1e
classDef grey fill:#e9ecef,stroke:#1e1e1e,color:#1e1e1eclass TokenBucket {
private tokens: number;
private lastRefill: number;
constructor(private capacity: number, private refillPerSec: number) {
this.tokens = capacity;
this.lastRefill = Date.now();
}
allow(): boolean {
const now = Date.now();
this.tokens = Math.min(this.capacity, this.tokens + ((now - this.lastRefill) / 1000) * this.refillPerSec);
this.lastRefill = now;
if (this.tokens < 1) return false;
this.tokens -= 1;
return true;
}
}High-level architectureDesign
The limiter sits in front of the API servers, usually at the API gateway or edge, so rejected work never reaches the backend.
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","actorBkg":"#e7f5ff","actorBorder":"#1e1e1e","noteBkgColor":"#fff9db","noteBorderColor":"#1e1e1e"}}}%%
graph LR
C["Clients"] --> G["API gateway / edge"]
G --> RL["Rate limiter"]
RL --> R[("Redis<br/>counters, buckets")]
RL -->|allow| API["API servers"]
RL -->|"429 reject"| C
class C blue
class G,RL orange
class R red
class API green
classDef orange fill:#ffd8a8,stroke:#1e1e1e,color:#1e1e1e
classDef blue fill:#a5d8ff,stroke:#1e1e1e,color:#1e1e1e
classDef green fill:#b2f2bb,stroke:#1e1e1e,color:#1e1e1e
classDef red fill:#ffc9c9,stroke:#1e1e1e,color:#1e1e1e
classDef violet fill:#d0bfff,stroke:#1e1e1e,color:#1e1e1e
classDef yellow fill:#ffec99,stroke:#1e1e1e,color:#1e1e1e
classDef grey fill:#e9ecef,stroke:#1e1e1e,color:#1e1e1eDistributed implementation with RedisScale
Every API node must agree on the count, so the counters live in Redis. Make the read-check-write atomic(all in one step) so two nodes can't both let a request through on the last slot. A Lua script runs atomically inside Redis:
-- Lua runs atomically inside Redis
local key = KEYS[1]
local limit = tonumber(ARGV[1])
local current = redis.call('INCR', key)
if current == 1 then redis.call('EXPIRE', key, ARGV[2]) end
if current > limit then redis.call('DECR', key) return 0 end
return 1That is one round trip per request, well under the 1ms budget when Redis is nearby. For huge scale, spread the keys across a small Redis cluster by client hash, using the same consistent hashing you would use for cache keys.
Where to put the limiterCompare
| Placement | Verdict |
|---|---|
| Client-side | Cosmetic only: clients can be changed. |
| API gateway | The standard answer: central, language-independent, protects every service. |
| Inside each service | Less consistent across nodes; only good if requests always hit the same node. |
| Reverse proxy (nginx) | The coarsest layer, often combined with an app-level limiter. |
Responses and error designHow it works
Rejected requests get 429 Too Many Requests plus headers the client can act on. Log which rule fired and return a stable error code in the body, so client retry loops behave sensibly.
HTTP/1.1 429 Too Many Requests
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
Retry-After: 30
{ "error": "rate_limited", "rule": "free-tier-per-minute" }Common interview mistakesPitfalls
Summary: key decisionsSummary
Clarify
What is limited (user, key, IP), at what granularity and scale.
Algorithm
Token bucket, with a sliding window where smoothness matters.
Placement
At the API gateway.
Storage
Counters in Redis, updated with an atomic Lua script.
Response
429 with Retry-After, then name the trade-offs of each choice.
Frequently asked questionsFAQ
Is rate limiting in-memory or do you always need Redis?
For a single server an in-process bucket is fine and adds zero latency. Across multiple API nodes you need shared state, and Redis is the usual answer because one counter can be read and incremented atomically. Saying in-process when sticky, Redis when shared is the nuance interviewers reward.
How does the token bucket allow bursts?
The bucket starts full at its capacity, so a client can immediately spend up to that many tokens in a burst. Afterward it refills at the steady rate until full again. That is exactly why token bucket suits APIs where legitimate traffic occasionally spikes but must not exceed a sustained average.
How is rate limiting related to caching?
Both are pre-computation layers and both use Redis-style stores, so the memory, eviction, and consistency lessons carry over. A rate-limiter read is essentially a cache lookup with atomic accounting, which is why it pairs with the cache patterns in our caching tutorial.
Now explain rate limiting to me out loud
I'll give you a design problem and push on the trade-offs, like a real interviewer.