Clarify the requirementsCore idea
Functional
1-on-1 messages with text, images and files. Group chats of up to 100 members. Sent → delivered → read ticks. Online / offline / last-seen status. History synced across devices.
Non-functional
50M daily active users and 10B messages a day. Delivery under 200ms at p99. Messages in a conversation arrive in order. At-least-once delivery: nothing is ever lost.
Messages: 10B / day ≈ 115K messages/sec
Message size: ~100 bytes of text + metadata
Storage: 10B × 100 B ≈ 1 TB / day
Connections: up to 50M open WebSockets
Group fan-out: one message to a 100-member group = 100 deliveriesHigh-level architectureDesign
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","actorBkg":"#e7f5ff","actorBorder":"#1e1e1e","noteBkgColor":"#fff9db","noteBorderColor":"#1e1e1e"}}}%%
graph LR
Client["Client<br/>mobile / web"] -->|WebSocket| LB["Load balancer<br/>L7, sticky"]
LB --> CS["Chat servers"]
CS --> PR[("Presence<br/>Redis")]
CS -->|store| DB[("Messages<br/>Cassandra")]
CS -->|publish| Q["Kafka"]
CS -->|"files"| S3[("S3")]
Q -->|"offline users"| Push["Push service<br/>APNs / FCM"]
class Client blue
class LB,CS orange
class PR red
class DB violet
class Q yellow
class S3 grey
class Push green
classDef orange fill:#ffd8a8,stroke:#1e1e1e,color:#1e1e1e
classDef blue fill:#a5d8ff,stroke:#1e1e1e,color:#1e1e1e
classDef green fill:#b2f2bb,stroke:#1e1e1e,color:#1e1e1e
classDef red fill:#ffc9c9,stroke:#1e1e1e,color:#1e1e1e
classDef violet fill:#d0bfff,stroke:#1e1e1e,color:#1e1e1e
classDef yellow fill:#ffec99,stroke:#1e1e1e,color:#1e1e1e
classDef grey fill:#e9ecef,stroke:#1e1e1e,color:#1e1e1eLoad balancer
L7 with sticky sessions: a WebSocket is long-lived, so each user stays on the same chat server for the whole connection.
Chat servers
Hold the open WebSockets, route messages, fan out group messages and update presence. They keep only connections (no chat data), so you scale by adding servers, plus a registry of which server each user is on.
Presence (Redis)
Tracks who is online using heartbeats with a TTL. Fast answers to “is this user online?”.
Kafka
Separates receiving a message from delivering it. Consumers handle delivery, push and analytics independently.
Messages (Cassandra)
Built for appends, partitioned by conversation_id and ordered by time. Handles 115K writes/sec.
Push service
Sends APNs (iOS) and FCM (Android) notifications to offline users, triggered from Kafka.
Message flowsHow it works
1-on-1 message
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","actorBkg":"#e7f5ff","actorBorder":"#1e1e1e","noteBkgColor":"#fff9db","noteBorderColor":"#1e1e1e"}}}%%
sequenceDiagram
participant A as User A
participant S as Chat server
participant DB as Cassandra
participant Q as Kafka
participant B as User B
participant P as Push service
A->>S: send message (WebSocket)
S->>S: validate, assign message ID
S->>DB: store message
S->>Q: publish message event
alt B is online
Q->>B: deliver via WebSocket
B-->>S: ACK (delivered)
S->>DB: status = delivered
else B is offline
Q->>P: send push notification
P->>B: APNs / FCM
B->>S: reconnect, ask for missed messages
S->>DB: messages since last ACK
S->>B: deliver them
endGroup message
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","actorBkg":"#e7f5ff","actorBorder":"#1e1e1e","noteBkgColor":"#fff9db","noteBorderColor":"#1e1e1e"}}}%%
sequenceDiagram
participant A as User A
participant S as Chat server
participant DB as Cassandra
participant Q as Kafka
participant B as User B
participant C as User C
A->>S: send to group
S->>S: look up members (cached)
S->>DB: store one copy
S->>Q: publish group event
Q->>B: deliver
Q->>C: deliverDeep dive: WebSocket vs HTTP pollingCompare
With HTTP polling the client keeps asking “any new messages?”, even when there are none. A WebSocket keeps one connection open in both directions, so the server can push a message the moment it arrives.
const ws = new WebSocket("wss://chat.example.com/ws");
ws.onopen = () => {
ws.send(JSON.stringify({ type: "join", userId: "user123" }));
};
ws.onmessage = (event) => {
const msg = JSON.parse(event.data);
displayMessage(msg);
};| Factor | HTTP polling | WebSocket |
|---|---|---|
| Latency | Up to the polling interval (1–5 s) | Instant push (under 100ms) |
| Bandwidth | Wasteful: 90%+ of responses are empty | Only real messages |
| Server load | High: constant new requests | Low: one open connection |
| Real-time | No | Yes, both directions |
Deep dive: message storageDesign
Chat is append-heavy with reads by time, which suits Cassandra: writes are fast (no read before write), data is naturally split by conversation, and “latest N messages” is one ordered read.
CREATE TABLE messages (
conversation_id UUID,
message_id TIMEUUID,
sender_id UUID,
content TEXT,
content_type TEXT, -- "text", "image", "file"
file_url TEXT, -- S3 URL for media
created_at TIMESTAMP,
PRIMARY KEY (conversation_id, message_id)
) WITH CLUSTERING ORDER BY (message_id DESC);Never load a whole conversation. Page backwards with a cursor:
SELECT * FROM messages
WHERE conversation_id = ?
AND message_id < ? -- cursor: last message_id of the previous page
LIMIT 50;Deep dive: delivery guaranteesHow it works
Send
The user sends the message.
Store
Saved in Cassandra with a unique ID. The sender sees “sent”.
Publish
The event goes to Kafka.
Deliver
Pushed to the recipient’s WebSocket, or a push notification if offline.
ACK
The recipient’s app confirms; the status becomes “delivered”, then “read” when opened.
Deep dive: presence (online status)Design
Heartbeatrecommended
The client pings every 30 seconds; the server stores presence in Redis with a 60-second TTL. If the pings stop, the key expires and the user shows as offline.
WebSocket lifecyclesimpler
Mark the user offline when the socket closes. Less reliable on mobile, where a backgrounded app may keep the connection open.
// Client
setInterval(() => {
ws.send(JSON.stringify({ type: "heartbeat", userId: "user123" }));
}, 30000);
// Server
redis.set("presence:user123", "online", "EX", 60);Deep dive: group chat fan-outScale
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","actorBkg":"#e7f5ff","actorBorder":"#1e1e1e","noteBkgColor":"#fff9db","noteBorderColor":"#1e1e1e"}}}%%
graph LR
A["User A sends to group<br/>members: B, C, D"] --> S["Chat server"]
S --> MC[("member list<br/>Redis cache")]
S --> ST[("store once<br/>Cassandra")]
S --> F["fan-out"]
F --> B["B"]
F --> C["C"]
F --> D["D"]
class A blue
class S orange
class MC red
class ST violet
class F yellow
class B,C,D green
classDef orange fill:#ffd8a8,stroke:#1e1e1e,color:#1e1e1e
classDef blue fill:#a5d8ff,stroke:#1e1e1e,color:#1e1e1e
classDef green fill:#b2f2bb,stroke:#1e1e1e,color:#1e1e1e
classDef red fill:#ffc9c9,stroke:#1e1e1e,color:#1e1e1e
classDef violet fill:#d0bfff,stroke:#1e1e1e,color:#1e1e1e
classDef yellow fill:#ffec99,stroke:#1e1e1e,color:#1e1e1e
classDef grey fill:#e9ecef,stroke:#1e1e1e,color:#1e1e1e| Approach | Write cost | Read cost | Use when |
|---|---|---|---|
| Fan-out on send | High: one delivery per member | Low: one query | Small groups (under ~100) |
| Fan-out on read | Low: one write | High: gathered at read time | Large groups (1000+) |
Common interview mistakesPitfalls
Summary: key decisionsSummary
| Decision | Recommendation | Why |
|---|---|---|
| Protocol | WebSocket | Persistent, two-way, low latency |
| Message DB | Cassandra | Built for appends, partitioned by conversation |
| Presence | Redis with TTL | Fast reads; expiry detects offline users |
| Group fan-out | On send (small groups) | Cheap reads; acceptable write cost under 100 members |
| Push notifications | Async via Kafka | Decoupled, retried, no delay on sending |
Frequently asked questionsFAQ
What is the difference between HTTP polling and WebSockets for chat?
Polling repeatedly requests the server for new messages, wasting bandwidth and adding latency, while WebSockets keep one persistent bidirectional connection so either side pushes instantly. Chat systems use WebSockets for the live path and a database they replay to late or offline users.
How do you know if a user is online or offline?
Track presence from live connection signals such as socket heartbeats and store that state in a fast store like Redis with a short TTL. Show typing indicators and last-seen from heartbeat timing; when a user disconnects, the TTL expires and marks them offline without waiting for an explicit logout.
Do you need to be online to receive messages?
No. Messages are persisted in the database, and an offline user's client fetches the backlog when it reconnects. That is why chat storage is a durability concern: delivery status, message IDs, and read receipts are state transitions over a store, not just socket events.
Now explain chat system to me out loud
I'll give you a design problem and push on the trade-offs, like a real interviewer.