How to Design a Chat System

Designing a chat system like WhatsApp, Slack, or iMessage tests your understanding of real-time communication, WebSocket protocols, and distributed message delivery. This tutorial covers the core building blocks with architecture diagrams and deep dives into each component.

1. Requirements Clarification

Functional Requirements

• 1-on-1 messaging: Users can send and receive text, images, and files
• Group chats: Up to 100 members per group
• Delivery status: Sent → Delivered → Read indicators
• Presence: Online/offline/last seen status
• Message history: Sync across devices

Non-Functional Requirements

• Scale: 50M daily active users, 10B messages/day
• Latency: <200ms for 99th percentile message delivery
• Ordering: Messages within a conversation must arrive in sequence
• Reliability: At-least-once delivery (no lost messages)

Back-of-Envelope Estimates

Messages: 10B / day = ~115K messages/sec
Message size: ~100 bytes (text metadata)
Daily storage: 10B × 100B = ~1 TB/day
Connections: 50M concurrent WebSocket connections
Fan-out: Group message (100 members) = 100 writes per send

2. High-Level Architecture

graph LR
    Client["Client (Mobile/Web)"]
    LB["Load Balancer
(L7 - Sticky Sessions)"]
    ChatServer["Chat Server
(WebSocket)"]
    PresenceService["Presence Service
(Redis)"]
    MessageQueue["Message Queue
(Kafka)"]
    MessageDB["Message Store
(Cassandra)"]
    PushService["Push Notification
(APNs / FCM)"]
    FileStorage["File Storage
(S3)"]

    Client <-->|"WebSocket"| LB
    LB --> ChatServer
    ChatServer --> PresenceService
    ChatServer -->|"Publish message"| MessageQueue
    ChatServer -->|"Store message"| MessageDB
    ChatServer -->|"Upload files"| FileStorage
    MessageQueue -->|"Deliver to offline users"| PushService

    style Client fill:#FAF6EE,stroke:#E8DFC8
    style LB fill:#FAF6EE,stroke:#E8DFC8
    style ChatServer fill:#D97A2B,stroke:#B86418,color:#fff
    style PresenceService fill:#FAF6EE,stroke:#E8DFC8
    style MessageQueue fill:#FAF6EE,stroke:#E8DFC8
    style MessageDB fill:#FAF6EE,stroke:#E8DFC8
    style PushService fill:#FAF6EE,stroke:#E8DFC8
    style FileStorage fill:#FAF6EE,stroke:#E8DFC8

Component Responsibilities

• Load Balancer: L7 with sticky sessions. Routes each user to the same chat server for the duration of their session. WebSocket connections are long-lived : you can't randomly route mid-connection.
• Chat Server: The core. Maintains WebSocket connections, handles message routing, fan-out for groups, and presence updates. Stateless : horizontal scaling by adding more servers.
• Presence Service: Redis-backed. Tracks who's online/offline using TTL-based heartbeats. Fast reads for the "is this user online?" check.
• Message Queue (Kafka): Decouples message ingestion from delivery. Each message is an event. Consumers handle fan-out, push notifications, and analytics independently.
• Message Store (Cassandra): Append-optimized. Partitioned by conversation_id, clustered by timestamp. Handles 115K writes/sec with proper partitioning.
• Push Service: Sends APNs (iOS) and FCM (Android) notifications for offline users. Triggered by Kafka consumers.

3. Sequence Diagrams

1-on-1 Message Flow

sequenceDiagram
    participant A as User A
    participant S as Chat Server
    participant DB as Cassandra
    participant Q as Kafka
    participant B as User B
    participant P as Push Service

    A->>S: Send message (WebSocket)
    S->>S: Validate, assign message ID
    S->>DB: Store message
    S->>Q: Publish message event
    alt User B online
        Q->>B: Deliver via WebSocket
        B-->>S: ACK (delivered)
        S->>DB: Update status to "delivered"
    else User B offline
        Q->>P: Send push notification
        P->>B: APNs/FCM notification
        B->>S: Reconnect, fetch missed messages
        S->>DB: Query messages since last ACK
        S->>B: Deliver queued messages
    end

Group Message Flow (Fan-out on Send)

sequenceDiagram
    participant A as User A
    participant S as Chat Server
    participant DB as Cassandra
    participant Q as Kafka
    participant B as User B
    participant C as User C

    A->>S: Send message to group
    S->>S: Look up group members (cached)
    S->>DB: Store one copy of message
    S->>Q: Publish group message event
    Q->>B: Deliver to User B
    Q->>C: Deliver to User C
    Note over B,C: Each user gets their own copy in their conversation feed

4. Deep Dive: WebSocket vs HTTP Polling

HTTP polling is wasteful : the client repeatedly asks "any new messages?" even when there are none. WebSocket maintains a persistent, full-duplex connection between client and server. Messages flow bidirectionally with minimal overhead.

// WebSocket connection
const ws = new WebSocket("wss://chat.example.com/ws");

ws.onopen = () => {
  ws.send(JSON.stringify({ type: "join", userId: "user123" }));
};

ws.onmessage = (event) => {
  const msg = JSON.parse(event.data);
  displayMessage(msg);
};

Why WebSocket Wins

5. Deep Dive: Message Storage

Chat messages are append-heavy with time-ordered reads. Cassandra is ideal because: writes are fast (no read-before-write), data is naturally partitioned by conversation ID, and time-based queries are efficient with clustering keys.

CREATE TABLE messages (
  conversation_id UUID,
  message_id TIMEUUID,
  sender_id UUID,
  content TEXT,
  content_type TEXT,      -- "text", "image", "file"
  file_url TEXT,          -- S3 URL for media
  created_at TIMESTAMP,
  PRIMARY KEY (conversation_id, message_id)
) WITH CLUSTERING ORDER BY (message_id DESC);

Why Cassandra?

• Write path: No read-before-write. Append directly. Critical for 115K msgs/sec.
• Partitioning: Data is partitioned by conversation_id. All messages for a chat live on the same node : no cross-partition joins.
• Time queries: Clustering by message_id DESC means "get last 50 messages" is a single partition read.
• Replication: Tunable consistency. Use QUORUM for writes (no lost messages), ONE for reads (fast).

Message Pagination

Never load an entire conversation. Use cursor-based pagination:

-- Get last 50 messages
SELECT * FROM messages 
WHERE conversation_id = ? 
AND message_id < ?   -- cursor (last message_id from previous page)
LIMIT 50;

6. Deep Dive: Delivery Guarantees

graph TD
    Send["User sends message"]
    Store["Store in Cassandra
(assign unique ID)"]
    Publish["Publish to Kafka"]
    Deliver["Deliver to recipient"]
    ACK["Recipient ACKs"]
    StatusUpdate["Update status to 'delivered'"]

    Send --> Store
    Store --> Publish
    Publish --> Deliver
    Deliver --> ACK
    ACK --> StatusUpdate

    style Send fill:#FAF6EE,stroke:#E8DFC8
    style Store fill:#D97A2B,stroke:#B86418,color:#fff
    style Publish fill:#FAF6EE,stroke:#E8DFC8
    style Deliver fill:#FAF6EE,stroke:#E8DFC8
    style ACK fill:#FAF6EE,stroke:#E8DFC8
    style StatusUpdate fill:#FAF6EE,stroke:#E8DFC8

Chat systems need at-least-once delivery. Messages must not be lost, but duplicates are acceptable (the client deduplicates using message ID). For exactly-once semantics in group chats, use a combination of idempotent message IDs and deduplication on the client side.

Message Statuses

• Sent: Message stored in DB, published to Kafka
• Delivered: Recipient's client ACKed receipt
• Read: Recipient opened the conversation and viewed the message

7. Deep Dive: Presence and Online Status

Track which users are online. Two approaches:

Approach 1: Heartbeat (Recommended)

Client sends a ping every 30 seconds. Server stores presence in Redis with TTL of 60 seconds. If the heartbeat stops, the key expires and the user is marked offline.

// Client-side heartbeat
setInterval(() => {
  ws.send(JSON.stringify({ type: "heartbeat", userId: "user123" }));
}, 30000);

// Server-side
redis.set("presence:user123", "online", "EX", 60);

Approach 2: WebSocket Lifecycle

Server tracks connection state directly. When WebSocket closes, mark user offline. Simpler but less reliable : mobile apps may keep the connection open but background the app.

Which to Choose?

Heartbeat is more reliable for mobile. Use a combination: WebSocket lifecycle for desktop, heartbeat for mobile. Store both in Redis for fast reads.

8. Deep Dive: Group Chat Fan-out

graph TD
    A["User A sends to group
(members: B, C, D)"]
    Server["Chat Server"]
    Cache["Member list cache
(Redis)"]
    Store["Store 1 message copy
in Cassandra"]
    FanOut["Fan-out: create N copies
(one per recipient)"]
    BFeed["B's feed"]
    CFeed["C's feed"]
    DFeed["D's feed"]

    A --> Server
    Server --> Cache
    Cache --> Server
    Server --> Store
    Server --> FanOut
    FanOut --> BFeed
    FanOut --> CFeed
    FanOut --> DFeed

    style A fill:#FAF6EE,stroke:#E8DFC8
    style Server fill:#D97A2B,stroke:#B86418,color:#fff
    style Cache fill:#FAF6EE,stroke:#E8DFC8
    style Store fill:#FAF6EE,stroke:#E8DFC8
    style FanOut fill:#FAF6EE,stroke:#E8DFC8

For group messages, use a fan-out approach: when a user sends a message to a group, the chat server creates N copies of the message (one per recipient) and stores them in each recipient's conversation feed. This trades write amplification for read simplicity : fetching a conversation is a single query.

Fan-out on Send vs Fan-out on Read

9. Common Interview Mistakes

• Using HTTP for real-time messaging: Polling wastes bandwidth and adds latency. Always propose WebSocket first.
• Not handling message ordering: Messages within a conversation must arrive in sequence. Use monotonic message IDs (Snowflake or TIMEUUID).
• Ignoring offline message queuing: Messages sent while the recipient is offline must be stored and delivered later. Don't drop them.
• Not planning for reconnection: Mobile clients frequently disconnect and reconnect. The server must resume from the last acknowledged message.
• Forgetting about end-to-end encryption: WhatsApp uses the Signal Protocol. Mention this in interviews even if you don't implement it.
• Ignoring message size limits: Set a max message size (e.g., 100KB for text, 100MB for files). Reject oversized messages at the API layer.

10. Summary: Key Decisions