Load Balancing Explained

Load balancing distributes incoming traffic across multiple servers. Without it, a single server handles all requests, becoming a bottleneck and single point of failure. Every distributed system design starts with a load balancer.

1. Why Load Balancing Matters

graph LR
    Client1["Client 1"]
    Client2["Client 2"]
    Client3["Client 3"]
    LB["Load Balancer"]
    S1["Server 1"]
    S2["Server 2"]
    S3["Server 3"]

    Client1 --> LB
    Client2 --> LB
    Client3 --> LB
    LB --> S1
    LB --> S2
    LB --> S3

    style Client1 fill:#FAF6EE,stroke:#E8DFC8
    style Client2 fill:#FAF6EE,stroke:#E8DFC8
    style Client3 fill:#FAF6EE,stroke:#E8DFC8
    style LB fill:#D97A2B,stroke:#B86418,color:#fff
    style S1 fill:#FAF6EE,stroke:#E8DFC8
    style S2 fill:#FAF6EE,stroke:#E8DFC8
    style S3 fill:#FAF6EE,stroke:#E8DFC8

• Horizontal scaling: Add more servers to handle increased load, instead of upgrading a single server.
• Fault tolerance: If one server dies, traffic routes to healthy servers. No downtime.
• Consistency: Ensure no single server is overwhelmed while others sit idle.

2. Load Balancing Algorithms

Round Robin

Requests are distributed sequentially: Server 1, Server 2, Server 3, Server 1, ... Simple and fair, but ignores server load. If one server is slower, it becomes a bottleneck.

Weighted Round Robin

Assign weights based on capacity. A server with weight 3 receives 3x the traffic. Useful when servers have different hardware specs.

Least Connections

Route to the server with fewest active connections. Best for long-lived connections like WebSocket or database connections. Naturally balances load because slow servers accumulate connections.

IP Hash

Hash the client's IP to determine which server handles the request. Same client always reaches the same server (session affinity). Problem: if a server goes down, all its sessions are lost.

Least Response Time

Route to the server with lowest average response time. Adapts to real-time performance but requires the load balancer to track response metrics.

3. L4 vs L7 Load Balancers

graph TD
    subgraph "L4 - Transport Layer"
        L4["Inspects: IP address, port
TCP/UDP level
Faster, no content inspection"]
    end
    subgraph "L7 - Application Layer"
        L7["Inspects: Headers, cookies, URLs
HTTP/HTTPS level
Content-based routing"]
    end

    style L4 fill:#FAF6EE,stroke:#D97A2B
    style L7 fill:#FAF6EE,stroke:#D97A2B

L4 (Transport Layer)

Operates at TCP/UDP level. Makes routing decisions based on IP address and port number. Faster because it doesn't inspect packet contents. Used for: database connections, game servers, streaming.

L7 (Application Layer)

Operates at HTTP/HTTPS level. Can inspect headers, cookies, URLs, and request bodies. Enables content-based routing:

# L7 routing rules
/api/*       → backend-servers
/static/*    → cdn-servers
/websocket/* → realtime-servers

4. Health Checks

graph LR
    LB["Load Balancer"]
    HC["Health Check"]
    S1["Server 1 ✓"]
    S2["Server 2 ✗"]
    S3["Server 3 ✓"]

    LB --> HC
    HC -->|"Probe every 10s"| S1
    HC -->|"Probe every 10s"| S2
    HC -->|"Probe every 10s"| S3
    HC -->|"Remove from pool"| S2

    style LB fill:#D97A2B,stroke:#B86418,color:#fff
    style HC fill:#FAF6EE,stroke:#E8DFC8
    style S1 fill:#D4EDDA,stroke:#28A745
    style S2 fill:#F8D7DA,stroke:#DC3545
    style S3 fill:#D4EDDA,stroke:#28A745

Load balancers continuously probe backend servers. If a server fails a health check, it is removed from the pool until it recovers.

• Active health checks: Load balancer sends probe requests (HTTP GET /health)
• Passive health checks: Load balancer monitors real traffic for errors (5xx responses, timeouts)

5. Sticky Sessions

Some applications require the same client to always reach the same server (e.g., session state stored in memory). Sticky sessions use cookies or IP hash to pin a client to a server.

Trade-off: Uneven distribution if traffic is skewed. Better to store session state in Redis (shared) rather than relying on sticky sessions.

6. Common Interview Mistakes

• Not using health checks: A dead server continues receiving traffic until clients report errors.
• Choosing the wrong algorithm: Round robin for long-lived connections wastes server capacity. Use least connections.
• Single load balancer: The LB itself becomes a single point of failure. Use a pair with VIP failover (e.g., AWS ELB).
• Ignoring SSL termination: Let the load balancer handle SSL/TLS to offload encryption from backend servers.
• Not considering geographic distribution: For global users, use DNS-based load balancing (Route 53) to route to the nearest region.

7. Summary