Why load balancing mattersCore idea
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","actorBkg":"#e7f5ff","actorBorder":"#1e1e1e","noteBkgColor":"#fff9db","noteBorderColor":"#1e1e1e"}}}%%
graph LR
C1["Client 1"] --> LB["Load Balancer"]
C2["Client 2"] --> LB
C3["Client 3"] --> LB
LB --> S1["Server 1"]
LB --> S2["Server 2"]
LB --> S3["Server 3"]
class C1,C2,C3 blue
class LB orange
class S1,S2,S3 green
classDef orange fill:#ffd8a8,stroke:#1e1e1e,color:#1e1e1e
classDef blue fill:#a5d8ff,stroke:#1e1e1e,color:#1e1e1e
classDef green fill:#b2f2bb,stroke:#1e1e1e,color:#1e1e1e
classDef red fill:#ffc9c9,stroke:#1e1e1e,color:#1e1e1e
classDef violet fill:#d0bfff,stroke:#1e1e1e,color:#1e1e1e
classDef yellow fill:#ffec99,stroke:#1e1e1e,color:#1e1e1e
classDef grey fill:#e9ecef,stroke:#1e1e1e,color:#1e1e1eHorizontal scaling
Add more servers to handle more load, instead of buying one bigger server.
Fault tolerance
If one server dies, traffic goes to the healthy ones. Users see no downtime.
Even load
No single server is overwhelmed while others sit idle.
Load balancing algorithmsHow it works
The algorithm is the rule the load balancer uses to pick a server for each request.
Round Robinsimple
Server 1, Server 2, Server 3, Server 1… Simple and fair, but ignores load: a slow server still gets its full share.
Weighted Round Robincapacity
Give each server a weight. A server with weight 3 gets 3x the traffic. Useful when machines have different specs.
Least Connectionslong-lived
Send to the server with the fewest open connections. Best for WebSockets and database connections: slow servers pile up connections and get skipped.
IP Hashsticky
Hash the client IP to pick a server, so the same client always lands on the same one. If that server dies, its sessions are lost.
Least Response Timeadaptive
Send to the server with the lowest average response time. Adapts in real time, but the balancer must track response metrics.
| Algorithm | Best for | Drawback |
|---|---|---|
| Round Robin | Equal-capacity servers, short requests | Ignores server load |
| Least Connections | Long-lived connections (WebSocket) | Requires connection tracking |
| IP Hash | Session affinity without shared state | Uneven distribution, session loss on failure |
| Least Response Time | Heterogeneous servers | Requires metrics collection |
L4 vs L7 load balancersCompare
The “L” is the network layer the balancer works at. The lower it works, the less it can see, and the faster it is.
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","actorBkg":"#e7f5ff","actorBorder":"#1e1e1e","noteBkgColor":"#fff9db","noteBorderColor":"#1e1e1e"}}}%%
graph TD
subgraph L4["L4 · transport layer"]
A["Sees: IP address + port<br/>TCP / UDP level<br/>fast, no content inspection"]
end
subgraph L7["L7 · application layer"]
B["Sees: headers, cookies, URL<br/>HTTP / HTTPS level<br/>content-based routing"]
end
class A blue
class B violet
classDef orange fill:#ffd8a8,stroke:#1e1e1e,color:#1e1e1e
classDef blue fill:#a5d8ff,stroke:#1e1e1e,color:#1e1e1e
classDef green fill:#b2f2bb,stroke:#1e1e1e,color:#1e1e1e
classDef red fill:#ffc9c9,stroke:#1e1e1e,color:#1e1e1e
classDef violet fill:#d0bfff,stroke:#1e1e1e,color:#1e1e1e
classDef yellow fill:#ffec99,stroke:#1e1e1e,color:#1e1e1e
classDef grey fill:#e9ecef,stroke:#1e1e1e,color:#1e1e1e| L4 (transport) | L7 (application) | |
|---|---|---|
| Routes on | IP address and port | Headers, cookies, URL path, body |
| Speed | Faster: never reads the content | Slower: parses every HTTP request |
| Can do | Plain forwarding | URL routing, TLS termination, caching, auth |
| Used for | Databases, game servers, streaming | Web apps and APIs |
An L7 balancer can send different URL paths to different server pools:
/api/* → backend-servers
/static/* → cdn-servers
/websocket/* → realtime-serversHealth checksHow it works
%%{init: {"look":"handDrawn","handDrawnSeed":7,"theme":"base","fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","themeVariables":{"fontFamily":"Virgil, \"Segoe Print\", \"Comic Sans MS\", cursive","fontSize":"16px","primaryColor":"#fff4e6","primaryBorderColor":"#1e1e1e","primaryTextColor":"#1e1e1e","secondaryColor":"#e7f5ff","tertiaryColor":"#ebfbee","lineColor":"#1e1e1e","textColor":"#1e1e1e","mainBkg":"#fff4e6","nodeBorder":"#1e1e1e","clusterBkg":"#f8f9fa","edgeLabelBackground":"#ffffff","actorBkg":"#e7f5ff","actorBorder":"#1e1e1e","noteBkgColor":"#fff9db","noteBorderColor":"#1e1e1e"}}}%%
graph LR
LB["Load Balancer"] --> HC["Health checker"]
HC -->|"GET /health every 10s"| S1["Server 1 ✓"]
HC -->|"GET /health every 10s"| S2["Server 2 ✗"]
HC -->|"GET /health every 10s"| S3["Server 3 ✓"]
class LB orange
class HC yellow
class S1,S3 green
class S2 red
classDef orange fill:#ffd8a8,stroke:#1e1e1e,color:#1e1e1e
classDef blue fill:#a5d8ff,stroke:#1e1e1e,color:#1e1e1e
classDef green fill:#b2f2bb,stroke:#1e1e1e,color:#1e1e1e
classDef red fill:#ffc9c9,stroke:#1e1e1e,color:#1e1e1e
classDef violet fill:#d0bfff,stroke:#1e1e1e,color:#1e1e1e
classDef yellow fill:#ffec99,stroke:#1e1e1e,color:#1e1e1e
classDef grey fill:#e9ecef,stroke:#1e1e1e,color:#1e1e1eActive checks
The load balancer sends probe requests itself, such as HTTP GET /health, on a timer.
Passive checks
The load balancer watches real traffic and marks a server bad after errors (5xx responses, timeouts).
Sticky sessionsDesign
Some apps keep session state in a server's memory, so the same user must keep reaching the same server. Sticky sessions pin a client to one server using a cookie or an IP hash.
Common interview mistakesPitfalls
How to bring it up in the interviewIn the interview
One sentence like that covers the layer, the algorithm, where state lives and how failures are handled.
Frequently asked questionsFAQ
What are the main load-balancing algorithms?
Round robin cycles servers in order, least connections routes to the least busy server, and consistent or IP hashing pins clients to servers when sticky sessions are needed. Weighted variants let stronger servers take a larger share of traffic.
What is the difference between L4 and L7 load balancing?
L4 balancing routes on transport-layer data like IP and port, so it is fast and sees only connection metadata. L7 balancing inspects HTTP headers and paths, enabling URL-based routing, caching, and authentication at the cost of more CPU per request.
How do you detect that a backend is unhealthy?
Active health checks periodically probe an endpoint and remove failing servers, while passive checks mark servers after repeated timeouts or 5xx responses. Combine both: keep probing removed servers so they rejoin automatically once they recover.
Now explain load balancing to me out loud
I'll give you a design problem and push on the trade-offs, like a real interviewer.