Open on desktop
Antimetal's interactive diagrams require a larger screen. Open this page on your laptop or desktop to continue.
Load Balancer Deep Dive
§1Step 2 — High-Level Design
Layer 4 vs Layer 7 balancing, sticky sessions, health checks, and least-connection algorithms.
Connect Redis to the load balancer to store session-to-backend mappings for sticky session support.
Redis stores the mapping between user sessions and backend servers, enabling the load balancer to route requests from the same user to the same backend server (sticky sessions).
Some backends maintain in-memory state (shopping carts, websocket connections) that can't be shared across instances. Sticky sessions ensure the user always reaches the backend holding their state.
Sticky sessions reduce the load balancer's ability to distribute load evenly. If be-1 has 1000 stickied sessions and gets hot, those sessions can't be moved to be-2 without breaking them. Prefer stateless backends (state in Redis/DB) over sticky sessions.
AWS ALB supports sticky sessions via cookies. HAProxy supports sticky sessions via Redis. Nginx Plus uses the sticky directive. Many legacy enterprise apps still require sticky sessions.
10M active sessions × 30 bytes per mapping (session ID + backend ID) = 300MB Redis. Lookup is O(1) at < 1ms. Redis handles 1M session lookups/second.
§2Step 3 — Deep Dive
Redis stores the mapping between user sessions and backend servers, enabling the load balancer to route requests from the same user to the same backend server (sticky sessions).
| Algorithm | Session affinity | Handles slow backends? | Overhead | Best for | Cost | Ops burden |
|---|---|---|---|---|---|---|
| Round Robin | No | No | O(1) | Stateless, homogeneous backends | Low | Low |
| Least Connections | No | Yes | O(n) | Variable request duration ✓ | Low | Low |
| IP Hash | Yes (per client IP) | No | O(1) | Session-based, sticky sessions | Low | Low |
| Weighted Round Robin | No | Partial | O(1) | Heterogeneous server capacity | Low | Low |
| Random with 2 choices | No | Yes | O(1) | Distributed LBs, no coordination | Low | Low |
Load balancing algorithms — Round Robin for equal servers, Least Connections for variable load.
upstream api_backends {
least_conn;
server backend-1:8080 weight=1 max_fails=3 fail_timeout=30s;
server backend-2:8080 weight=1 max_fails=3 fail_timeout=30s;
server backend-3:8080 weight=1 max_fails=3 fail_timeout=30s;
keepalive 64;
}
server {
listen 80;
location /api/ {
proxy_pass http://api_backends;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header X-Real-IP $remote_addr;
proxy_connect_timeout 2s;
proxy_read_timeout 10s;
proxy_next_upstream error timeout http_503;
}
}| Component | Why Add It | Tradeoff |
|---|---|---|
| Redis for Session Affinity | Some backends maintain in-memory state (shopping carts, websocket connections) that can't be shared across instances. | Sticky sessions reduce the load balancer's ability to distribute load evenly. |
Design decision tradeoffs
Backend 2 fails health check. Load balancer should drain existing connections and stop routing new requests to it.
be-2 starts returning 500 errors but doesn't crash — health checks miss it because they only check TCP connectivity. How do you implement active health checks using HTTP probes, track error rate per backend, and remove be-2 from rotation based on error thresholds?
A traffic surge hits lb-1. Round-robin sends equal requests to all backends, but be-3 and be-4 are on slower hardware and fall behind. be-1 and be-2 are idle while be-3/be-4 queue grows to 5000 requests. How do you switch to least-connections or weighted round-robin algorithms?
§3Step 4 — Wrap Up
| Decision | Choice | Why |
|---|---|---|
| Redis for Session Affinity | Redis stores the mapping between user sessions and backend servers, enabling the load balancer to route requests from the same user to the same backend server (sticky sessions). | Some backends maintain in-memory state (shopping carts, websocket connections) that can't be shared across instances. |
Key design decisions