Load Balancing Strategies

Layer 4 load balancing operates on the transport layer: it forwards TCP/UDP connections based on IP addresses and ports without looking inside the packets — very fast and protocol-agnostic, but unaware of application content. Layer 7 load balancing understands the application protocol (typically HTTP): it can route based on the URL path, host header, cookies, or method (e.g. send /api to one pool and /images to another), terminate TLS, and do content-based routing. L7 is more flexible and feature-rich; L4 is lower-latency and simpler.

Round-robin distributes requests by simply cycling through the server list in order — simple and fair when requests are uniform. But if some requests are long-running, a server can accumulate heavy work while others sit idle. Least-connections adapts by routing each new request to the server with the fewest active connections, which better balances load when request durations vary widely. Weighted variants of both let you account for servers of different capacities. The right choice depends on how uniform your request workloads are.

A health check is a probe (a TCP connect, or an HTTP request to a /health endpoint) the load balancer issues to each backend at intervals. If a server fails the check (timeouts, error status, too few successes), the load balancer removes it from the rotation so no traffic is sent to a broken instance; when it passes again, it is restored. This is what makes a pool self-healing and enables zero-downtime deploys (a draining server fails readiness and is gracefully removed). Good health checks reflect real readiness, not just "the process is up."

Session affinity (sticky sessions) ensures a client's requests always reach the same backend, typically tracked via a cookie or source IP. It is useful when a server holds local session state, so the client must return to the server that has its data. The drawbacks: load becomes uneven (some servers get busier clients), scaling is harder (new servers do not help existing sticky clients), and a server failure loses those sessions. The better long-term fix is to externalize session state (e.g. Redis) so any server can handle any request, removing the need for stickiness.

With naive hash(key) % N distribution, changing N (adding/removing a server) changes the modulus and reshuffles nearly every key to a different server — catastrophic for caches (mass invalidation) or stateful sharding. Consistent hashing places servers and keys on a hash ring; a key maps to the next server clockwise. When a server is added or removed, only the keys in its arc move — roughly 1/N of keys — leaving the rest stable. Virtual nodes smooth out distribution. It is fundamental to distributed caches (e.g. Memcached clients) and sharded data stores.