What is load balancing? How load balancers work and key algorithms

Load balancing is the process of distributing a set of tasks — usually incoming requests or traffic — over a set of servers so that overall processing becomes more efficient and no single machine is overloaded. A load balancer is the component that sits in front of a pool of servers and decides, for every request, which server should handle it. Done well, it improves response time, makes the service resilient to individual server failures, and lets you scale horizontally by adding more machines. This guide walks through how load balancers work, the Layer 4 vs Layer 7 distinction, the main routing algorithms, and how to pick the right one.

What a load balancer does

At its core, a load balancer is a reverse-proxy style component that intercepts traffic destined for a service and forwards each request to one member of a backend pool. In doing so it solves several problems at once:

• Availability. If one server goes down or stops responding, the load balancer stops sending it traffic and routes new requests to the healthy servers, keeping the service up. This is called failover.
• Scalability. Because traffic is spread across many servers, you can handle more requests by adding servers to the pool instead of making one machine more powerful.
• Performance. Distributing load avoids unevenly overloading some compute nodes while other nodes sit idle, which improves response time and utilization.
• Single entry point. Clients and DNS see one address; the complexity of many servers is hidden behind it.

The load balancer itself is typically aware of the health of the pool: it monitors servers, for example by fetching a known page or checking a health endpoint, and only routes traffic to members that respond. When a server comes back online, the balancer starts routing to it again.

Layer 4 vs Layer 7 load balancing

Load balancers are commonly described by the OSI layer they operate on. The two most common types are Layer 4 and Layer 7.

Layer 4 (transport) load balancing works at the transport layer and makes routing decisions using only header information such as IP address, port, and protocol. It does not look inside the payload. Because it does less inspection, Layer 4 balancing is very fast and has low overhead, which makes it a good fit for TCP and UDP traffic, database connections, VoIP, and other long-lived or high-throughput workloads.

Layer 7 (application) load balancing works at the application layer and can inspect the HTTP request itself — the URL path, headers, cookies, and body. This lets it route requests intelligently, for example sending /api traffic to one backend pool and /images to another, or using cookies for session affinity. It is more feature-rich but carries more processing overhead per request, so it is best suited to HTTP workloads where that intelligence pays off.

In practice many real-world deployments use both: a Layer 4 balancer in front for speed and a Layer 7 balancer for application-aware routing.

Load balancing algorithms

A load balancer needs a rule for choosing which server gets each request. These rules are called algorithms or scheduling methods, and they fall into two broad families: static algorithms, which ignore the current state of the servers, and dynamic algorithms, which take the servers' live load into account.

Round robin

Round robin is the simplest static method. The first request goes to the first server, the second to the second server, and so on down to the last, then it cycles back to the first. It is easy to set up and works well when all servers have roughly equal capacity and requests take similar time to process. Its weakness is that it ignores how loaded each server actually is.

Weighted round robin

Weighted round robin is round robin with a weight assigned to each server, so that more powerful servers receive proportionally more requests — and, depending on the implementation, receive them first. It is a good choice when servers in the pool differ in capacity.

Least connections

Least connections is a dynamic method that routes each new request to the server with the fewest active connections at that moment. It suits workloads where requests vary a lot in processing time or are long-lived, such as WebSocket connections, because a server with a few heavy connections is not confused for a lightly loaded one based on connection count alone. It can also be weighted.

Hashing

Hash-based methods map each request to a server using a hash of some value, most commonly the client IP address (IP hash). Because the same client always hashes to the same server, this provides session persistence (affinity) without reading cookies — useful for stateful applications. This is why IP-hash balancing is a popular choice for certain types of workloads.

Randomized and power-of-two choices

Randomized static load balancing simply assigns tasks to servers at random. Power of two choices picks two servers at random and sends the request to whichever of the two is less loaded. The latter is a surprisingly effective dynamic method: with only two random samples it balances load far better than pure random assignment and is used in distributed systems where a central scheduler is undesirable.

Health checks and failover

Load balancing is often used to implement failover — the continuation of service after the failure of one or more components. The servers are monitored continually; when one becomes unresponsive, the balancer is informed and stops sending it traffic, and when it comes back online the balancer restarts routing to it. For failover to keep capacity when a server drops, the pool needs at least one server in excess of the service's capacity (often called N+1 redundancy). Note that this technique can make the load balancer itself a single point of failure, so production setups usually balance at more than one level.

Choosing an algorithm

There is no universal best algorithm — the right choice depends on your traffic and your servers:

• Equal servers, similar requests: start with round robin. It is simple and predictable.
• Servers of different capacity: weighted round robin (or weighted least connections).
• Long-lived or variable-length connections (WebSockets, media): least connections.
• Stateful sessions without cookies: IP hashing for session affinity.
• HTTP with distinct resource types (paths, headers): a Layer 7 balancer so routing can be content-aware.

The practical pattern is to start with the simplest method that meets your needs, add health checks, and move to a more dynamic algorithm only when your workload shows it needs one.

FAQ

What is the difference between Layer 4 and Layer 7 load balancing?

Layer 4 load balancing makes routing decisions from transport-layer headers only (IP address, port, protocol) and is fast with low overhead, making it good for TCP/UDP and many long-lived workloads. Layer 7 load balancing inspects the HTTP request — path, headers, cookies, body — which allows content-aware routing and session affinity but costs more processing per request. Many systems use a Layer 4 balancer in front of a Layer 7 balancer to get both speed and intelligence.

What is the best load balancing algorithm?

There is no single best algorithm. Round robin is simplest and works for equal-capacity servers with similar requests; weighted round robin suits servers of different capacity; least connections suits long-lived or variable-length connections; and IP hashing gives session affinity for stateful apps. Pick the simplest method that meets your needs, then move to a more dynamic algorithm only if your workload requires it.

What is session affinity in load balancing?

Session affinity (also called sticky sessions or persistence) ensures that requests from a given client are consistently sent to the same server, which matters for stateful applications that keep session data in memory on one server. It can be achieved with cookies at Layer 7 or with IP hashing so that the same client always routes to the same server.

Does load balancing make the load balancer a single point of failure?

It can. If the load balancer itself fails, all the traffic through it stops. Production setups mitigate this by running load balancers in redundant pairs, balancing at multiple levels, or using DNS-level failover, so the balancing layer itself does not become the bottleneck.