How a Load Balancer Sticks
Part two of load balancing strategies: the sticky rules, source-IP hash, URL hash, consistent hashing, Maglev, and cookie-based sticky sessions, each with its own visualizer.
A dedicated look at the sticky load-balancing rules, the ones that send the same key to the same server. Why you want it (sessions and caches), then each rule with its own analogy and visualizer: source-IP hash, URL hash, the reshuffling problem, consistent hashing (the ring), Maglev (a fast table), and cookie-based sticky sessions. Part two of two; part one covered the even-spreading rules.
Part one covered the rules that spread requests evenly (how a load balancer spreads the load). This part is the opposite instinct: sometimes you want the same client, or the same request, to keep landing on the same server. Here is why, and the family of rules that do it.
Think of a regular at a barbershop who always books the same barber. Not because that barber is less busy, but because that barber remembers exactly how they like their hair. The server is the barber, and what it remembers is the reason to come back to it. That memory is stickiness, and it matters for two things.
Sessions. A server may be holding something about you in its memory: your login, your shopping cart. If your next request lands on a different server, that memory is gone. Caches. If each server caches whatever it has served, then sending the same page to the same server means it is probably still warm in that server's cache, a fast hit instead of a slow miss. Both want the same thing: the same key back to the same server.
Hashing by client: source-IP hash
The simplest way to be sticky: take something that identifies the client, its IP address, and run it through a hash to pick a server. The same IP always hashes to the same number, so the same client always lands on the same server. No cookie, no memory on the balancer, and it works at layer 4.
Hash the client's IP to pick a server, so a given client always lands on the same one. No cookie needed, and it works at layer 4.
The catch is real: when many clients sit behind one address (a big office, a mobile carrier, a corporate proxy), they all share an IP, so they all hash to the same server, and the load goes lopsided.
Hashing by content: URL hash
Instead of the client, hash the request: the URL. Now the same page always goes to the same server. This is the trick behind cache tiers. If /cats.jpg always lands on server 2, then server 2 keeps it cached and serves it warm every time, and you never store the same object on all three servers.
Hash the URL to pick a server, so a given page always lands on the same one. Perfect for caches: each item lives on one server, so it stays warm.
The weakness is the mirror image of the last one: a single very popular URL becomes a hotspot on one server.
The problem: what happens when a server joins or leaves
Both of those used plain hashing. Here is what that plainly means, with real numbers. Say you have 3 servers, numbered 0, 1, and 2, and you want to place a photo. Run the photo's name through a hash (a scrambler that always turns the same name into the same number): say cat.jpg comes out as 27. Then take the remainder after dividing by the number of servers.
server = hash(key) mod N
So cat.jpg is 27 mod 3, which is 0: it lives on server 0. Easy.
Now add a fourth server, so you divide by 4 instead of 3. Now cat.jpg is 27 mod 4, which is 3: a different server. And it is not just cat.jpg. Change the divisor from 3 to 4 and almost every photo lands on a new remainder, so almost every photo moves. Adding one server reshuffles nearly the whole library. For a cache that means it all goes cold at once; for sessions, almost everyone is logged out.
The reason it hurts so much: every photo's server depends on the total number of servers. Change the total, and nearly every answer changes. That is the problem the next idea fixes, and it is one of the prettiest in system design.
Consistent hashing: the ring
The fix is to stop dividing by the number of servers at all, and put everything on a circle instead.
Picture a clock face. First, drop your servers onto it: server A at 12, server B at 4, server C at 8. Then drop each photo onto the same clock, wherever its hash lands: say cat.jpg at 2 and dog.jpg at 6.
The rule is one line: a photo belongs to the first server you reach walking forward around the clock. So cat.jpg at 2 walks forward and reaches server B at 4, and B holds it. dog.jpg at 6 walks forward and reaches server C at 8.
Here is the shift that makes it work. A photo's server no longer depends on how many servers there are. It depends only on which server is the next one ahead of it, which is a local thing.
So watch what happens when a new server D joins at position 3. cat.jpg at 2 now reaches D at 3 before it ever gets to B, so it moves from B to D. But dog.jpg at 6 still reaches C at 8, unchanged, and so does every photo past position 4. Only the few photos sitting just behind the new server move. Removing a server is the mirror image: only its photos walk forward to the next server, and no one else is touched.
How few is a few? About the number of photos divided by the number of servers. With 5 servers and 10 photos, each server holds around 2, so adding or removing one disturbs only about 2 of the 10, and the other 8 do not budge. That is the whole trick, and it is why the name is consistent: the mapping stays mostly the same as servers come and go. Watch a server leave and rejoin below, and count how few move.
Start: three servers on the ring. Each key belongs to the next server going clockwise.
One detail you can skip on a first read: to keep the slices even, each real server is actually dropped at many points around the circle (called virtual nodes), so no single server ends up owning a giant arc by luck.
Maglev: one fast table
Consistent hashing works, but notice that to find a key's server you still have to walk the ring from the key until you reach the next server. Google's Maglev does the very same job without any walking: it works out every answer ahead of time and writes them into one lookup table.
Think of a call center. You always dial the same extension number, and a board on the wall maps each extension to an agent: extension 7 rings agent B, extension 8 rings agent C. You dial, the board connects you in a single glance, and it is filled in so that every agent gets an even share of the extensions. That instant lookup (no walking) and the perfectly even split are the whole appeal, which is why Maglev sits in the very fastest balancers, the ones moving millions of connections a second.
There is a catch, though, and it is the honest difference from the ring. Two things are in play: your extension number, which never changes, and the board, which decides where that number rings. When an agent leaves, the board is re-printed. Your number is the same, but it may now ring a different agent, and the departed agent's callers all have to be moved somewhere, because the one who was handling them is gone. Keeping the board even can shift a few other extensions too. So a server change is not without impact here, and Maglev in fact reshuffles a little more than the ring does: it trades a bit of the ring's stability for that even table and instant lookup. That is the deal, and it is why a cache (which hates losing its contents) leans on the ring, while a hyperscale front door (which lives on speed and evenness and rarely changes servers) leans on Maglev.
Google's Maglev builds a single fixed lookup table instead of a ring: near-perfect even distribution and O(1) lookups, ideal for very fast layer 4 balancers. The table is rebuilt only when servers join or leave.
Sticky sessions: the cookie
Hashing infers stickiness from the client or the URL. The most direct way is to just remember the decision. On your first request the load balancer picks a server and hands back a cookie naming it. Every later request carries that cookie, the balancer reads it, and sends you straight back to the same server, where your session lives. It is the barber giving you a loyalty card with their name on it.
First request, no cookie yet. The load balancer picks a server (server 2) and sends back a cookie that names it.
It works, but it is a bit of an anti-pattern at scale: load goes uneven as sessions pile up on lucky servers, and you cannot cleanly retire a server while sessions are pinned to it. The cleaner fix is to stop keeping session state on the server at all (put it in a shared store), so any server can serve anyone, and you do not need stickiness for sessions in the first place.
What actually gets used
With every rule now on the table, across both parts, it helps to be honest about what real systems mostly reach for, because the working list is shorter than it looks.
For plain spreading, the defaults are boring on purpose: round-robin, or least-connections when requests vary in cost, or power-of-two-choices inside modern service meshes. Load-aware enough, and with no stickiness to reason about.
Stickiness shows up only when it earns its keep, and mostly in two places. Caches lean on consistent hashing: a tier of cache servers, or a distributed cache like memcached, maps each key to a node with a ring, so nodes can come and go without dumping everything. When you hear "consistent hashing" in production, it is almost always a cache or a data store deciding which node owns a key. The very largest front doors, the fleets of layer 4 balancers greeting millions of connections a second, lean on Maglev, because its table is fast, even, and deterministic: every balancer in the fleet builds the exact same table and routes the same connection the same way, with no coordination between them. That last property is the real reason Google built it, and why similar table-based schemes run at other large providers.
Session stickiness with a cookie is the odd one out: used, but treated as a smell. The modern habit is to stop keeping session state on the server at all, put it in a shared store like Redis, so any server can serve anyone and you never pin a user to a machine.
One newer twist worth knowing, since it ties back to the AI work: consistent hashing with a load cap (so a single hot key cannot swamp one node) is starting to appear in inference serving, where you want the same prompt to keep landing on the server that already has its work cached, without overloading that server. Same idea as everything here, tuned for a new workload.
The short version: spread with round-robin or least-connections, reach for consistent hashing when a cache or store needs stable ownership, use Maglev at the network layer at massive scale, and prefer stateless servers over sticky sessions.
Where this goes
That is the sticky half of load balancing: hash by client (source-IP) or by content (URL), survive servers coming and going with consistent hashing or Maglev, and pin sessions with a cookie when you must. With both halves covered, spreading and sticking, the load balancer is fully in hand. Next we make the whole system faster in a different way: caching and CDNs, keeping answers close so you do not do the same work twice.
