Consistent Hashing Explained Visually

hash(key) % N is simple and, for a fixed N, perfectly uniform if the hash function is uniform. The problem is what happens when N changes. Because the modulus itself changes, the target node for almost every key changes too — not just the keys that logically belong to the new or removed node.

For a cache, that means a scale-out event — the moment meant to relieve load — instead invalidates nearly the entire cache at once. Every client's next read misses, every miss falls through to the origin database, and the database that was already under pressure now takes the full read traffic it was being protected from. This is the mechanism behind cascading failures during "routine" capacity changes.

Consistent hashing exists to fix exactly this one property: when the node count changes by one, only the keys that node was responsible for should move. Everything else should stay put.

Consistent hashing maps both nodes and keys onto the same circular hash space — typically 0 to 2^32 − 1, wrapping back to 0. Each node gets a position on the ring by hashing its identity (cache-a, cache-b, …). Each key is hashed the same way, and it belongs to the first node found walking clockwise from the key's position.

Adding a node inserts one new point on the ring. Only the keys between that new point and the previous node clockwise — the arc it now owns — move to it; every other key's nearest clockwise node is unchanged. Removing a node deletes its point, and only the keys in its arc fall through to the next node clockwise. In both cases, the fraction of keys that move is close to 1/N, not (N-1)/N.

This is the entire mechanism: a sorted ring of node positions, a query that finds the first node at or after a key's hash, and the property that only a local arc is affected by any single node change.

One point per physical node sounds fair, but random hash positions are not evenly spaced — with only 4-5 points on a 32-bit circle, gaps vary widely by chance, so one node can end up owning a much larger arc than another purely by the luck of its hash. Worse, when that node fails, its entire arc falls to exactly one neighbor, which now absorbs a large, sudden spike instead of a spread-out one.

Virtual nodes fix both problems: each physical node is hashed to many points on the ring (commonly 100-200), not one. Key ownership is still "first point clockwise," but now each physical node's total share is the sum of many small, scattered arcs, which averages out close to 1/N of the ring regardless of hash luck on any single point. When a node fails, its many small arcs fall to many different neighbors instead of dumping everything on one.

More virtual nodes reduce load variance but increase ring size and lookup/membership bookkeeping — there is a real trade-off, not a free upgrade. Systems like Amazon Dynamo and Apache Cassandra default to somewhere in the 100-256 vnodes-per-node range because it drives per-node load variance down to a few percent without an unreasonable ring size.

Trigger. The cache cluster scales from 4 nodes to 5 to relieve cache-b, which has been running hot. The new node cache-e is added to the ring with only 1 virtual node, matching the (already too low) setting the other 4 nodes were configured with. Symptom: minutes after the scale-out, cache-b is still evicting aggressively and its miss rate has barely moved, while cache-e sits nearly idle.

Root mechanism. With one point per node, cache-e's single ring position happened to land inside an arc far from cache-b's share — it stole keys from cache-d instead. cache-b's arc was never touched, because consistent hashing only ever moves keys in the new node's arc, and that arc's location is entirely up to where its one hash point happens to fall. Low vnode count means high variance in exactly where relief lands, and there is no guarantee it lands where the hot spot is.

Evidence. A histogram of keys owned per node shows cache-b at 38% of total keys against an expected ~20%, and cache-e at 3%. Recovery: raise every node's virtual-node count (e.g., to 150) and rebuild the ring — this redistributes existing arcs immediately, without waiting for further scale events. Prevention: treat vnode count as a capacity-planning parameter with a measured variance target, monitor per-node key-count distribution continuously, and alert on skew — not just on absolute node memory.

Reach for consistent hashing when node membership changes at runtime and each remap has a real cost: distributed caches, sharded key-value stores, and request routers that need session affinity across a fleet that autoscales. The benefit is proportional to how often membership changes and how expensive a full remap is — a cluster that never resizes gets little from the extra mechanism.

If the cluster size is fixed and known ahead of time, plain modular hashing is simpler, faster to reason about, and has zero ring-maintenance cost — do not add consistent hashing for a problem that does not exist yet. If uneven load is a concern independent of resizing (a few very popular keys, not node count changes), consistent hashing alone will not fix it — that needs key-level mitigation such as splitting hot keys or adding a request-level cache in front of the ring.

For durability on top of placement, pair the ring with replication: store each key on the next R-1 distinct physical nodes walking clockwise from its primary, not just the first. That is what gives a ring fault tolerance, not just balanced placement — placement and replication are two different guarantees layered on the same ring.