How WhatsApp scales, from one server to a billion users

What breaks

Every phone holds a connection open all day so messages can arrive instantly, and one server can only keep so many of those alive at once.

A messenger is not request-and-reply: each phone keeps a socket open continuously so a message can be pushed the moment it arrives. The resource that runs out is not CPU per request but the number of simultaneous connections one machine can hold — its memory and file descriptors. Long before the users are many, one server hits its connection ceiling.

The fix

Run many connection servers and spread the open sockets across them, behind a layer-4 load balancer built for long-lived connections rather than short web requests. Each server holds its share — WhatsApp famously tuned single machines to millions of connections each on Erlang, but the shape is the same: shard the connections across a fleet.

What breaks

To deliver a message the server has to look up which connection server the recipient is on — and every message hitting the database for that routing is saturating it.

Once connections are spread across a fleet, sending a message means finding which server holds the recipient's socket. Doing that lookup in the database on every single message turns the router into the bottleneck — routing is a per-message read, and messages are the whole product.

The fix

Keep the presence map — which user is on which connection server — in a fast in-memory store that every server can read, rather than the database. WhatsApp did this in Mnesia, Erlang's in-memory database. Routing a message becomes a memory lookup, and the database is left for durable account data.

What breaks

Half the recipients are offline, so their messages have to be stored until their phone reconnects — and one database holding every undelivered message is the wall.

A message to someone whose phone is off cannot just be dropped: it has to wait, durably, until they reconnect, then be delivered in order and cleared. Funnelling every undelivered message — for hundreds of millions of users — through one database makes it the bottleneck, and the access pattern (write once, read once soon after, delete) is not what a relational primary is best at.

The fix

Give each user a durable queue of undelivered messages in a horizontally scaled store, keyed by user, that spreads the writes across many partitions instead of one primary. When the phone reconnects, its queue is drained in order and cleared. The write-once, read-once, delete pattern fits a wide-column store far better than a relational database.

What breaks

One region is holding connections for the whole planet — the connection fleet is stretched, and a phone on another continent pays a round-trip across the ocean on every message.

Messaging is real-time, so distance is felt directly: a message routed through a region on the far side of the world adds hundreds of milliseconds each way. A single region is both a capacity wall for the sheer number of connections and a latency wall for the people far from it.

The fix

Run the connection fleet in multiple regions and pin each phone to the nearest, so its socket and its messages are handled close to home. A cross-region layer carries messages between people in different regions, but the common case — two people near each other — never leaves the region.