System design interview question

Design a rate limiter

Every public API needs one: “design a rate limiter” asks you to cap how often each client can call you, on every single request, across a whole fleet of servers, without the check itself becoming the bottleneck. It tests algorithms, shared state and how a guard should fail, all in a small space. Here is how to walk into it, and then how to practise it until the answer is yours.

Practise API Rate Limiter

Multiple choicePick the design from four
Fill the gapsFill the gaps in one drawn for you
Build from scratchBuild it on a blank canvas

Practise it on a canvas

Free with an account.

The question, as an interviewer would set it

A public API is being abused: a few clients fire thousands of requests a second and starve everyone else. The team wants to cap each client to a fixed number of requests per time window and reject the rest immediately, before any real work is done. The check has to be cheap, because it runs on every single request, and it has to be consistent across a fleet of identical API servers behind a load balancer, so a client cannot slip past simply by being routed to a different server each time. The limits themselves change rarely.

What it is really testing

Every phrase that matters in the brief is a requirement in disguise. Spot them before you draw a single box.

“cap each client to a fixed number of requests per time window and reject the rest immediately”
A component that counts each client's requests and refuses the ones over the limit, inline.
“consistent across a fleet of identical API servers”
One shared store for the counters, so every server sees the same count.

Back of the envelope

Requests: 50,000 a second across the fleetAn assumption to say out loud. If the interviewer has a number in mind, they will give it to you.
Clients: 10 million × ~20 bytes of count each → ~200 MBSmall enough to hold in memory. The hard part is every server seeing the same number.
Count updates: One per request → ~50,000 writes a secondEvery request is a read-and-increment, so wherever the counts live must be very fast at tiny writes.
Budget per check: A millisecond or twoIt runs before every request, so it must cost almost nothing next to the work it guards.

Questions to ask first

Limit by what — user, API key or IP address?
One limit for everyone, or different tiers per plan?
Refuse outright, or slow the client down?
One server or many, in one region or several?
How exact must it be — is going slightly over acceptable?

The trade-offs you will be pushed on

1.Fixed window, sliding window or token bucket — what goes wrong at the edges of each?
2.Where do the counts live so every server sees the same number?
3.The store holding the counts goes down. Let requests through, or refuse them?
4.What should an over-limit client be told, and how does it know when to try again?

Now answer it

The same design always gets the same score, with the reason behind every requirement — then defend it against the trade-offs above.