JustPaste.it

Designing a Rate Limiter: Algorithms to Compare in a System Design Interview

Rate limiter design is a popular mid-level system design question. Interviewers mainly want to hear you compare algorithms and handle the distributed case.

Clarify first

Per user, per IP, or per API key? Hard limit or soft? What happens when the limit is hit (HTTP 429 with a Retry-After header)?

Algorithms

  • Fixed window counter: simple, but allows bursts of 2x at window edges.
  • Sliding window log: exact, but stores a timestamp per request, so memory heavy.
  • Sliding window counter: weights the previous window's count; good accuracy with little memory.
  • Token bucket: tokens refill at a steady rate and allow controlled bursts. Most common in practice.
  • Leaky bucket: smooths output to a constant rate; good for protecting downstream services.

Distributed design

Store counters in Redis with atomic operations (INCR with EXPIRE, or a Lua script for token bucket) so all API servers share state. Discuss the latency cost, local caching with slight over-admission, and what happens if Redis is down (fail open vs fail closed).

Where to place it

API gateway or middleware, before expensive work. Return headers like X-RateLimit-Remaining so clients can back off.

Wrap-up

Summarize the tradeoffs you chose. For timed practice with follow-up questions, see https://www.linkjob.ai/ (Linkjob.ai interview prep).

Disclosure: I work on Linkjob AI.