Implement a Rate Limiter in Java
Problem
A rate limiter caps how many requests a client may make in a period of time; the token bucket algorithm keeps a bucket of tokens that refills at a steady rate and spends one token per request.
Build a rate limiter that allows a burst of up to 3 requests and then 1 request per second on average, and show which requests in a sequence are accepted.
Java Program
public class TokenBucketRateLimiter {
private final int capacity;
private final double refillPerMs;
private double tokens;
private long lastRefill;
TokenBucketRateLimiter(int capacity, double tokensPerSecond, long startMs) {
this.capacity = capacity;
this.refillPerMs = tokensPerSecond / 1000.0;
this.tokens = capacity; // start full, so an initial burst is allowed
this.lastRefill = startMs;
}
synchronized boolean tryAcquire(long nowMs) {
tokens = Math.min(capacity, tokens + (nowMs - lastRefill) * refillPerMs); // top up for elapsed time
lastRefill = nowMs;
if (tokens >= 1) {
tokens -= 1;
return true;
}
return false;
}
public static void main(String[] args) {
// 3 requests of burst, then 1 new request per second.
// Times are passed in explicitly, so the demo behaves the same on every run.
TokenBucketRateLimiter limiter = new TokenBucketRateLimiter(3, 1.0, 0);
long[] requestTimes = {0, 100, 200, 300, 1300, 1400, 3500};
for (long t : requestTimes) {
System.out.println("t=" + t + "ms -> " + (limiter.tryAcquire(t) ? "allowed" : "rejected"));
}
}
}Output
Core Logic
Instead of running a timer, the bucket is refilled lazily: each request first adds the tokens earned since the previous request, then tries to spend one.
- 1The bucket starts full with
capacitytokens, which allows an initial burst. - 2
tryAcquireaddselapsed × refillPerMstokens, capped at the capacity withMath.min. - 3If at least one token is available it's spent and the request is allowed; otherwise it's rejected.
- 4The method is
synchronizedso concurrent requests can't spend the same token twice.
Key Point: The current time is passed in rather than read from System.currentTimeMillis(), which makes the limiter easy to test and the demo print the same result on every run.
Why: Each request does a fixed amount of arithmetic and the limiter stores only two numbers.
Key Concepts
Approach 2: Fixed Window Counter
public class FixedWindowRateLimiter {
private final int limit;
private final long windowMs;
private long windowStart = -1;
private int count;
FixedWindowRateLimiter(int limit, long windowMs) {
this.limit = limit;
this.windowMs = windowMs;
}
synchronized boolean tryAcquire(long nowMs) {
long window = nowMs / windowMs * windowMs; // start of the window this request falls in
if (window != windowStart) {
windowStart = window; // new window: reset the counter
count = 0;
}
if (count < limit) {
count++;
return true;
}
return false;
}
public static void main(String[] args) {
FixedWindowRateLimiter limiter = new FixedWindowRateLimiter(3, 1000); // 3 per second
long[] requestTimes = {0, 100, 200, 300, 999, 1000, 1001};
for (long t : requestTimes) {
System.out.println("t=" + t + "ms -> " + (limiter.tryAcquire(t) ? "allowed" : "rejected"));
}
}
}
Output
Core Logic
The simplest limiter counts requests in fixed one-second windows and resets the count whenever a new window starts.
- 1
nowMs / windowMs * windowMsrounds the time down to the start of its window. - 2When a request falls into a new window, the counter resets to zero.
- 3Requests are allowed while the counter is below the limit.
Key Point: Fixed windows allow bursts at the boundary — up to twice the limit within a short span around the window change — which is the main reason token buckets or sliding windows are preferred.
Why: Each request updates one counter and one window start.
Key Concepts
Interview Questions
How is a sliding window different from a fixed window?
A sliding window counts requests over the last N seconds measured from now, avoiding the burst allowed at fixed-window boundaries, at the cost of storing timestamps or weighted counts.
How would you rate-limit across several servers?
Keep the counters in a shared store such as Redis, so every server sees the same token count for a client.