Rate Limiting in Go With a Ticker and the Rate Package

Two different needs hide behind "rate limiting". A client calling someone else's API must not call it faster than the provider allows. A server must not let one caller consume capacity meant for everyone. Both are the same mechanism, a limit on events per unit of time, applied on either side of the connection. Go's answer is short: a ticker for the simplest case, and the rate package for everything else.

This article covers both, the semantics of the token bucket, the server-side pattern, and testing. It is related to semaphores, which limit concurrency rather than rate, and it leads to building a token bucket. Everything ran on Go 1.25.6 with golang.org/x/time v0.14.0.

The simplest limiter: a ticker

A time.Ticker sends a value at a regular interval. Waiting for it before each action paces the actions evenly:

package main

import (
	"fmt"
	"time"
)

func main() {
	ticker := time.NewTicker(20 * time.Millisecond) // 50 calls a second
	defer ticker.Stop()

	start := time.Now()
	for i := 1; i <= 5; i++ {
		<-ticker.C // wait for permission
		fmt.Println("call", i)
	}
	fmt.Println("five calls took about", time.Since(start).Round(20*time.Millisecond))
}

call 1
call 2
call 3
call 4
call 5
five calls took about 100ms

Five calls take about 100 ms, one every 20 ms. The ticker is the right tool for a background job that must not run faster than a fixed pace, and for crude client-side pacing. Its limits are real: it cannot burst (if you were idle for a second, you still get one call per tick afterwards), it does not queue (ticks that nobody receives are dropped), and it is global to whatever reads the channel. For anything more flexible, a token bucket.

The token bucket and golang.org/x/time/rate

A token bucket holds up to b tokens and refills at r tokens per second. Each event takes a token; with none available, the event is refused or must wait. The consequence is the behavior most services want: an idle caller accumulates tokens and can burst up to b calls, then is held to r per second. golang.org/x/time/rate implements it:

package main

import (
	"fmt"

	"golang.org/x/time/rate"
)

func main() {
	lim := rate.NewLimiter(10, 3) // 10 tokens a second, a bucket of 3

	for i := 1; i <= 6; i++ {
		fmt.Println("request", i, "allowed:", lim.Allow())
	}
}

request 1 allowed: true
request 2 allowed: true
request 3 allowed: true
request 4 allowed: false
request 5 allowed: false
request 6 allowed: false

The bucket starts full, so the first three requests are allowed at once (the burst), and the next three, arriving in the same instant, are refused because the tokens are spent. Waiting 100 ms would refill one. rate.NewLimiter(r, b) takes a rate (rate.Limit, events per second, or rate.Every(d) to say "one per d") and a burst size; a burst of 1 is a strict limiter, and rate.Inf means no limit.

Three ways to ask

The limiter offers three styles, for three situations:

Allow() answers immediately: yes, and a token is taken, or no. It suits servers that reject over-limit requests.

Wait(ctx) blocks until a token is available or the context ends, and returns an error if the context's deadline would pass before the wait could finish. It suits clients that must not exceed a provider's limit but may wait:

package main

import (
	"context"
	"fmt"
	"time"

	"golang.org/x/time/rate"
)

func main() {
	lim := rate.NewLimiter(rate.Every(20*time.Millisecond), 1) // one per 20ms, no burst

	start := time.Now()
	for i := 1; i <= 4; i++ {
		if err := lim.Wait(context.Background()); err != nil {
			fmt.Println("wait:", err)
			return
		}
		fmt.Println("call", i)
	}
	fmt.Println("four calls took about", time.Since(start).Round(20*time.Millisecond))

	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Millisecond)
	defer cancel()
	fmt.Println("with a 5ms deadline:", lim.Wait(ctx) != nil)
}

call 1
call 2
call 3
call 4
four calls took about 60ms
with a 5ms deadline: true

The first call is immediate (the bucket starts with its one token), and the next three each wait about 20 ms, so four calls take about 60 ms. The last line shows the useful property: Wait fails at once when the delay it would need exceeds the context's deadline, instead of sleeping and then failing, so a request that cannot be served in time is rejected early.

Reserve() returns a reservation with the delay required, without blocking. It lets you decide: wait, drop or schedule. Call Cancel on the reservation to give the token back if you decide not to proceed.

Server side: one limiter per client

A single global limiter protects the server's total capacity and lets one noisy client use all of it. The usual fix is a limiter per client, keyed by IP address, API key or user, with the limiters stored in a map. The map needs a lock, and it needs cleanup, because an unbounded map keyed by client is a memory leak that an attacker can feed:

package main

import (
	"net/http"
	"sync"
	"time"

	"golang.org/x/time/rate"
)

type entry struct {
	lim  *rate.Limiter
	seen time.Time
}

// PerClient hands out one limiter per key.
type PerClient struct {
	mu      sync.Mutex
	clients map[string]*entry
	rate    rate.Limit
	burst   int
}

func NewPerClient(r rate.Limit, burst int) *PerClient {
	return &PerClient{clients: map[string]*entry{}, rate: r, burst: burst}
}

func (p *PerClient) get(key string, now time.Time) *rate.Limiter {
	p.mu.Lock()
	defer p.mu.Unlock()
	e, ok := p.clients[key]
	if !ok {
		e = &entry{lim: rate.NewLimiter(p.rate, p.burst)}
		p.clients[key] = e
	}
	e.seen = now
	return e.lim
}

// Sweep removes limiters idle for longer than idle.
func (p *PerClient) Sweep(now time.Time, idle time.Duration) int {
	p.mu.Lock()
	defer p.mu.Unlock()
	removed := 0
	for k, e := range p.clients {
		if now.Sub(e.seen) > idle {
			delete(p.clients, k)
			removed++
		}
	}
	return removed
}

// Middleware rejects over-limit callers with 429 and a Retry-After header.
func (p *PerClient) Middleware(keyOf func(*http.Request) string, next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		lim := p.get(keyOf(r), time.Now())
		if !lim.Allow() {
			w.Header().Set("Retry-After", "1")
			http.Error(w, "rate limit exceeded", http.StatusTooManyRequests)
			return
		}
		next.ServeHTTP(w, r)
	})
}

PerClient creates a limiter on first sight of a key and remembers when it was last used; Sweep removes the idle ones, to be called from a ticker every minute or so. The middleware answers 429 with Retry-After when Allow says no. The key function is a design decision: the remote IP is easy and wrong behind a proxy (use the forwarded-for address your proxy sets, trusted only from the proxy), and an authenticated user or API key is better when you have one. A test shows it working with two clients:

package main

import (
	"net/http"
	"net/http/httptest"
	"testing"
	"time"
)

func TestEachClientHasItsOwnBucket(t *testing.T) {
	p := NewPerClient(1, 2) // one a second, burst of two
	h := p.Middleware(
		func(r *http.Request) string { return r.Header.Get("X-Client") },
		http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {}),
	)

	call := func(client string) int {
		req := httptest.NewRequest("GET", "/", nil)
		req.Header.Set("X-Client", client)
		rec := httptest.NewRecorder()
		h.ServeHTTP(rec, req)
		return rec.Code
	}

	for i := 0; i < 2; i++ {
		if code := call("a"); code != 200 {
			t.Fatalf("client a request %d = %d, want 200 (within burst)", i+1, code)
		}
	}
	if code := call("a"); code != 429 {
		t.Errorf("client a third request = %d, want 429", code)
	}
	if code := call("b"); code != 200 {
		t.Errorf("client b first request = %d, want 200: buckets must be independent", code)
	}
}

func TestSweepRemovesIdleClients(t *testing.T) {
	p := NewPerClient(1, 1)
	now := time.Now()
	p.get("old", now.Add(-time.Hour))
	p.get("fresh", now)

	if removed := p.Sweep(now, 10*time.Minute); removed != 1 {
		t.Errorf("Sweep removed %d, want 1", removed)
	}
}

=== RUN   TestEachClientHasItsOwnBucket
--- PASS: TestEachClientHasItsOwnBucket (0.00s)
=== RUN   TestSweepRemovesIdleClients
--- PASS: TestSweepRemovesIdleClients (0.00s)
PASS
ok  	example.test/lab	1.296s

The first test checks the three things that matter: the burst passes, the next request is refused, and a different client is unaffected. The second checks the cleanup, which is the part people forget.

Testing without sleeping

Rate limiter tests that call time.Sleep are slow and flaky. The rate package takes a time argument on its ...N methods (AllowN(t, n), ReserveN(t, n), WaitN with a context), so a test can supply its own clock and be exact:

package main

import (
	"testing"
	"time"

	"golang.org/x/time/rate"
)

func TestRefillOverTime(t *testing.T) {
	lim := rate.NewLimiter(1, 1) // one a second
	t0 := time.Unix(1_000_000, 0)

	if !lim.AllowN(t0, 1) {
		t.Fatal("the first token should be available")
	}
	if lim.AllowN(t0, 1) {
		t.Error("a second request at the same instant should be refused")
	}
	if lim.AllowN(t0.Add(500*time.Millisecond), 1) {
		t.Error("half a second is not enough to refill one token")
	}
	if !lim.AllowN(t0.Add(time.Second), 1) {
		t.Error("after a full second one token should be back")
	}
}

=== RUN   TestRefillOverTime
--- PASS: TestRefillOverTime (0.00s)
PASS
ok  	example.test/lab	1.252s

The test runs instantly and exactly, because the limiter's idea of "now" comes from the arguments. Prefer this over sleeping for anything about rates and refills.

Where to enforce a rate limit

A rate limit can live at several layers, and the layer determines what it can see. In your Go service, in-process, it knows your handlers and users, and costs nothing extra, but each instance enforces its own count. At a reverse proxy or API gateway, one limit covers every instance and sees raw traffic, but knows less about who the caller is. In a shared store such as Redis, a limit is global and exact, at the price of a network call per request. Many systems use two layers: a coarse, cheap limit at the edge to absorb abuse, and a finer in-process limit that knows the user or the plan. Whatever the layer, the response matters as much as the limit: a 429 with a Retry-After header tells a well-behaved client when to come back, and clients that retry immediately without backoff are why the header exists (retry with backoff).

Choosing and caveats

Use a ticker for fixed pacing of a loop. Use rate.Limiter for bursts, per-client limits and context-aware waiting. For a limit across several server instances (a global quota), an in-process limiter is not enough: the limiter must live in shared storage such as Redis, or each instance gets a share of the total. Remember that the rate limits events, not concurrency; slow requests can still pile up under a generous rate, so pair a rate limit with a concurrency bound (semaphores). And decide what over-limit means: reject (429 with Retry-After), delay, or degrade. The package is documented at pkg.go.dev/golang.org/x/time/rate, and the HTTP semantics of 429 and Retry-After are in RFC 6585.