Cache HTTP Responses in Go Handlers With Middleware

Some handlers are expensive and answer the same question over and over: a rendered article page, a leaderboard, a search that most people run with identical terms. Caching the response in front of the handler can turn a 60 ms database round trip into a map lookup. It is also an easy way to serve user A's page to user B, so the interesting part of this middleware is not the map but the rules about what goes in and what the key contains.

This article builds one, runs it against concurrent traffic, and lists the rules I would not skip. It builds on cache headers and ETags, and belongs to the web cluster that ends in a production HTTP server. It ran on Go 1.25.6 with golang.org/x/sync v0.19.0 for singleflight.

What is cached, and under what key

A cache entry is a snapshot of a response: the status code, the headers worth keeping and the body bytes. To capture it, the middleware runs the handler against a recording ResponseWriter and stores what it wrote. Everything after that is lookup.

The key decides correctness. Using only the path gives you the classic bug where ?page=2 returns page one. The key needs the method (only GET and HEAD are cacheable here), the full URL including the query string with its parameters in a stable order, and the value of every request header the response varies on. If the handler's output depends on Accept-Language, or if a compression layer upstream changes the body per Accept-Encoding, those values belong in the key. A good habit is to make the handler declare its dependencies with a Vary response header, and let the cache read that to build keys, because then the cache stays correct when the handler changes. The simplified version below varies on a fixed list you pass in.

The middleware

package main

import (
	"bytes"
	"fmt"
	"net/http"
	"net/http/httptest"
	"net/url"
	"sort"
	"strings"
	"sync"
	"sync/atomic"
	"time"

	"golang.org/x/sync/singleflight"
)

type entry struct {
	status  int
	header  http.Header
	body    []byte
	expires time.Time
}

type Cache struct {
	mu      sync.RWMutex
	entries map[string]*entry
	ttl     time.Duration
	vary    []string
	group   singleflight.Group
	now     func() time.Time
}

func NewCache(ttl time.Duration, vary ...string) *Cache {
	return &Cache{entries: map[string]*entry{}, ttl: ttl, vary: vary, now: time.Now}
}

func (c *Cache) key(r *http.Request) string {
	q := r.URL.Query()
	names := make([]string, 0, len(q))
	for k := range q {
		names = append(names, k)
	}
	sort.Strings(names) // ?a=1&b=2 and ?b=2&a=1 are the same request
	var sb strings.Builder
	sb.WriteString(r.URL.Path)
	for _, k := range names {
		vals := q[k]
		sort.Strings(vals)
		sb.WriteString("|" + k + "=" + url.QueryEscape(strings.Join(vals, ",")))
	}
	for _, h := range c.vary {
		sb.WriteString("|" + h + ":" + r.Header.Get(h))
	}
	return sb.String()
}

// cacheable decides, before running the handler, whether this request may be served from or stored in a shared cache.
func cacheable(r *http.Request) bool {
	return r.Method == http.MethodGet &&
		r.Header.Get("Authorization") == "" &&
		r.Header.Get("Cookie") == "" &&
		!strings.Contains(r.Header.Get("Cache-Control"), "no-cache")
}

type recorder struct {
	header http.Header
	status int
	body   bytes.Buffer
}

func (r *recorder) Header() http.Header         { return r.header }
func (r *recorder) WriteHeader(code int)        { r.status = code }
func (r *recorder) Write(p []byte) (int, error) { return r.body.Write(p) }

func (c *Cache) Middleware(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		if !cacheable(r) {
			w.Header().Set("Cache-Status", "bypass")
			next.ServeHTTP(w, r)
			return
		}
		key := c.key(r)
		c.mu.RLock()
		e := c.entries[key]
		c.mu.RUnlock()

		state := "hit"
		if e == nil || !c.now().Before(e.expires) {
			state = "miss"
			// Concurrent misses for one key share a single run of the handler.
			v, _, shared := c.group.Do(key, func() (any, error) {
				rec := &recorder{header: http.Header{}, status: http.StatusOK}
				next.ServeHTTP(rec, r)
				fresh := &entry{status: rec.status, header: rec.header, body: rec.body.Bytes(), expires: c.now().Add(c.ttl)}
				if fresh.status == http.StatusOK && fresh.header.Get("Set-Cookie") == "" &&
					!strings.Contains(fresh.header.Get("Cache-Control"), "private") {
					c.mu.Lock()
					c.entries[key] = fresh
					c.mu.Unlock()
				}
				return fresh, nil
			})
			e = v.(*entry)
			if shared {
				state = "coalesced"
			}
		}
		for k, v := range e.header {
			w.Header()[k] = v
		}
		w.Header().Set("Cache-Status", state)
		w.WriteHeader(e.status)
		w.Write(e.body)
	})
}

func main() {
	var calls atomic.Int64
	slow := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		calls.Add(1)
		time.Sleep(50 * time.Millisecond) // the expensive part
		if r.URL.Path == "/private" {
			w.Header().Set("Cache-Control", "private")
		}
		fmt.Fprintf(w, "page=%s lang=%s", r.URL.Query().Get("page"), r.Header.Get("Accept-Language"))
	})
	clock := time.Now()
	cache := NewCache(time.Minute, "Accept-Language")
	cache.now = func() time.Time { return clock }
	srv := httptest.NewServer(cache.Middleware(slow))
	defer srv.Close()

	get := func(path string, hdr ...string) string {
		req, _ := http.NewRequest("GET", srv.URL+path, nil)
		for i := 0; i+1 < len(hdr); i += 2 {
			req.Header.Set(hdr[i], hdr[i+1])
		}
		res, err := http.DefaultClient.Do(req)
		if err != nil {
			panic(err)
		}
		res.Body.Close()
		return res.Header.Get("Cache-Status")
	}

	fmt.Println("cold:", get("/a?page=1"), "| again:", get("/a?page=1"), "| reordered query:", get("/a?x=1&page=1"), get("/a?page=1&x=1"))
	fmt.Println("other page:", get("/a?page=2"), "| other language:", get("/a?page=1", "Accept-Language", "de"))
	fmt.Println("with Authorization:", get("/a?page=1", "Authorization", "Bearer t"), "| private response, twice:", get("/private"), get("/private"))
	fmt.Println("handler calls so far:", calls.Load())

	clock = clock.Add(2 * time.Minute)
	fmt.Println("after the TTL:", get("/a?page=1"))

	// 20 simultaneous requests for a key nobody has asked for yet.
	before := calls.Load()
	var wg sync.WaitGroup
	states := make([]string, 20)
	for i := range states {
		wg.Add(1)
		go func() { defer wg.Done(); states[i] = get("/hot") }()
	}
	wg.Wait()
	counts := map[string]int{}
	for _, s := range states {
		counts[s]++
	}
	fmt.Println("20 concurrent cold requests ->", counts, "handler runs:", calls.Load()-before)
}

cold: miss | again: hit | reordered query: miss hit
other page: miss | other language: miss
with Authorization: bypass | private response, twice: miss miss
handler calls so far: 7
after the TTL: miss
20 concurrent cold requests -> map[coalesced:20] handler runs: 1

Walk through the results as a checklist of the rules. The second identical request is a hit. The first of ?x=1&page=1 and ?page=1&x=1 was a miss, because that URL had never been seen, and its reordered twin was a hit: query parameters are sorted when the key is built, so both spellings share one entry. A different page and a different Accept-Language are separate entries, because the key has both. A request with an Authorization header bypassed the cache altogether. The /private response, which marks itself private, was a miss both times: the handler ran twice, because the middleware refused to store it. The fake clock lets the test jump past the TTL without sleeping, which is why now is a field and not a direct call to time.Now; the same trick is covered in testing with a fake clock.

The stampede guard

The last line of output is the reason singleflight is in the import list. When a popular entry is missing, whether it was never cached or it just expired, every request that arrives before the first one finishes also misses. With a 50 ms handler and a thousand requests per second, that is fifty copies of the expensive work, launched at the moment the system is already doing it. singleflight.Group.Do runs the function once per key and hands the result to every caller waiting on it. The 20 concurrent requests ran the handler once. singleflight reports a result as shared to every caller when more than one was waiting, so all twenty show coalesced, including the one whose request actually ran the function.

Two cautions about it. The shared function runs under the first caller's request, so if that client disconnects and its context is cancelled, the work is cancelled for everyone waiting; in a real handler, detach the work from the request context (context.WithoutCancel) or run it with its own timeout. And an error is shared too: if the one run fails, all waiters get the failure, which is usually right, though you should not cache the error. See singleflight for duplicate work for the pattern in general.

Why a short TTL beats clever invalidation

The hardest problem in caching is knowing when the data changed. A response cache in front of handlers cannot see your writes, so there are two honest strategies. The first is a short TTL, 5 to 60 seconds, and acceptance of that much staleness. For a public article page or a leaderboard this is nearly always fine, and it needs no coordination. The second is explicit invalidation: when a write happens, delete the keys it affects, which requires you to know which keys those are. Query strings make that hard (a change to one article affects its page, the list pages, the feeds and any search that matched it). I start with the TTL, measure the hit rate, and only add invalidation for the specific pages where users notice staleness, such as the page an editor has just saved. The caching rules in RFC 9111 describe the same trade-off for shared caches in general.

What must never be cached

I treat these as hard rules and write tests for them:

• Responses to requests with credentials. Authorization, session cookies and API keys mean the answer may be specific to a user. Bypass, as cacheable does. If your pages are public even when a cookie is present (an analytics cookie, say), choose the key components deliberately, rather than ignoring the cookie by default.
• Responses that set cookies. A cached Set-Cookie hands the same session to every visitor.
• Non-200 responses. Caching a 500 for a minute prolongs an outage. Caching a 404 is sometimes right, but only on purpose and with a shorter TTL.
• Anything that marks itself private or no-store. The handler knows its data better than the middleware does.

Limits of an in-memory cache

The map above has no size cap, so a crawler requesting millions of distinct URLs fills memory. Bound it: an LRU with a maximum entry count or byte size (the hashicorp/golang-lru package is the usual choice), and cap the body size you are willing to store. Each replica has its own cache, so a deploy empties it and N replicas mean N cold starts; for a larger setup move the entries to Redis, as in caching API responses with Redis, or let a CDN do it with proper cache headers. And remember the cheapest cache of all is HTTP's own: if the response carries Cache-Control: public, max-age=60, the browser and any CDN will do this for you without a line of Go.