Conditional GET and ETags for Go JSON APIs Explained
Make JSON API revalidation cheap in Go: ETags from a revision column, collection tags, exact If-None-Match matching, 304 headers and 10x less work.
Most conditional GET tutorials end at the same point: a middleware hashes the response body, uses the hash as the ETag, and compares it with If-None-Match. It works, it saves bandwidth, and it leaves most of the possible savings on the table, because the server has already done everything it was trying to avoid: queried the database, built the structs and encoded the JSON. The hash is only available once the work is finished.
Tested with Go 1.25.6 and modernc.org/sqlite v1.50.0, macOS on Apple Silicon (arm64)
An API can do better, because it knows something a generic middleware does not: when a row last changed. This article builds ETags from a revision number, so that a 304 Not Modified is decided before any of the expensive work, and measures the difference against the hash approach on single items and a 500-item list. The generic hash middleware and its compression trap are covered in ETags and If-None-Match. The write-side counterpart, If-Match for optimistic locking, is in CRUD with ETags, and the headers around it are in cache headers. Everything ran on Go 1.25.6 with modernc.org/sqlite v1.50.0.
The exchange, briefly
The server sends ETag: "r7" with a response, a validator for that version of the resource. A client that stores the response sends If-None-Match: "r7" on its next request. If the tag still describes the current version, the server answers 304 Not Modified with no body, and the client reuses what it has. Otherwise it sends the new representation with the new tag.
For this to save work on the server, the question "has it changed?" must be cheaper than the question "what is it?".
A store with a revision
package main
import (
"database/sql"
"fmt"
"sync/atomic"
_ "modernc.org/sqlite"
)
type Item struct {
ID int64 `json:"id"`
Name string `json:"name"`
Notes string `json:"notes"`
Rev int64 `json:"rev"`
}
type Store struct{ db *sql.DB }
var dbs atomic.Int64
// Open creates n items. Every write takes the next value from one global counter, rev, so a row's rev changes
// whenever the row does, and MAX(rev) over a table changes whenever any row is inserted or updated.
func Open(n int) *Store {
db, err := sql.Open("sqlite", fmt.Sprintf("file:et%d?mode=memory&cache=shared", dbs.Add(1)))
if err != nil {
panic(err)
}
db.SetMaxOpenConns(1)
for _, q := range []string{
`CREATE TABLE items (id INTEGER PRIMARY KEY, name TEXT NOT NULL, notes TEXT NOT NULL, rev INTEGER NOT NULL)`,
`CREATE TABLE counter (rev INTEGER NOT NULL)`,
`INSERT INTO counter VALUES (0)`,
} {
if _, err := db.Exec(q); err != nil {
panic(err)
}
}
s := &Store{db}
for i := 1; i <= n; i++ {
s.Put(int64(i), fmt.Sprintf("item %d", i), "some longer free text so the body has a realistic size")
}
return s
}
func (s *Store) nextRev() int64 {
var r int64
s.db.QueryRow(`UPDATE counter SET rev = rev + 1 RETURNING rev`).Scan(&r)
return r
}
func (s *Store) Put(id int64, name, notes string) {
s.db.Exec(`INSERT INTO items (id, name, notes, rev) VALUES (?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET name = excluded.name, notes = excluded.notes, rev = excluded.rev`, id, name, notes, s.nextRev())
}
func (s *Store) Delete(id int64) { s.db.Exec(`DELETE FROM items WHERE id = ?`, id) }
func (s *Store) Get(id int64) (Item, bool) {
var it Item
err := s.db.QueryRow(`SELECT id, name, notes, rev FROM items WHERE id = ?`, id).Scan(&it.ID, &it.Name, &it.Notes, &it.Rev)
return it, err == nil
}
func (s *Store) List() []Item {
rows, _ := s.db.Query(`SELECT id, name, notes, rev FROM items ORDER BY id`)
defer rows.Close()
out := []Item{}
for rows.Next() {
var it Item
rows.Scan(&it.ID, &it.Name, &it.Notes, &it.Rev)
out = append(out, it)
}
return out
}
// ItemRev is the cheap question: one indexed lookup that returns a number, no body.
func (s *Store) ItemRev(id int64) (int64, bool) {
var r int64
err := s.db.QueryRow(`SELECT rev FROM items WHERE id = ?`, id).Scan(&r)
return r, err == nil
}
// ListState is the cheap question for a collection: how many rows and the newest revision among them.
func (s *Store) ListState() (n int, maxRev int64) {
s.db.QueryRow(`SELECT COUNT(*), COALESCE(MAX(rev), 0) FROM items`).Scan(&n, &maxRev)
return
}Every write takes the next value of a single counter table and stores it in the row's rev column. Two properties follow. A row's rev changes whenever the row changes. And MAX(rev) over the table changes whenever any row is inserted or updated. ItemRev and ListState are the cheap questions: one indexed lookup returning a number, and one aggregate returning a count and the newest revision.
You do not need a global counter if the table already has a trustworthy updated_at or a per-row version column, as in the CRUD article. A timestamp with second resolution is a poor choice, since two updates in the same second give the same tag. Use a version integer or a timestamp with enough resolution and a tiebreak.
Three handlers
package main
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"net/http"
"strconv"
"strings"
)
// Matches implements the If-None-Match comparison for GET: a list of tags, "*", and the weak prefix, which is
// ignored because GET uses weak comparison.
func Matches(header, etag string) bool {
if header == "" {
return false
}
for _, t := range strings.Split(header, ",") {
t = strings.TrimPrefix(strings.TrimSpace(t), "W/")
if t == "*" || t == strings.TrimPrefix(etag, "W/") {
return true
}
}
return false
}
func tag(s string) string { return `"` + s + `"` }
func writeJSON(w http.ResponseWriter, v any, etag string) {
w.Header().Set("Content-Type", "application/json")
if etag != "" {
w.Header().Set("ETag", etag)
}
// private: it depends on who asks. no-cache: a cache may store it but must revalidate before every use.
w.Header().Set("Cache-Control", "private, no-cache")
w.Header().Add("Vary", "Authorization")
json.NewEncoder(w).Encode(v)
}
func notModified(w http.ResponseWriter, etag string) {
w.Header().Set("ETag", etag)
w.Header().Set("Cache-Control", "private, no-cache")
w.Header().Add("Vary", "Authorization")
w.WriteHeader(http.StatusNotModified)
}
// Three ways to serve the same two resources, so they can be compared.
// Plain: no validators.
func Plain(s *Store) *http.ServeMux {
mux := http.NewServeMux()
mux.HandleFunc("GET /v1/items/{id}", func(w http.ResponseWriter, r *http.Request) {
id, _ := strconv.ParseInt(r.PathValue("id"), 10, 64)
it, ok := s.Get(id)
if !ok {
http.NotFound(w, r)
return
}
writeJSON(w, it, "")
})
mux.HandleFunc("GET /v1/items", func(w http.ResponseWriter, r *http.Request) { writeJSON(w, map[string]any{"data": s.List()}, "") })
return mux
}
// Hashed: do all the work, hash the bytes, and skip only the transfer.
func Hashed(s *Store) *http.ServeMux {
respond := func(w http.ResponseWriter, r *http.Request, v any) {
body, _ := json.Marshal(v)
sum := sha256.Sum256(body)
etag := tag(hex.EncodeToString(sum[:8]))
if Matches(r.Header.Get("If-None-Match"), etag) {
notModified(w, etag)
return
}
w.Header().Set("Content-Type", "application/json")
w.Header().Set("ETag", etag)
w.Header().Set("Cache-Control", "private, no-cache")
w.Header().Add("Vary", "Authorization")
w.Write(body)
}
mux := http.NewServeMux()
mux.HandleFunc("GET /v1/items/{id}", func(w http.ResponseWriter, r *http.Request) {
id, _ := strconv.ParseInt(r.PathValue("id"), 10, 64)
it, ok := s.Get(id)
if !ok {
http.NotFound(w, r)
return
}
respond(w, r, it)
})
mux.HandleFunc("GET /v1/items", func(w http.ResponseWriter, r *http.Request) { respond(w, r, map[string]any{"data": s.List()}) })
return mux
}
// Versioned: derive the tag from a revision number, and answer 304 before loading or encoding anything.
func Versioned(s *Store) *http.ServeMux {
mux := http.NewServeMux()
mux.HandleFunc("GET /v1/items/{id}", func(w http.ResponseWriter, r *http.Request) {
id, _ := strconv.ParseInt(r.PathValue("id"), 10, 64)
rev, ok := s.ItemRev(id)
if !ok {
http.NotFound(w, r)
return
}
etag := tag(fmt.Sprintf("r%d", rev))
if Matches(r.Header.Get("If-None-Match"), etag) {
notModified(w, etag)
return
}
it, ok := s.Get(id)
if !ok {
http.NotFound(w, r)
return
}
writeJSON(w, it, tag(fmt.Sprintf("r%d", it.Rev))) // the revision of the row we actually read
})
mux.HandleFunc("GET /v1/items", func(w http.ResponseWriter, r *http.Request) {
n, maxRev := s.ListState()
etag := tag(fmt.Sprintf("n%d-r%d", n, maxRev))
if Matches(r.Header.Get("If-None-Match"), etag) {
notModified(w, etag)
return
}
writeJSON(w, map[string]any{"data": s.List()}, etag)
})
return mux
}Plain sends no validators. It is the baseline.
Hashed marshals the response, hashes the bytes with SHA-256, takes the first eight bytes as the tag, and compares. Everything that precedes the hash has already happened.
Versioned asks the store for the revision first. For an item it builds the tag "r7" from the row's revision; if the request's If-None-Match matches, it answers 304 immediately. Only on a mismatch does it load the row and encode it. For the list the tag is "n500-r501", the row count and the newest revision.
Details that matter in the code:
Matches implements the comparison. If-None-Match is a comma-separated list, can be *, and may contain weak tags with a W/ prefix. For a GET the specification uses weak comparison, so W/"r7" matches "r7". A tag that is not quoted does not match. A hand-written == between the header and the tag is the most common bug here, and it fails the moment a proxy adds W/ or a client sends two tags.
The ETag on a 200 comes from the row actually read. The versioned handler computes a tag to compare, then loads the row and uses the row's own rev for the response. If the row changed between the two queries, the response carries the newer tag instead of an older one that would have labelled newer data. For a hard guarantee, do the revision check and the load in one transaction.
A 304 repeats the validators. notModified sets ETag and Cache-Control (and Vary) on the 304, as the HTTP specification requires: they update the stored response's metadata.
Cache-Control: private, no-cache is the right default for authenticated JSON. private keeps shared caches from storing a response that depends on the caller. no-cache does not mean "do not store"; it means "store, but revalidate before every reuse", which is exactly the behavior ETags exist for. Vary: Authorization tells caches that different credentials get different responses.
What came back
package main
import (
"fmt"
"net/http"
"net/http/httptest"
"strings"
)
func do(h http.Handler, path string, inm string) *httptest.ResponseRecorder {
req := httptest.NewRequest("GET", path, nil)
if inm != "" {
req.Header.Set("If-None-Match", inm)
}
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
return rec
}
func main() {
s := Open(500)
h := Versioned(s)
fmt.Println("== a conditional GET on one item")
first := do(h, "/v1/items/7", "")
etag := first.Header().Get("ETag")
fmt.Printf("first request: %d ETag=%s %d bytes\n", first.Code, etag, first.Body.Len())
again := do(h, "/v1/items/7", etag)
fmt.Printf("revalidate with the tag: %d ETag=%s %d bytes Cache-Control=%q\n", again.Code, again.Header().Get("ETag"), again.Body.Len(), again.Header().Get("Cache-Control"))
fmt.Println("\n== the comparison rules")
for _, c := range []struct{ label, header string }{
{"exact match", etag}, {"weak form of the same tag", "W/" + etag}, {"a list containing it", `"x", ` + etag + `, "y"`},
{"wildcard", "*"}, {"a different tag", `"r1"`}, {"unquoted", strings.Trim(etag, `"`)},
} {
fmt.Printf("If-None-Match: %-34s -> %d\n", c.header, do(h, "/v1/items/7", c.header).Code)
}
fmt.Println("\n== after an edit the old tag stops matching")
s.Put(7, "item 7", "edited")
changed := do(h, "/v1/items/7", etag)
fmt.Printf("after an edit: %d ETag=%s %d bytes\n", changed.Code, changed.Header().Get("ETag"), changed.Body.Len())
fmt.Println("\n== the collection tag moves on insert, update and delete")
list := do(h, "/v1/items", "")
t0 := list.Header().Get("ETag")
fmt.Printf("500 items: %s (%d bytes)\n", t0, list.Body.Len())
s.Put(501, "new", "x")
t1 := do(h, "/v1/items", "").Header().Get("ETag")
s.Put(3, "item 3", "changed")
t2 := do(h, "/v1/items", "").Header().Get("ETag")
s.Delete(2)
t3 := do(h, "/v1/items", "").Header().Get("ETag")
fmt.Println("after insert: ", t1)
fmt.Println("after update: ", t2)
fmt.Println("after delete of a middle row:", t3)
fmt.Println("all four distinct:", t0 != t1 && t1 != t2 && t2 != t3 && t0 != t3)
}== a conditional GET on one item
first request: 200 ETag="r7" 98 bytes
revalidate with the tag: 304 ETag="r7" 0 bytes Cache-Control="private, no-cache"
== the comparison rules
If-None-Match: "r7" -> 304
If-None-Match: W/"r7" -> 304
If-None-Match: "x", "r7", "y" -> 304
If-None-Match: * -> 304
If-None-Match: "r1" -> 200
If-None-Match: r7 -> 200
== after an edit the old tag stops matching
after an edit: 200 ETag="r501" 52 bytes
== the collection tag moves on insert, update and delete
500 items: "n500-r501" (51641 bytes)
after insert: "n501-r502"
after update: "n501-r503"
after delete of a middle row: "n500-r503"
all four distinct: trueThe first request returned the item with ETag="r7". Revalidating with the tag returned a 304 with zero bytes, and the ETag and Cache-Control headers repeated. The comparison table shows the rules: the exact tag, its weak form, a list containing it and * all produced a 304; a different tag produced a 200; the unquoted r7 produced a 200, since it is not a valid tag.
After an edit, the item's tag moved to "r501", and the old tag no longer matched: the client got the new body, 52 bytes, and the new tag.
The collection tag started at "n500-r501" for 500 items, 51,641 bytes. After an insert it became "n501-r502"; after an update "n501-r503" (the count unchanged, the revision moved); and after deleting a row from the middle "n500-r503". The delete case is why the count is in the tag: the newest revision did not change when a row other than the newest was removed, and MAX(rev) alone would have missed it. All four tags are different, as the last line confirms.
Tests
package main
import (
"net/http"
"testing"
)
func TestMatchesFollowsTheComparisonRules(t *testing.T) {
cases := []struct {
header, etag string
want bool
}{
{`"a"`, `"a"`, true},
{`W/"a"`, `"a"`, true},
{`"a"`, `W/"a"`, true},
{`"x", "a" ,"y"`, `"a"`, true},
{`*`, `"a"`, true},
{`"b"`, `"a"`, false},
{`a`, `"a"`, false},
{``, `"a"`, false},
}
for _, c := range cases {
if got := Matches(c.header, c.etag); got != c.want {
t.Errorf("Matches(%q, %q) = %v, want %v", c.header, c.etag, got, c.want)
}
}
}
func TestA304CarriesTheValidatorsAndNoBody(t *testing.T) {
h := Versioned(Open(3))
etag := do(h, "/v1/items/2", "").Header().Get("ETag")
rec := do(h, "/v1/items/2", etag)
if rec.Code != http.StatusNotModified || rec.Body.Len() != 0 || rec.Header().Get("ETag") != etag || rec.Header().Get("Cache-Control") == "" {
t.Fatalf("%d %q %v", rec.Code, rec.Body.String(), rec.Header())
}
}
func TestEveryStrategyChangesItsTagWhenTheDataChanges(t *testing.T) {
for name, build := range map[string]func(*Store) *http.ServeMux{"hashed": Hashed, "versioned": Versioned} {
s := Open(10)
h := build(s)
before := do(h, "/v1/items/4", "").Header().Get("ETag")
s.Put(4, "item 4", "different")
after := do(h, "/v1/items/4", before)
if after.Code != 200 || after.Header().Get("ETag") == before {
t.Errorf("%s: an edit was not noticed (%d)", name, after.Code)
}
}
}
func TestMissingItemsAreNot304(t *testing.T) {
h := Versioned(Open(2))
if rec := do(h, "/v1/items/99", "*"); rec.Code != 404 {
t.Fatalf("got %d", rec.Code)
}
}=== RUN TestMatchesFollowsTheComparisonRules
--- PASS: TestMatchesFollowsTheComparisonRules (0.00s)
=== RUN TestA304CarriesTheValidatorsAndNoBody
--- PASS: TestA304CarriesTheValidatorsAndNoBody (0.01s)
=== RUN TestEveryStrategyChangesItsTagWhenTheDataChanges
--- PASS: TestEveryStrategyChangesItsTagWhenTheDataChanges (0.03s)
=== RUN TestMissingItemsAreNot304
--- PASS: TestMissingItemsAreNot304 (0.00s)
PASS
ok example.test/lab 1.412sThe tests pin the comparison cases in a table, the shape of a 304 (no body, validators repeated), that both strategies notice an edit, and that a missing item is a 404 even when the client sent If-None-Match: *. The wildcard means "any current representation", and an item that does not exist has none.
What it saves
package main
import (
"net/http"
"testing"
)
func bench(b *testing.B, build func(*Store) *http.ServeMux, path string, revalidate bool) {
s := Open(500)
h := build(s)
inm := ""
if revalidate {
inm = do(h, path, "").Header().Get("ETag")
}
b.ReportAllocs()
b.ResetTimer()
for b.Loop() {
do(h, path, inm)
}
}
func BenchmarkItemPlain(b *testing.B) { bench(b, Plain, "/v1/items/7", false) }
func BenchmarkItemHashed304(b *testing.B) { bench(b, Hashed, "/v1/items/7", true) }
func BenchmarkItemVersioned304(b *testing.B) { bench(b, Versioned, "/v1/items/7", true) }
func BenchmarkListPlain(b *testing.B) { bench(b, Plain, "/v1/items", false) }
func BenchmarkListHashed304(b *testing.B) { bench(b, Hashed, "/v1/items", true) }
func BenchmarkListVersioned304(b *testing.B) { bench(b, Versioned, "/v1/items", true) }goos: darwin
goarch: arm64
pkg: example.test/lab
cpu: Apple M1 Max
BenchmarkItemPlain-10 129218 8539 ns/op 7069 B/op 45 allocs/op
BenchmarkItemHashed304-10 138430 8931 ns/op 7477 B/op 49 allocs/op
BenchmarkItemVersioned304-10 170125 6905 ns/op 6971 B/op 38 allocs/op
BenchmarkListPlain-10 3259 378402 ns/op 259482 B/op 4541 allocs/op
BenchmarkListHashed304-10 2953 401505 ns/op 260045 B/op 4545 allocs/op
BenchmarkListVersioned304-10 31162 38546 ns/op 7036 B/op 42 allocs/op
PASS
ok example.test/lab 7.526sOn an Apple M1 Max, with a request that carries a matching If-None-Match and a 500-row table:
| Case | Plain 200 | Hashed 304 | Versioned 304 |
|---|---|---|---|
| One item | 8.5 µs | 8.9 µs | 6.9 µs |
| 500-item list | 378 µs | 401 µs | 38.5 µs |
For a single row, nothing much changes: the lookup is cheap whichever way you do it, and the hash adds a little. For the list the numbers separate. The hashed 304 cost 401 µs, slightly more than the plain 200 at 378 µs, because it did the same work and then hashed 51 KB. The versioned 304 cost 38.5 µs, about ten times less, with 42 allocations in place of 4,545. Both save the transfer of 51 KB to the client; only the versioned one saves the server.
Whether this matters depends on the traffic. A dashboard that polls a list every few seconds from many clients, most of whom see no change, is the ideal case. An endpoint called once per user is not worth the machinery.
Choosing and cautions
Use the hash approach when you cannot say when data changed: composed responses, responses assembled from several services. It is generic, and it saves bandwidth.
Use revisions when one resource maps to rows you control. Add the revision to the table once and it serves ETags, If-Match and cache keys alike (the Redis cache uses the same number as its fence).
A tag describes a representation, not just the data. If the response depends on the Accept-Language header, on a field filter in the query string, or on the caller's permissions, the tag must include or vary on them, or you will serve one user's 304 to another. Vary handles the headers; query parameters are part of the URL, which caches already key on.
Strong or weak. A strong tag promises byte-identical content. The revision tags here are strong for a given encoder. If a compression layer or a proxy rewrites bodies, expect them to weaken it, and do not rely on byte-range semantics.
Do not treat 304 as authorization. The handler above checks that the item exists, and in a real service it must also check that the caller may see it before answering 304, or the tag becomes an oracle for resources the caller cannot read.
Pagination interacts. A tag on a cursor page reflects the page, so a new item at the top changes the first page's tag and not later ones. That is correct, and it is why a collection tag on a large paged list is better applied to the first page only (cursor pagination).
The semantics are defined in RFC 9110, section 13, and the cache-control directives in RFC 9111.
