Fix GraphQL N Plus One Queries in Go With Dataloaders

GraphQL makes one thing easy and one thing quietly expensive. The easy part is that a client can ask for books with their authors in a single request. The expensive part is that your server resolves the author field once for every book on the list. With twenty books that is twenty separate database lookups for authors, and with two hundred it is two hundred. This is the N plus one problem: one query for the list, and N more for something attached to each item.

REST has it too, whenever a handler loops and queries inside the loop. GraphQL makes it systematic, because the engine calls your field resolver once per object and the resolver has no idea that its neighbours are about to ask the same question. The standard remedy is a dataloader: collect the keys that are requested close together, run one query for all of them, and give each caller its own result. This article builds a generic one in Go, shows the effect with numbers, and covers the rules that decide whether a loader is correct. It continues the gqlgen server from the previous article, and the batching idea shows up again in singleflight. It ran on Go 1.25.6.

What goes wrong without a loader

A resolver for Book.author looks harmless:

func (r *bookResolver) Author(ctx context.Context, b *Book) (*Author, error) {
    return r.db.AuthorByID(ctx, b.AuthorID) // one query per book
}

Query { books { title author { name } } } returns twenty books, so this function runs twenty times, each doing a SELECT ... WHERE id = $1. If the authors table is small and the database is on the same host, you may not notice at first. Add network latency of a millisecond and a nested level, and the page that was fast in development takes a second in production.

The fix is not to have the parent query join the authors. That would make the books resolver fetch authors even when the client did not ask for them, which defeats the point of GraphQL. The loader keeps the work lazy: nothing is fetched unless a field is selected, and when it is, one query serves every object in the response.

How a dataloader works

The mechanism is the same everywhere:

1. A resolver calls loader.Load(key) and blocks.
2. The loader appends the key to the current batch and, if this is the first key, starts a short timer.
3. When the timer fires, or the batch reaches its size limit, the loader calls your batch function once with all the keys.
4. The batch function returns values by key. Each blocked Load receives its own.

It works because the engine runs sibling resolvers in separate goroutines, so twenty Load calls arrive within microseconds of each other. A wait of one or two milliseconds is long enough to catch them and short enough to be invisible.

A generic loader

Here is a version in about sixty lines, using generics so one type serves authors, users, anything keyed by a comparable type:

package main

import (
	"sync"
	"time"
)

// Loader batches Load calls made within a short window into one call to fetch.
type Loader[K comparable, V any] struct {
	fetch func(keys []K) (map[K]V, error)
	wait  time.Duration
	max   int

	mu    sync.Mutex
	batch *batch[K, V]
}

type batch[K comparable, V any] struct {
	keys []K
	once sync.Once
	done chan struct{}
	vals map[K]V
	err  error
}

func NewLoader[K comparable, V any](wait time.Duration, max int, fetch func([]K) (map[K]V, error)) *Loader[K, V] {
	return &Loader[K, V]{fetch: fetch, wait: wait, max: max}
}

func (l *Loader[K, V]) Load(key K) (V, error) {
	l.mu.Lock()
	b := l.batch
	if b == nil {
		b = &batch[K, V]{done: make(chan struct{})}
		l.batch = b
		time.AfterFunc(l.wait, func() { l.flush(b) })
	}
	b.keys = append(b.keys, key)
	if len(b.keys) >= l.max {
		l.batch = nil // a full batch is closed to new keys at once; the next Load starts a fresh one
		go l.flush(b)
	}
	l.mu.Unlock()

	<-b.done
	return b.vals[key], b.err
}

// flush runs at most once per batch, whichever of the timer and the size limit gets there first.
func (l *Loader[K, V]) flush(b *batch[K, V]) {
	b.once.Do(func() {
		l.mu.Lock()
		if l.batch == b {
			l.batch = nil
		}
		keys := unique(append([]K(nil), b.keys...))
		l.mu.Unlock()

		b.vals, b.err = l.fetch(keys)
		close(b.done)
	})
}

func unique[K comparable](in []K) []K {
	seen := make(map[K]bool, len(in))
	out := in[:0]
	for _, k := range in {
		if !seen[k] {
			seen[k] = true
			out = append(out, k)
		}
	}
	return out
}

Three decisions in it are worth a look.

The wait window. time.AfterFunc starts the clock when the first key of a batch arrives. Too long adds latency to every request; too short splits one logical batch into several queries. One to five milliseconds is typical.

The size cap. When the batch reaches max, it is closed at once and flushed without waiting for the timer. This bounds the length of the IN (...) list. A thousand-element list is legal in PostgreSQL and a poor plan in several others.

Deduplication. Twenty books by five authors ask for five distinct ids. The loader hands the fetch function unique keys, and every caller still gets an answer through the map.

Measuring it

The program uses a fake database that counts queries, twenty books with five distinct authors, and resolves every author concurrently the way an engine would:

package main

import (
	"fmt"
	"sync"
	"sync/atomic"
	"time"
)

type Author struct{ ID, Name string }
type Book struct{ ID, Title, AuthorID string }

// DB counts every query so the cost of each approach is visible.
type DB struct {
	queries atomic.Int64
	authors map[string]Author
}

func (d *DB) AuthorByID(id string) Author {
	d.queries.Add(1)
	return d.authors[id]
}

func (d *DB) AuthorsByIDs(ids []string) map[string]Author {
	d.queries.Add(1) // SELECT ... WHERE id IN (...)
	out := make(map[string]Author, len(ids))
	for _, id := range ids {
		out[id] = d.authors[id]
	}
	return out
}

func main() {
	db := &DB{authors: map[string]Author{}}
	var books []Book
	for i := 0; i < 20; i++ {
		aid := fmt.Sprintf("a%d", i%5) // 20 books, 5 distinct authors
		db.authors[aid] = Author{ID: aid, Name: "Author " + aid}
		books = append(books, Book{ID: fmt.Sprintf("b%d", i), Title: fmt.Sprintf("Book %d", i), AuthorID: aid})
	}

	// resolveAll mimics what a GraphQL engine does: it resolves the author field of every book concurrently.
	resolveAll := func(author func(Book) Author) []string {
		names := make([]string, len(books))
		var wg sync.WaitGroup
		for i, b := range books {
			wg.Add(1)
			go func() {
				defer wg.Done()
				names[i] = author(b).Name
			}()
		}
		wg.Wait()
		return names
	}

	names := resolveAll(func(b Book) Author { return db.AuthorByID(b.AuthorID) })
	fmt.Printf("naive:   %d queries for %d books, first names %v\n", db.queries.Load(), len(books), names[:3])

	db.queries.Store(0)
	loader := NewLoader(2*time.Millisecond, 100, func(ids []string) (map[string]Author, error) {
		return db.AuthorsByIDs(ids), nil
	})
	names = resolveAll(func(b Book) Author { a, _ := loader.Load(b.AuthorID); return a })
	fmt.Printf("batched: %d queries for %d books, first names %v\n", db.queries.Load(), len(books), names[:3])

	db.queries.Store(0)
	small := NewLoader(time.Second, 5, func(ids []string) (map[string]Author, error) {
		return db.AuthorsByIDs(ids), nil
	})
	start := time.Now()
	names = resolveAll(func(b Book) Author { a, _ := small.Load(b.AuthorID); return a })
	fmt.Printf("max=5:   %d queries, finished before the 1s timer: %v\n", db.queries.Load(), time.Since(start) < 500*time.Millisecond)
	_ = names
}

naive:   20 queries for 20 books, first names [Author a0 Author a1 Author a2]
batched: 1 queries for 20 books, first names [Author a0 Author a1 Author a2]
max=5:   4 queries, finished before the 1s timer: true

The naive path issued 20 queries. The loader issued 1 and returned the same names in the same order. With the cap set to 5 it issued 4 queries, since 20 loads fit exactly four batches of five, and it finished well before the one second timer, proving the size limit and not the clock flushed them.

A bug the numbers exposed

My first loader flushed a full batch from a goroutine but left it installed as the current batch until that goroutine got the lock. Loads that arrived in between were appended to a batch already on its way to the database, and the keys added after the copy would never have been fetched. I did not see wrong names, because the lab dedupes and the stragglers happened to share ids, but the query count came out as two or three on different runs, which gave it away.

The fix is to close the batch under the same lock that decides it is full: set the current batch to nil, then flush it. Every Load after that starts a new one. The final code also wraps the flush in a sync.Once, because the timer and the size limit can both fire for the same batch. Run a loader under go run -race and with repeated runs before trusting it. The numbers above were stable across several runs and the race detector stayed quiet.

Rules that decide correctness

One loader per request. Create the loader in middleware, put it in the request context, and read it from resolvers. A loader shared between requests would batch keys from different users together and, worse, cache results across authorisation scopes. The cached author a signed-in admin could see must never reach another user's response.

Return values for every key, in a form the caller can match. The lab's fetch returns a map. With a slice-based API the common bug is returning results in the database's order instead of the order of the keys. Match by key, always.

Decide what a missing key means. A book whose author was deleted has no row. Return a zero value and an error, or a nil pointer, according to what the schema allows. The Book.author field in the earlier article is non-null, so a missing author would be an error that nulls the parent. The nullability of the field is part of the loader's contract.

Propagate cancellation. The batch function should use a context that outlives any single caller but still stops when the request ends. In practice, use the request's context for loaders created per request.

Error handling. A failing batch fails every caller in it, as the lab's err field does. If you want per-key errors, make the result type carry one.

One-to-many needs a different key. Author.books is not looked up by book id. The loader's key is the author id and its value is a slice of books, from SELECT ... WHERE author_id IN (...) grouped in Go. It is the same shape with a different fetch function.

Using a library

You do not have to write this yourself. github.com/vikstrous/dataloadgen is a generics-based loader with the same shape and works with gqlgen. The older graph-gophers/dataloader package is also widely used. Both offer batching windows, caches and per-key errors. Writing the sixty lines above is worthwhile once, because the failure modes in the previous section are easier to avoid when you have met them.

Where batching is not the answer

Sometimes the better fix is further up. If a query always needs authors, join them in the list query. If the data is read constantly and changes rarely, a cache in front of AuthorByID removes the repeated lookups without any batching. And if you find yourself building loaders for every relationship, check the shape of the schema: deeply nested selection that mirrors your database is the usual cause, and cost limits on query depth stop a client from asking for the worst case. The same N plus one pattern in plain SQL is dealt with in database access in Go, and the standard library's database/sql documentation is the reference for the IN list handling your batch function will need, and the dataloadgen README documents the library version of this loader.