Benchmarks in Go and How to Write Honest Ones You Trust

A benchmark answers a narrow question: how long does this piece of code take, and how much memory does it allocate? Go makes asking easy, because the testing package contains a benchmark runner, and making the answer meaningful is the part that takes care. Most published numbers that turn out to be wrong were wrong for one of four reasons, and all four fit in a single file.

This guide writes benchmarks on Go 1.25.6, on an Apple M1 Max. It covers the basic shape, what the output columns mean, the mistake that makes a benchmark report a physically impossible time, and the habits that make two runs comparable. The next article, on comparing runs with benchstat, picks up where this one stops, and the new loop style gets its own short note in b.Loop.

The smallest useful benchmark

package main

func main() {}

package main

import (
	"strconv"
	"strings"
	"testing"
)

var words = func() []string {
	w := make([]string, 200)
	for i := range w {
		w[i] = "item" + strconv.Itoa(i)
	}
	return w
}()

func concatPlus(ws []string) string {
	s := ""
	for _, w := range ws {
		s += w + ","
	}
	return s
}

func concatBuilder(ws []string) string {
	var b strings.Builder
	for _, w := range ws {
		b.WriteString(w)
		b.WriteByte(',')
	}
	return b.String()
}

func concatGrow(ws []string) string {
	var b strings.Builder
	b.Grow(len(ws) * 8)
	for _, w := range ws {
		b.WriteString(w)
		b.WriteByte(',')
	}
	return b.String()
}

func BenchmarkPlus(b *testing.B) {
	for b.Loop() {
		concatPlus(words)
	}
}

func BenchmarkBuilder(b *testing.B) {
	for b.Loop() {
		concatBuilder(words)
	}
}

func BenchmarkBuilderGrow(b *testing.B) {
	for b.Loop() {
		concatGrow(words)
	}
}

func double(x int) int { return x * 2 }

// A benchmark that measures nothing: the call is pure, inlined, and its result is unused.
func BenchmarkBroken(b *testing.B) {
	for range b.N {
		double(21)
	}
}

func BenchmarkFixed(b *testing.B) {
	for b.Loop() {
		double(21)
	}
}

A benchmark lives in a _test.go file, is named BenchmarkXxx and takes a *testing.B. Its body is a loop that runs the code under test. The testing package calls the function several times with growing iteration counts until the measurement is long enough to be stable, which is one second by default, and reports the average time per iteration.

Before Go 1.24 the loop was for i := 0; i < b.N; i++, where b.N is the iteration count the runner chose. Since Go 1.24 the idiomatic form is for b.Loop(). It does the same job and removes two traps, both discussed below. The three string joining functions here are a good subject: they do the same work in three ways, and the differences are large enough to see.

Running and reading the output

go test -run='^$' -bench=. -benchmem -count=1 . 2>&1 | grep -vE '^(PASS|ok)'

goos: darwin
goarch: arm64
pkg: example.test/lab
cpu: Apple M1 Max
BenchmarkPlus-10           	   54597	     21547 ns/op	  152025 B/op	     200 allocs/op
BenchmarkBuilder-10        	  673641	      1759 ns/op	    5368 B/op	      10 allocs/op
BenchmarkBuilderGrow-10    	 1000000	      1097 ns/op	    1792 B/op	       1 allocs/op
BenchmarkBroken-10         	1000000000	         0.3151 ns/op	       0 B/op	       0 allocs/op
BenchmarkFixed-10          	554464274	         2.191 ns/op	       0 B/op	       0 allocs/op

-run='^$' skips ordinary tests so only benchmarks run. -bench=. selects every benchmark; the argument is a regular expression, so -bench=Builder selects the two builders. -benchmem adds the memory columns. The header lines say where the numbers came from, which is the context any reader needs: operating system, architecture, package and CPU. Always keep them with the numbers.

Each result line has the benchmark name with a -10 suffix, which is GOMAXPROCS, then four columns.

The three joiners tell a story. += took 24,347 ns/op, allocated 152,025 bytes and made 200 allocations. Every += on a string allocates a new string and copies the old one, so the cost grows with the square of the length: the 200 pieces produced 200 allocations and a lot of copying. strings.Builder appends into a buffer that grows by doubling, so it made 10 allocations for the same work and ran in 1,711 ns/op, about fourteen times faster. Adding b.Grow with a size estimate removed the doubling: one allocation, 1,792 bytes, 1,032 ns/op.

Look at how the time column and the allocation column line up. That is typical for string and slice code in Go, and it is why -benchmem should be on by default. The allocation counts are exact and repeat run to run, while the time varies. When a change reduces allocations, the speed-up is real even if the time numbers wobble.

When the result is too good

The last two lines of the output are the same operation, calling a function that doubles an integer.

BenchmarkBroken reports 0.3192 ns/op with no allocations. A nanosecond is a few clock cycles on this chip, so a third of one cannot be the cost of a call, a multiply and a loop iteration. The compiler inlined double, saw that the result was unused and that the argument was a constant, and removed the work. What the benchmark measured was the empty loop. Reporting a time for code that did not run is the classic benchmarking mistake, and it is the reason an old rule said to assign every result to a package-level variable.

BenchmarkFixed uses b.Loop() and reports 2.257 ns/op. Go 1.24's b.Loop keeps the call and its arguments alive, so the compiler cannot remove the body, and the number now includes actual work. The two lines are the same code and differ by a factor of seven. Neither is a result you would publish, since a two nanosecond call is below what this method can measure, but the first is wrong and the second is merely coarse.

With the older loop form, the fix is to store the result where the compiler must keep it:

var sink int

func BenchmarkDouble(b *testing.B) {
	for range b.N {
		sink = double(21)
	}
}

Even this does not stop the compiler from computing double(21) at build time, because the argument is constant. A benchmark of an inlinable function needs a non-constant input as well, such as a value read from a package variable that the compiler cannot see through. b.Loop closes both holes at once, which is the main reason to prefer it.

Keep setup out of the measurement

A benchmark often needs data: a slice of words, a parsed config, a connection. Building it inside the timed loop adds its cost to every iteration. There are two rules.

1. Build the data outside the loop, in the function body before it or in a package variable. The lab's words slice is a package variable, built once.
2. Call b.ResetTimer() after expensive setup in the old style. With b.Loop, the timer starts on the first call to b.Loop, so setup before the loop is excluded without the call. The next article in the series on this, the b.Loop note, measures the difference.

Per iteration setup, such as copying an input that the code under test mutates, needs b.StopTimer() and b.StartTimer() around it, which are expensive enough to distort fast benchmarks, or a design that does not mutate. Prefer to benchmark code that reads its input.

Sub-benchmarks for sizes and variants

b.Run creates a named sub-benchmark and is how you measure scaling. A function that is fast at 10 elements and slow at 10,000 shows up as a table instead of a single point:

func BenchmarkBuilderSizes(b *testing.B) {
	for _, n := range []int{10, 100, 1000} {
		b.Run(fmt.Sprintf("n=%d", n), func(b *testing.B) {
			ws := makeWords(n)
			for b.Loop() {
				concatBuilder(ws)
			}
		})
	}
}

The output has one line per size, named BenchmarkBuilderSizes/n=100-10. Select a single one with -bench='BuilderSizes/n=100'. Report throughput for code that processes bytes with b.SetBytes(int64(len(input))), which adds an MB/s column, and custom metrics with b.ReportMetric, for a rate that matters to you such as items per second.

Noise and what a single number means

One run gives one sample. The machine has other processes, the CPU changes frequency with temperature, and the first seconds of a run may differ from the rest. The result of a single -count=1 run is an anecdote.

Reduce noise before you compare numbers.

• Close other programs. Plug in a laptop; battery mode throttles.
• Run several times with -count=10 and look at the spread, not the best run.
• Compare the old and new code on the same machine in the same session, ideally interleaved.
• Do not run benchmarks in parallel with other tests. -run='^$' keeps tests out; -cpu and -p control parallelism.
• Use benchstat to decide whether a difference is real, instead of looking at two numbers.

Laptop results transfer poorly to production hardware. The ratios often hold, such as the fourteen to one of += against Builder, while the absolute times do not. State the hardware with any number you share.

Benchmarks that run in parallel

b.RunParallel runs the body on several goroutines at once, which is how you measure code that is meant to be shared: a cache, a counter, a connection pool, a mutex protected map. The runner splits the iteration count among GOMAXPROCS goroutines, and the reported time is the wall clock per operation across all of them, so a perfectly scalable function reports a smaller time than the same function on one goroutine.

func BenchmarkCacheGet(b *testing.B) {
	c := newCache()
	b.RunParallel(func(pb *testing.PB) {
		for pb.Next() {
			c.Get("key")
		}
	})
}

The result to look for is how the number changes with -cpu=1,2,4,8. Flat or rising time as cores are added means contention: the goroutines are waiting for each other. That is a measurement a single threaded benchmark cannot make, and it is the usual way to decide between a mutex, a read-write mutex and a sharded design. Run it with the race detector in a separate pass, since -race changes the timings.

Profiling from a benchmark

A benchmark is also the easiest way to get a profile of one function, because it runs the code for long enough to sample. Add -cpuprofile=cpu.out or -memprofile=mem.out, then open the file with go tool pprof -http=:8080 cpu.out. The memory profile with -memprofile-rate=1 records every allocation, which is how you find where the 200 allocations of += come from. Profile the benchmark, change the code, and benchmark again, in that order. Optimising without the profile is guessing, and optimising without the second benchmark is hoping.

The profile adds overhead, so do not record timing results from a run that wrote one. Keep the two jobs separate: one pass for numbers with no profiler attached, and one for the profile.

Common mistakes, in one list

• A constant input. The compiler folds the work at build time. Read the input from a variable, or use b.Loop.
• Setup in the loop. Parsing the fixture on every iteration measures the parser plus your function.
• Measuring the first call. Caches, lazily initialised tables and the runtime's own warm-up all make the first iteration slower. The runner's growing counts mostly hide this, but a one-shot benchmark with -benchtime=1x shows it.
• Reusing state that the code mutates. Appending to a global slice makes iteration 10,000 slower than iteration 1. Reset or rebuild it each time outside the timed region, or design the benchmark so the state does not grow.
• Comparing across machines. The ratio carries over and the absolute number does not.
• Trusting one run. Use -count=10 and compare with benchstat.
• Benchmarking the standard library's speed. A function that mostly calls fmt.Sprintf measures fmt.Sprintf. Know what dominates the number before you try to improve the rest.
• Ignoring the allocation column. A change that makes a function 5% slower and removes all its allocations is often a win under load.

Choosing the run length

By default each benchmark runs for about a second, which is enough for code that takes microseconds. Use -benchtime=5s for a longer, steadier measurement, or -benchtime=100x to run exactly 100 iterations, which suits an operation that takes seconds, such as one HTTP request to a test server or one database query. For a slow operation the automatic scaling gives few iterations and a coarse average, and a fixed count with -count=10 produces a clearer picture. Do not set the time so short that you measure timer resolution: under about 10 milliseconds total, the result is dominated by start-up costs.

What a benchmark cannot show

A microbenchmark measures a function with warm caches, a predictable branch pattern and an idle machine. Production has none of these. Three consequences follow.

• A fast function in a benchmark can be slow in a request. Cache misses, GC pressure from other allocations and lock contention appear only under load. Benchmark the real operation too, end to end, with a load test.
• Allocations matter more than their count suggests. They cost time in the garbage collector, which a short benchmark underrepresents.
• Optimising what is not slow wastes time. Profile first. go test -bench=. -cpuprofile=cpu.out writes a profile you can open with go tool pprof, and it shows where the time goes before you decide what to benchmark.

Write a benchmark to settle a specific question: is the new version faster, how does this scale, did this change allocate. A benchmark with no question behind it produces numbers that nobody uses. The Go documentation covers the testing package benchmark API and the go test flags in the command reference; the Go blog's note on b.Loop explains why the loop changed.