Preallocate Slices and Maps in Go and Measure the Gain

If you know how many elements a slice will hold, saying so costs one argument and saves a surprising amount of work. A Go slice that grows through append reallocates and copies its backing array each time it fills, producing garbage the collector must later reclaim. A map that grows rehashes its entries. Both have a size hint that avoids it. This article measures how much, for slices from 10 to 100000 elements and maps of 10 and 1000, and shows the growth sequence so the numbers make sense.

It is part of reducing allocations, alongside escape analysis and string concatenation. Garbage from growth is exactly what an allocation profile shows. The cornerstone is the Go performance guide.

How growth works

append doubles the capacity of small slices and grows large ones more slowly. The lab prints the capacity after each reallocation for a slice filled one element at a time:

[4 8 16 32 64 128 256 512 848 1280 1792 2560 3408 5120]

Up to 256 elements the capacity doubles, and beyond that the growth factor falls towards 1.25 with rounding to the allocator's size classes (848, not 1024, after 512). Filling 5000 elements took 14 allocations, each copying everything so far. The copying is geometric, so the total work is about twice the final size, which is not the problem. The problem is the garbage: every superseded array is dead memory, and the sum of all the arrays is several times the final one. For 100000 ints the program allocated 4.1 MB to build a 0.8 MB slice.

The experiment

Four ways to fill a slice, two ways to fill a map, each at several sizes. append, no hint is the baseline.

package main

import (
	"fmt"
	"slices"
	"testing"
)

var sink []int
var msink map[int]int

func appendNoHint(n int) []int {
	var s []int
	for i := 0; i < n; i++ {
		s = append(s, i)
	}
	return s
}

func appendHinted(n int) []int {
	s := make([]int, 0, n)
	for i := 0; i < n; i++ {
		s = append(s, i)
	}
	return s
}

func indexAssign(n int) []int {
	s := make([]int, n)
	for i := 0; i < n; i++ {
		s[i] = i
	}
	return s
}

func slicesGrow(n int) []int {
	s := slices.Grow([]int(nil), n)
	for i := 0; i < n; i++ {
		s = append(s, i)
	}
	return s
}

func mapNoHint(n int) map[int]int {
	m := map[int]int{}
	for i := 0; i < n; i++ {
		m[i] = i
	}
	return m
}

func mapHinted(n int) map[int]int {
	m := make(map[int]int, n)
	for i := 0; i < n; i++ {
		m[i] = i
	}
	return m
}

func main() {
	fmt.Println("growth of an appended slice (capacity after each reallocation):")
	var s []int
	last := -1
	var caps []int
	for i := 0; i < 5000; i++ {
		s = append(s, i)
		if cap(s) != last {
			last = cap(s)
			caps = append(caps, last)
		}
	}
	fmt.Println(caps)
	fmt.Println()

	for _, n := range []int{10, 1000, 100000} {
		fmt.Printf("-- %d elements\n", n)
		bench(fmt.Sprintf("append, no hint"), func(b *testing.B) {
			for b.Loop() {
				sink = appendNoHint(n)
			}
		})
		bench("make with capacity, then append", func(b *testing.B) {
			for b.Loop() {
				sink = appendHinted(n)
			}
		})
		bench("make with length, then index", func(b *testing.B) {
			for b.Loop() {
				sink = indexAssign(n)
			}
		})
		bench("slices.Grow, then append", func(b *testing.B) {
			for b.Loop() {
				sink = slicesGrow(n)
			}
		})
		if n <= 1000 {
			bench("map, no hint", func(b *testing.B) {
				for b.Loop() {
					msink = mapNoHint(n)
				}
			})
			bench("map, make with size hint", func(b *testing.B) {
				for b.Loop() {
					msink = mapHinted(n)
				}
			})
		}
	}
}

growth of an appended slice (capacity after each reallocation):
[4 8 16 32 64 128 256 512 848 1280 1792 2560 3408 5120]

-- 10 elements
append, no hint                               105.3 ns/op       248 B/op      5 allocs/op
make with capacity, then append                22.8 ns/op        80 B/op      1 allocs/op
make with length, then index                   22.9 ns/op        80 B/op      1 allocs/op
slices.Grow, then append                       28.2 ns/op        80 B/op      1 allocs/op
map, no hint                                  283.4 ns/op       520 B/op      5 allocs/op
map, make with size hint                      175.6 ns/op       376 B/op      4 allocs/op
-- 1000 elements
append, no hint                              2514.9 ns/op     25208 B/op     12 allocs/op
make with capacity, then append              1157.0 ns/op      8192 B/op      1 allocs/op
make with length, then index                 1090.0 ns/op      8192 B/op      1 allocs/op
slices.Grow, then append                     1207.9 ns/op      8192 B/op      1 allocs/op
map, no hint                                35824.6 ns/op     74456 B/op     22 allocs/op
map, make with size hint                     9851.0 ns/op     36992 B/op      6 allocs/op
-- 100000 elements
append, no hint                            358930.5 ns/op   4101411 B/op     28 allocs/op
make with capacity, then append             75515.3 ns/op    802821 B/op      1 allocs/op
make with length, then index                64939.5 ns/op    802822 B/op      1 allocs/op
slices.Grow, then append                    79772.2 ns/op    802822 B/op      1 allocs/op

package main

import (
	"flag"
	"fmt"
	"sort"
	"testing"
)

func init() {
	testing.Init()
	flag.Set("test.benchtime", "200ms")
}

// bench runs f five times and prints the median, which damps the noise of a laptop.
func bench(name string, f func(b *testing.B)) {
	const runs = 5
	ns := make([]float64, 0, runs)
	bs := make([]int64, 0, runs)
	as := make([]int64, 0, runs)
	for i := 0; i < runs; i++ {
		r := testing.Benchmark(f)
		ns = append(ns, float64(r.T.Nanoseconds())/float64(r.N))
		bs = append(bs, r.AllocedBytesPerOp())
		as = append(as, r.AllocsPerOp())
	}
	sort.Float64s(ns)
	sort.Slice(bs, func(i, j int) bool { return bs[i] < bs[j] })
	sort.Slice(as, func(i, j int) bool { return as[i] < as[j] })
	fmt.Printf("%-38s %12.1f ns/op %9d B/op %6d allocs/op\n", name, ns[runs/2], bs[runs/2], as[runs/2])
}

Results

Slices

The gain is largest where the work is smallest. At 10 elements, a hint turned 5 allocations into 1 and the time from 105 ns to 23 ns, a 4.6 times speedup, because the unhinted version allocates arrays of 4 and 8 and 16 elements on the way to 10. At 1000 elements it is 2.2 times; at 100000, 4.8 times, with 5 times fewer bytes allocated. A hint is cheap everywhere and matters most for small slices created in hot paths, which is where an allocation per element of growth shows up as latency.

The three hinted methods are close. make with length and index assignment is a few percent faster than make with capacity and append, because it skips the append's length bookkeeping, and the difference (64.9 µs against 75.5 µs at 100000) is within what a laptop produces from run to run. slices.Grow is the convenient form when you start from an existing slice: it adds room for n more elements in one allocation, and costs a little extra for the call.

Maps

For a map of 1000 entries a hint made the fill 3.6 times faster and halved the memory. The map grows by splitting its tables as it fills, and each growth rehashes entries; with a hint, the runtime allocates enough space once. The benefit at 10 entries is smaller, 1.6 times, because a map of 10 entries has little growing to do in any case. Use a hint whenever you know the count, for example when building a lookup table from a slice: make(map[K]V, len(items)).

What the hint does not fix

• Wrong hints. A capacity far above the real size wastes memory, and one below it just behaves like no hint for the overflow. A fair estimate (the length of the input, or the upper bound of a query's limit) is enough.
• Lifetime. A preallocated slice kept in a long lived struct pins its whole backing array, including unused capacity. If you build a large slice with spare room and keep a small result, copy it to a right sized one (slices that hold arrays).
• Zeroing. make([]int, n) zeroes the memory, which for a huge slice you will overwrite anyway costs a pass. For large slices, make with capacity and append avoids pre-zeroing the part you will fill, though both zero on allocation for pointer containing types.
• Appending to a slice you did not make. The hint belongs where the slice is created. A helper that receives and returns a slice can use slices.Grow to add room once.

Guidance

This is a safe optimisation in the sense that it cannot change behaviour, only the allocation count. It is also one a code reviewer can apply in thirty seconds, which is why it is worth knowing the numbers: a missing hint in a function called a million times is a measurable cost, and one in a function called twice is not.

Checklist

• Give make a capacity or length whenever you know the size or a bound.
• Size maps with make(map[K]V, n) when filling them in a loop.
• Use slices.Grow to extend an existing slice in one step.
• Measure with -benchmem; the allocation count is the clearest signal.
• Do not pre-size a long lived slice larger than you need.

The growth rules are in the Go specification on appending to slices and the helpers in the slices package.