Slices That Keep Huge Arrays Alive in Go Programs Silently

A slice in Go is three words: a pointer into an array, a length and a capacity. Slicing it is free because nothing is copied, and that is also the reason it can hold on to far more memory than it appears to. Take 32 bytes from the front of a 1 MB buffer and keep them, and you have kept the megabyte. The collector looks at what a pointer points into, not at how much of it you use. This article measures that for a sub-slice, a slice with a capped capacity and a clone, then does the same for slices of pointers that are shortened by popping or by dequeuing, which keep the removed values alive for the same reason.

It is one case of the broader list in Go memory leaks that are not goroutine leaks and belongs next to maps that never shrink and struct padding. To find a leak like this one in a service, use heap profiles. The cornerstone is the Go performance guide.

Part one: a small view of a big array

The program allocates a 1 MB block per iteration, keeps the first 32 bytes in a global slice, and lets the block go out of scope. After 200 iterations it forces two collections and reports how much the live heap grew.

package main

import (
	"fmt"
	"runtime"
	"slices"
)

func live() uint64 {
	runtime.GC()
	runtime.GC()
	var m runtime.MemStats
	runtime.ReadMemStats(&m)
	return m.HeapAlloc
}

type Big struct{ data [1 << 20]byte }

var kept [][]byte

// header returns the first 32 bytes of a 1 MB block.
func header(b []byte) []byte { return b[:32] }

func keepSubslice(n int) {
	kept = nil
	for i := 0; i < n; i++ {
		block := make([]byte, 1<<20)
		kept = append(kept, header(block))
	}
}

func keepClone(n int) {
	kept = nil
	for i := 0; i < n; i++ {
		block := make([]byte, 1<<20)
		kept = append(kept, slices.Clone(header(block)))
	}
}

func keepFullExpr(n int) {
	kept = nil
	for i := 0; i < n; i++ {
		block := make([]byte, 1<<20)
		h := block[:32:32] // caps the capacity but still points into the block
		kept = append(kept, h)
	}
}

// removing from a slice of pointers: a vacated slot past len still points at the removed value.
func removeAppend(n int) []*Big {
	s := make([]*Big, n)
	for i := range s {
		s[i] = new(Big)
	}
	for len(s) > 1 {
		s = append(s[:0], s[1:]...)
	}
	return s
}

func popReslice(n int) []*Big {
	s := make([]*Big, n)
	for i := range s {
		s[i] = new(Big)
	}
	for len(s) > 1 {
		s = s[:len(s)-1]
	}
	return s
}

func popClear(n int) []*Big {
	s := make([]*Big, n)
	for i := range s {
		s[i] = new(Big)
	}
	for len(s) > 1 {
		s[len(s)-1] = nil
		s = s[:len(s)-1]
	}
	return s
}

func popDelete(n int) []*Big {
	s := make([]*Big, n)
	for i := range s {
		s[i] = new(Big)
	}
	for len(s) > 1 {
		s = slices.Delete(s, len(s)-1, len(s))
	}
	return s
}

// a queue that pops from the front by reslicing.
func queueReslice(n int) []*Big {
	q := make([]*Big, 0, n)
	for i := 0; i < n; i++ {
		q = append(q, new(Big))
	}
	for len(q) > 1 {
		q = q[1:]
	}
	return q
}

func queueNil(n int) []*Big {
	q := make([]*Big, 0, n)
	for i := 0; i < n; i++ {
		q = append(q, new(Big))
	}
	for len(q) > 1 {
		q[0] = nil
		q = q[1:]
	}
	return q
}

func main() {
	const n = 200
	base := live()
	fmt.Printf("%d blocks of 1 MB per scenario, live heap above baseline after two collections\n\n", n)
	for _, c := range []struct {
		name string
		f    func(int)
	}{
		{"keep b[:32] (sub-slice)", keepSubslice},
		{"keep b[:32:32] (full slice expr)", keepFullExpr},
		{"keep slices.Clone(b[:32])", keepClone},
	} {
		c.f(n)
		fmt.Printf("%-38s %8d KB   (%d slices of cap %d)\n", c.name, (live()-base)>>10, len(kept), cap(kept[0]))
		kept = nil
		live()
	}
	fmt.Println()
	for _, c := range []struct {
		name string
		f    func(int) []*Big
	}{
		{"remove front with append(s[:0], s[1:]...)", removeAppend},
		{"pop from the end, s = s[:len(s)-1]", popReslice},
		{"pop from the end, clear the slot first", popClear},
		{"pop from the end, slices.Delete", popDelete},
		{"queue q = q[1:]", queueReslice},
		{"queue q[0] = nil; q = q[1:]", queueNil},
	} {
		s := c.f(50)
		fmt.Printf("%-42s %8d KB   (len %d)\n", c.name, (live()-base)>>10, len(s))
		runtime.KeepAlive(s)
	}
}

200 blocks of 1 MB per scenario, live heap above baseline after two collections

keep b[:32] (sub-slice)                  204813 KB   (200 slices of cap 1048576)
keep b[:32:32] (full slice expr)         204818 KB   (200 slices of cap 32)
keep slices.Clone(b[:32])                    31 KB   (200 slices of cap 32)

remove front with append(s[:0], s[1:]...)      1041 KB   (len 1)
pop from the end, s = s[:len(s)-1]            51217 KB   (len 1)
pop from the end, clear the slot first         1041 KB   (len 1)
pop from the end, slices.Delete                1041 KB   (len 1)
queue q = q[1:]                               51217 KB   (len 1)
queue q[0] = nil; q = q[1:]                    1041 KB   (len 1)

The sub-slice retained everything. 200 slices of 32 bytes held 200 blocks of 1 MB, 200 MB in total for 6.4 KB of data. The capacity column shows why: each kept slice still has a capacity of a full megabyte, which is the extent of the array it points into. cap(s) is a useful tell. A kept slice whose capacity is far above its length is usually pinning a larger array.

The full slice expression changed nothing. b[:32:32] sets the capacity to 32, and the growth was the same to within noise. It is a common misreading that it detaches the slice. It limits how far an append can write into the original array, which protects the neighbours from being overwritten, and the slice still points at the start of the same allocation.

Cloning is the fix. slices.Clone (or append([]byte(nil), b[:32]...), or bytes.Clone) allocates a new array of the length it copies. The block is then unreachable and the collector frees it. The retained heap was 31 KB.

Where it shows up in real code: a parser that returns tokens as sub-slices of its input buffer, a network read loop that keeps the header of a message after handing the buffer back, a cache keyed by a few bytes of a large request body, a bufio.Scanner.Bytes() result stored in a map. The Bytes() case is worse than the leak, because the scanner reuses its buffer, so the stored slice changes under you; the documentation tells you to copy.

Part two: pointers left behind a shorter slice

A shorter slice does not clear the slots it gave up. The array still holds the pointers that were in them, and since the array is still reachable through the slice (the length is smaller, but the array is the same), the collector sees those pointers and keeps their targets alive. With pointers to 1 MB structures the effect is large and easy to measure. Each scenario builds 50 *Big values and removes 49 of them.

Removing from the front by shifting was safe here, because the elements are moved down and the old tail slot ends up pointing at a value that is still in the slice. The leak is not the shifting itself. It is any operation that leaves a slot past len pointing at a value you meant to drop.

Popping by reslicing leaked 49 values. 49 slots past the end of the slice each held a pointer to a 1 MB structure, 49 MB plus the one still in use, and nothing in the program could reach them. The fix is one line: set the slot to nil before shrinking, or call slices.Delete, which since Go 1.22 zeroes the vacated slots for exactly this reason.

The queue leaked the same amount in the other direction. q = q[1:] moves the start of the slice forward, and the slots before it are no longer in the slice and are still in the array. The consumed items stay alive until the array is replaced, which for a queue that is refilled by append happens only on reallocation. A long running worker that dequeues this way can hold every item it ever processed between two growths. Clear the slot with q[0] = nil first, or use a ring buffer or a channel.

The same applies to slices.Compact and slices.DeleteFunc: since Go 1.22 they clear the tail, and code written with append tricks before that, or handwritten loops, does not. If you maintain code that predates it, look for s = s[:n] on slices of pointers, interfaces, maps, strings and slices; slices of plain integers and floats have nothing to keep alive.

Finding it

• Compare len and cap of the slices held in long lived structures. A large capacity on a small slice is the first clue.
• In a heap profile, inuse_space shows the allocation of the large block (the make or the read), while the retaining slice is somewhere else. Follow the type up to what holds it (heap profiles).
• Compare the live heap after collection with the size of the data the program claims to hold. Here the program claimed 6.4 KB and held 200 MB.
• Write the check as a test that forces two collections and compares HeapAlloc, as the lab does.

Rules that prevent it

1. Copy at the boundary where you decide to keep something. Not earlier, since copying everything wastes the cheapness of slices.
2. Use slices.Clone or bytes.Clone for small pieces of large buffers, and strings.Clone for strings.
3. Clear slots before shrinking a slice of pointers, or use slices.Delete, slices.DeleteFunc and slices.Compact, which do it.
4. Do not rely on s[:n:n] to detach. It limits writes, not retention.
5. Prefer a ring buffer, a linked structure or a channel for queues that live as long as the process.
6. Copy out of reused buffers (bufio.Scanner, net.Conn reads, sync.Pool buffers) before storing.

Checklist

• A slice keeps its whole backing array alive, whatever its length.
• cap(s) far above len(s) on something you store is a warning sign.
• A full slice expression does not detach a slice.
• Shrinking a slice of pointers leaves them behind until you nil them.
• slices.Clone and slices.Delete are the one line fixes.

The behaviour follows from the slice section of the language specification, and the clearing in Delete is described in the slices package documentation.