Strings in Go Are Bytes and That Changes Everything

Programmers arriving from Python, Java or JavaScript expect a string to be a list of characters, and most of what surprises them in Go follows from that expectation being wrong. A Go string is a sequence of bytes. It happens to hold UTF-8 text almost all the time, and the language and library work hard to make that pleasant, but underneath there is no character type. Once you hold the byte model in your head, len, indexing, range, slicing and conversion each stop being a special rule. They become one fact applied five ways. This is a foundation article in the Go basics path, and the next one, runes, bytes and Unicode, builds on it.

What a string actually is

At run time a string is a two word header: a pointer to the first byte and a length. You can see the size of that header, and the fact that it does not change with the text it describes:

package main

import (
	"fmt"
	"unicode/utf8"
	"unsafe"
)

func main() {
	short := "go"
	long := "héllo, 世界, this string is much longer than the other one"

	fmt.Println(unsafe.Sizeof(short), unsafe.Sizeof(long))

	s := "héllo, 世界"
	fmt.Println(len(s), utf8.RuneCountInString(s))

	for i := 0; i < len(s); i++ {
		fmt.Printf("%02x ", s[i])
	}
	fmt.Println()

	for i, r := range s {
		fmt.Printf("%d:%c ", i, r)
	}
	fmt.Println()
}

16 16
14 9
68 c3 a9 6c 6c 6f 2c 20 e4 b8 96 e7 95 8c 
0:h 1:é 3:l 4:l 5:o 6:, 7:  8:世 11:界

Both headers are 16 bytes on a 64-bit machine, however long the text is. The text héllo, 世界 is 9 characters and 14 bytes, and the two counts come from different functions: len counts bytes and utf8.RuneCountInString decodes the string to count code points. The hex dump shows why. The é is the two bytes c3 a9, and each of the two Chinese characters is three bytes. The last loop uses range, which does the decoding for you and reports the byte offset where each rune starts, so you see the index jump from 1 to 3 over the two-byte é.

The language specification defines a string as "a (possibly empty) sequence of bytes" and says that its length is the number of bytes. The standard reference for how this works in practice is the Go blog's Strings, bytes, runes and characters in Go, which I would read once even if you think you know it.

Indexing and range give different things

s[i] is always a byte (a uint8). It does not decode anything, and on a string with multi-byte characters it hands you the middle of a character without complaint. range is the decoding form: each iteration gives a rune (an int32) and the byte index it started at. If the bytes are not valid UTF-8, range yields the replacement character U+FFFD and advances one byte.

The practical rule I follow is this. Use indexing when you are parsing a format that is defined in bytes, such as an HTTP header, a delimiter search or a protocol with ASCII markers. Use range when the text is for people and each character counts. Never index to the "third character" of user input, because that is the third byte.

Strings are immutable

The bytes behind a string are read-only. The compiler enforces it:

package main

import "fmt"

func main() {
	s := "hello"
	s[0] = 'H'
	fmt.Println(s)
}

# example.test/lab
./main.go:7:2: cannot assign to s[0] (neither addressable nor a map index expression)

The error says you cannot assign to an index of a string, and that is the point. Immutability is what lets strings be shared freely between goroutines, used as map keys and sliced without copying. When you need to change text, you build a new string, or you convert to []byte, edit the copy and convert back. That conversion is not free, which is the next section.

What each conversion costs

Moving between string and []byte allocates and copies, because one is immutable and the other is not. The program below counts allocations with testing.AllocsPerRun, and compares a few common ways of building text:

package main

import (
	"fmt"
	"strings"
	"testing"
)

var sink string
var sinkBytes []byte

func main() {
	big := strings.Repeat("x", 4096)

	fmt.Println("string to bytes:", testing.AllocsPerRun(100, func() { sinkBytes = []byte(big) }))
	fmt.Println("bytes to string:", testing.AllocsPerRun(100, func() { sink = string(sinkBytes) }))
	fmt.Println("substring:", testing.AllocsPerRun(100, func() { sink = big[10:20] }))

	fmt.Println("concat with +=:", testing.AllocsPerRun(10, func() {
		s := ""
		for range 200 {
			s += "ab"
		}
		sink = s
	}))

	fmt.Println("strings.Builder:", testing.AllocsPerRun(10, func() {
		var b strings.Builder
		for range 200 {
			b.WriteString("ab")
		}
		sink = b.String()
	}))

	fmt.Println("Builder with Grow:", testing.AllocsPerRun(10, func() {
		var b strings.Builder
		b.Grow(400)
		for range 200 {
			b.WriteString("ab")
		}
		sink = b.String()
	}))
}

string to bytes: 1
bytes to string: 1
substring: 0
concat with +=: 199
strings.Builder: 7
Builder with Grow: 1

Each conversion between the two types was one allocation, and taking a substring was none. The += loop is the one to look at: building a 400 byte string from 200 appends took roughly one allocation per step, because every += creates a new string and copies the old contents into it. The Builder grows a buffer geometrically, so the same work needed a handful of allocations, and a Grow call that knows the final size brought it to one. For a loop that runs three times it does not matter. For one that runs per request over a few thousand items, it is the difference that shows up in a profile.

The compiler can sometimes avoid the copy. A conversion used only for a map lookup, a comparison or a range loop over []byte(s) is optimized in many cases, and the standard library has paths that avoid it too. Do not rely on that when you are designing an API. Take the type your caller already has, and provide both a string and a []byte form where the cost matters, as bytes and strings do for most functions.

Substrings share memory, and that can hurt

big[10:20] allocates nothing because it is a new header pointing into the old bytes. That is efficient and has a consequence that surprises people who store the result: the entire original string stays alive for as long as the substring does. If you read a 50 MB response body, keep a 12 byte token sliced out of it and discard the body, the 50 MB stays in memory.

You can see the sharing directly. unsafe.StringData returns a pointer to a string's first byte, so two strings that overlap have related pointers:

package main

import (
	"fmt"
	"strings"
	"unsafe"
)

func main() {
	big := "prefix:" + strings.Repeat("x", 1000)
	token := big[7:12]

	fmt.Println(unsafe.StringData(token) == unsafe.StringData(big[7:]))

	copied := strings.Clone(token)
	fmt.Println(unsafe.StringData(copied) == unsafe.StringData(big[7:]), copied == token)
}

true
false true

The substring points into big, and the clone, from strings.Clone (added in Go 1.18), is a fresh copy of just those five bytes that compares equal but shares nothing. When you keep a small piece of a large input around, clone it. This is one of the few memory leaks in Go that comes from the language's design rather than from a goroutine or a cache, and it is the reason strings.Clone exists.

Comparing, concatenating and building

Strings compare with ==, < and the rest, byte by byte, so "Z" < "a" is true and "ä" sorts after every ASCII letter. For human-facing sorting that respects a language's rules you need golang.org/x/text/collate, and for case-insensitive equality use strings.EqualFold, which handles Unicode folding, rather than lowercasing both sides.

For building, three rules cover most code. A single concatenation of a few parts, a + "/" + b, is fine and the compiler handles it well. A loop that appends should use a strings.Builder or collect parts into a slice and call strings.Join. And when you assemble output that goes straight to a writer, skip the intermediate string and call fmt.Fprintf on the writer.

A raw string literal, written with backticks, takes everything between them literally, including newlines and backslashes, and cannot contain a backtick. It is the right form for regular expressions, multi-line SQL and embedded JSON. An interpreted literal in double quotes processes escape sequences, so a newline or a Unicode code point can be written inside it. Both produce the same kind of string value, so choosing between them is a readability call, and the regex tester is a good place to check a pattern before it goes inside backticks.

Strings are not always valid UTF-8

Go does not validate a string when you create it. A string can hold any bytes, including ones that are not valid UTF-8, and that is what you get from a file, a network read or a legacy database column. utf8.ValidString tells you whether a string is well formed, and ranging over an invalid one produces U+FFFD for each bad byte. If you pass such a string to encoding/json the encoder replaces the bad bytes, and if you display it, you get the replacement character. The decision of what to do with invalid input, reject, repair or pass through, belongs at your system's boundary.

When the byte model is the wrong level

If your problem is about what a user sees as one character, the byte model is two levels too low. An emoji with a skin tone, or a letter with a combining accent, is several code points and several bytes, and neither len nor rune counting gives you a user-perceived length. The standard library has no grapheme cluster API, so truncating a display name to 20 characters needs golang.org/x/text or a segmentation library, and a plain []rune slice can still cut a flag emoji in half. Validate length in runes for most form fields, and decide deliberately whether that is enough. Runes, bytes and Unicode without the confusion goes through that in detail, and the next stop for conversions is converting between types.