Backward Compatible Changes in Go APIs and Their Limits

"We only added a field, so it is backward compatible." It is the sentence people say before a client breaks. Sometimes it is true. The trouble is that backward compatibility is not a property of a change on its own. It is a property of a change together with every client that exists, including how each one parses, what it assumes and what it does with values it did not expect.

This article does a small experiment: take a client written against version 1 of a JSON API, send it the responses a changed server would send, and see what happens. The ten changes cover the usual suspects. Then it turns the results into rules, for people who change servers and for people who write clients. It builds on API versioning, feeds into deprecating endpoints, and shares its concerns with strict JSON decoding. It ran on Go 1.25.6.

The old client

The client is the kind real teams write. It decodes into a struct, switches on the status, and assumes the list of items is never empty. It can be run in tolerant mode (the default for encoding/json) or strict mode (DisallowUnknownFields).

package main

import (
	"encoding/json"
	"fmt"
	"strings"
)

// Order is how the OLD client was written against v1 of the API.
type Order struct {
	ID     int      `json:"id"`
	Status string   `json:"status"`
	Total  float64  `json:"total"`
	Items  []string `json:"items"`
}

// oldClient decodes a response the way real clients do and then uses it the way real clients do.
func oldClient(body string, strict bool) (summary string, err error) {
	dec := json.NewDecoder(strings.NewReader(body))
	if strict {
		dec.DisallowUnknownFields()
	}
	var o Order
	if err := dec.Decode(&o); err != nil {
		return "", err
	}
	// Business logic written against the v1 enum.
	var label string
	switch o.Status {
	case "pending":
		label = "waiting"
	case "shipped":
		label = "on its way"
	case "cancelled":
		label = "cancelled"
	default:
		return "", fmt.Errorf("unhandled status %q", o.Status)
	}
	if len(o.Items) == 0 {
		return "", fmt.Errorf("order has no items") // the client assumes the field is always present
	}
	return fmt.Sprintf("order %d is %s, %d items, total %.2f", o.ID, label, len(o.Items), o.Total), nil
}

type change struct {
	name     string
	body     string // what the NEW server sends
	breaking string // what we expect to learn
}

func main() {
	v1 := `{"id":7,"status":"pending","total":19.5,"items":["a","b"]}`
	cases := []change{
		{"v1 baseline", v1, ""},
		{"add a new field", `{"id":7,"status":"pending","total":19.5,"items":["a","b"],"currency":"EUR"}`, ""},
		{"reorder fields", `{"items":["a","b"],"total":19.5,"status":"pending","id":7}`, ""},
		{"add an enum value", `{"id":7,"status":"refunded","total":19.5,"items":["a","b"]}`, ""},
		{"remove a field", `{"id":7,"status":"pending","total":19.5}`, ""},
		{"rename a field", `{"id":7,"state":"pending","total":19.5,"items":["a","b"]}`, ""},
		{"number becomes string", `{"id":"7","status":"pending","total":19.5,"items":["a","b"]}`, ""},
		{"id above 2^53", `{"id":9007199254740993123,"status":"pending","total":19.5,"items":["a","b"]}`, ""},
		{"null where a list was", `{"id":7,"status":"pending","total":19.5,"items":null}`, ""},
		{"nest a field", `{"id":7,"status":"pending","total":{"amount":19.5,"currency":"EUR"},"items":["a","b"]}`, ""},
	}
	fmt.Printf("%-24s %-9s %s\n", "change", "tolerant", "strict (DisallowUnknownFields)")
	for _, c := range cases {
		verdict := func(strict bool) string {
			_, err := oldClient(c.body, strict)
			if err != nil {
				msg := err.Error()
				if len(msg) > 48 {
					msg = msg[:48] + "..."
				}
				return "BREAKS: " + msg
			}
			return "ok"
		}
		t, s := verdict(false), verdict(true)
		if strings.HasPrefix(t, "BREAKS") {
			t = "BREAKS"
		}
		fmt.Printf("%-24s %-9s %s\n", c.name, t, s)
	}

	fmt.Println()
	s, err := oldClient(v1, false)
	fmt.Println("v1 result:", s, err)
	_, err = oldClient(cases[3].body, false)
	fmt.Println("new enum value:", err)
	_, err = oldClient(cases[8].body, false)
	fmt.Println("null list:", err)
	_, err = oldClient(cases[7].body, false)
	fmt.Println("huge id:", err)
}

change                   tolerant  strict (DisallowUnknownFields)
v1 baseline              ok        ok
add a new field          ok        BREAKS: json: unknown field "currency"
reorder fields           ok        ok
add an enum value        BREAKS    BREAKS: unhandled status "refunded"
remove a field           BREAKS    BREAKS: order has no items
rename a field           BREAKS    BREAKS: json: unknown field "state"
number becomes string    BREAKS    BREAKS: json: cannot unmarshal string into Go struct fie...
id above 2^53            ok        ok
null where a list was    BREAKS    BREAKS: order has no items
nest a field             BREAKS    BREAKS: json: cannot unmarshal object into Go struct fie...

v1 result: order 7 is waiting, 2 items, total 19.50 <nil>
new enum value: unhandled status "refunded"
null list: order has no items
huge id: <nil>

What happened

Look at the "breaks" column and notice the variety: some failures are decoding errors from encoding/json (cannot unmarshal string into Go struct field), some are logic errors in the client's own code (unhandled status "refunded", order has no items), and one only appears in other languages. That variety is the reason the rule "tests passed" is not enough. A decode that succeeds can still produce a program that does the wrong thing.

The safe ones

Adding an optional field to a response is the canonical safe change, and it is safe because Go's decoder ignores keys it does not know. The strict decoder, which some teams choose deliberately to catch typos, rejects it. If you control your clients, that strictness is your choice. If you do not, expect that some will have made it, and note it in the compatibility policy: clients must ignore unknown fields.

Reordering fields is safe for any JSON decoder worth the name, because objects are unordered by definition. Clients that parse by position (rare, but they exist with CSV-like arrays) are the exception.

Adding an optional field to a request is also safe: old clients do not send it, and the server must treat absence as the old behaviour. Loosening validation (accepting longer strings, more characters, a wider range) is safe for requests as well.

The surprising ones

New enum values. The experiment's most useful row. The new status refunded is valid JSON of the right type. The old client's switch has a default branch that returns an error, so it breaks. The server team thinks "I added a value"; the client team thinks "the server returned something outside the contract". Both are right. Treat enums as open in your documentation ("clients must handle unknown values by showing a neutral state or ignoring the item") and say it before the first client is written. Closed enums need a new field or a new version to extend.

Nullability. A list that was always present and is now null when empty breaks a client that took the first element or ranged over it expecting a slice (in Go, ranging over a nil slice is fine, but len(o.Items) == 0 triggers the client's own check). Pick one empty representation and keep it. If a field can become absent or null, document that from the start.

Defaults and ordering. Changing the default sort order of a list, the default page size, or the default value of an omitted parameter changes behaviour without changing any shape. A client that relied on "newest first" now processes the oldest items first. Treat defaults as part of the contract.

Validation. Tightening what you accept (a shorter maximum length, a new required field, a stricter pattern) breaks requests that used to work. The error is a 422 on a request that worked yesterday.

Status codes and error shapes. Changing 200 to 201, or 404 to 410, or the structure of error bodies, breaks clients that branch on them. Status codes and problem details are contract too.

Meaning. Keeping the shape and changing what a field means is the worst kind. A total that used to include tax and now excludes it passes every decoder and silently corrupts every invoice. There is no mechanical check for this; it needs a human to say "this is a new field".

Timing and limits. A response that is correct and now takes four times as long, or a rate limit that was lowered, can break a client with a tight timeout. These are also contract.

The language matters

The id row is "ok" in Go and wrong elsewhere. A JSON number above 2^53 fits in a Go int64 and in a JSON document, but a JavaScript client parses it into a double and loses the low digits. If you grow identifiers beyond that range, existing JavaScript clients silently read wrong ids. Send big ids as strings from the start. The same sort of difference applies to date formats, to integers versus floats in dynamically typed languages and to Unicode normalisation. When you ask whether a change is safe, ask it for every client language you have.

Rules for people changing the server

1. Only add. New optional fields, new endpoints, new optional parameters, new enum values (if documented as open).
2. Never remove or rename in place. Add the new name, keep the old one, mark the old one deprecated, measure usage, and remove after the sunset (deprecation).
3. Do not change types or meanings. If it means something different, give it a different name.
4. Keep defaults. Add a parameter for the new behaviour and leave the default as it was.
5. Keep errors stable. Machine-readable codes are API; messages are not.
6. Announce the rules. A short compatibility policy page that says what you promise and what clients must tolerate is worth more than a perfect change.

Rules for people writing clients

1. Ignore unknown fields, in decoders and in the structs you design.
2. Handle unknown enum values with a default branch that degrades gracefully instead of failing the whole response.
3. Treat missing and null as possible for anything not documented as required.
4. Do not rely on order of fields, or of lists the API does not promise to sort.
5. Pin what you can (an API version header, a client library version) so changes arrive when you choose.

Catching it before it ships

The cheapest protection is a compatibility test suite that keeps the old clients' expectations as fixtures. Store real v1 requests and responses in the repository, run them against the new code on every change, and fail when a recorded request stops being accepted or a recorded response loses a field. The shared contract article shows how to compare an OpenAPI document with the previous release to detect removed fields and tightened types, and contract tests record what each consumer actually uses, so the answer to "can I remove this?" is data.

The limit of the idea

Backward compatibility is a promise to keep old clients working, and it has a price: every decision you cannot reverse accumulates. At some point the cost of carrying the past is greater than the cost of a new version, and the honest move is a deliberate break with a long runway, not a thousand quiet compromises. Know the difference between a change that is compatible and a change you hope nobody notices. The Go project's own compatibility promise is a good model of how to write one down, and the semantic versioning specification defines the vocabulary of major, minor and patch changes that API versions borrow.