Tell Null Missing and Zero Apart in Go JSON Decoding

Three different things can happen to a field in a JSON document: it can be absent, it can be present with the value null, or it can be present with a value that happens to be zero (0, false, ""). For a person reading the document they mean three different things. For an update endpoint they mean "leave it alone", "clear it" and "set it to nothing in particular", and getting them confused is how a profile loses its bio when someone changes a name. Go's type system has two representations for these things at best, and the standard library's JSON decoder collapses some of them.

This article lays out what each decoding strategy can and cannot tell apart, with a table produced by running the code, then builds the small generic type that tells all three apart and shows how it behaves on encoding with the omitzero option added in Go 1.24. It is the decoding half of the story behind PATCH endpoints, it builds on marshaling JSON in Go, and it informs request validation. Everything ran on Go 1.25.6.

What each representation can see

The program decodes four documents into a struct that has a field in each of five styles, and prints what each style ends up knowing.

package main

import (
	"encoding/json"
	"fmt"
	"time"
)

type Doc struct {
	Plain  int             `json:"plain"`
	Ptr    *int            `json:"ptr"`
	PtrPtr **int           `json:"ptrptr"`
	Raw    json.RawMessage `json:"raw"`
	Opt    Optional[int]   `json:"opt"`
}

func describe(d Doc) string {
	ptr := "nil"
	if d.Ptr != nil {
		ptr = fmt.Sprint(*d.Ptr)
	}
	pp := "nil"
	if d.PtrPtr != nil {
		pp = "non-nil outer"
		if *d.PtrPtr == nil {
			pp += ", nil inner"
		} else {
			pp += fmt.Sprintf(", inner=%d", **d.PtrPtr)
		}
	}
	raw := "nil"
	if d.Raw != nil {
		raw = string(d.Raw)
	}
	opt := "absent"
	switch {
	case d.Opt.Null:
		opt = "null"
	case d.Opt.Present:
		opt = fmt.Sprintf("value %d", d.Opt.Value)
	}
	return fmt.Sprintf("plain=%d | ptr=%s | ptrptr=%s | raw=%s | opt=%s", d.Plain, ptr, pp, raw, opt)
}

type Out struct {
	OmitEmpty  *int          `json:"omitempty,omitempty"`
	OmitZeroT  time.Time     `json:"when,omitzero"`
	OmitEmptyT time.Time     `json:"when_old,omitempty"`
	Count      int           `json:"count,omitzero"`
	Opt        Optional[int] `json:"opt,omitzero"`
}

func main() {
	fmt.Println("== decoding one field in five documents")
	for _, in := range []string{`{}`, `{"plain":null,"ptr":null,"ptrptr":null,"raw":null,"opt":null}`, `{"plain":0,"ptr":0,"ptrptr":0,"raw":0,"opt":0}`, `{"plain":7,"ptr":7,"ptrptr":7,"raw":7,"opt":7}`} {
		var d Doc
		if err := json.Unmarshal([]byte(in), &d); err != nil {
			panic(err)
		}
		fmt.Printf("%-72s\n    %s\n", in, describe(d))
	}

	fmt.Println("\n== encoding: omitempty versus omitzero (Go 1.24+)")
	zero := 0
	for _, c := range []struct {
		label string
		v     Out
	}{
		{"all zero values", Out{}},
		{"pointer to 0, count 0", Out{OmitEmpty: &zero}},
		{"optional absent", Out{Opt: Optional[int]{}}},
		{"optional null", Out{Opt: Clear[int]()}},
		{"optional zero value", Out{Opt: Set(0)}},
	} {
		b, _ := json.Marshal(c.v)
		fmt.Printf("%-28s %s\n", c.label, b)
	}
}

== decoding one field in five documents
{}                                                                      
    plain=0 | ptr=nil | ptrptr=nil | raw=nil | opt=absent
{"plain":null,"ptr":null,"ptrptr":null,"raw":null,"opt":null}           
    plain=0 | ptr=nil | ptrptr=nil | raw=null | opt=null
{"plain":0,"ptr":0,"ptrptr":0,"raw":0,"opt":0}                          
    plain=0 | ptr=0 | ptrptr=non-nil outer, inner=0 | raw=0 | opt=value 0
{"plain":7,"ptr":7,"ptrptr":7,"raw":7,"opt":7}                          
    plain=7 | ptr=7 | ptrptr=non-nil outer, inner=7 | raw=7 | opt=value 7

== encoding: omitempty versus omitzero (Go 1.24+)
all zero values              {"when_old":"0001-01-01T00:00:00Z"}
pointer to 0, count 0        {"omitempty":0,"when_old":"0001-01-01T00:00:00Z"}
optional absent              {"when_old":"0001-01-01T00:00:00Z"}
optional null                {"when_old":"0001-01-01T00:00:00Z","opt":null}
optional zero value          {"when_old":"0001-01-01T00:00:00Z","opt":0}

Look at the four documents in turn.

The empty document {}. Nothing is present. The plain int is 0, the pointers are nil, the RawMessage is nil and the Optional says absent.

Everything null. This is where the styles diverge. The plain int is 0, exactly as when the key was missing. The *int is nil, exactly as when the key was missing. The **int is also nil. The RawMessage holds null, four bytes, and the Optional reports null. Only the last two can tell this document from the empty one.

Everything zero. The plain int is 0 again, identical to the first two cases. The *int is a non-nil pointer to 0, which is the one thing a pointer can do that a plain field cannot: tell zero from not-zero. The **int is a non-nil outer pointer to a pointer to 0. The RawMessage is 0, the Optional is a present value 0.

Everything seven. All five agree, and nothing is hard.

Collect that into a table of what each style can distinguish:

The double pointer does not work

You will find advice to use **T because "the outer pointer says whether the key was present, the inner whether the value was null". It is a clean idea and encoding/json does not implement it: for a JSON null the decoder sets the pointer you give it to nil, and with **int the pointer it sets is the outer one. The run above shows the outcome: ptrptr=nil for the null document, indistinguishable from absent. For this library, skip it.

RawMessage as the cheap option

json.RawMessage is a []byte that the decoder fills with the raw bytes of the value instead of interpreting them. Because the decoder only touches the field when the key is present, nil means absent. Because it stores bytes verbatim, a null value is stored as the bytes null, which is not nil. That is enough to tell the three states apart with no custom code:

• field == nil: absent.
• string(field) == "null": explicit null.
• anything else: a value, which you then decode with json.Unmarshal.

The cost is that you do the second decode yourself, and the type of the field in your struct no longer documents what the value should be. For one or two fields in one handler, it is a perfectly good solution. The approach in the PATCH article, a whole body as map[string]json.RawMessage, is the same idea applied to every key at once.

A generic Optional type

For API models with many optional fields, a typed wrapper reads better and carries the parsing with it.

package main

import (
	"bytes"
	"encoding/json"
)

// Optional records the three states a JSON field can be in.
type Optional[T any] struct {
	Present bool // the key appeared in the document
	Null    bool // and its value was null
	Value   T
}

func (o *Optional[T]) UnmarshalJSON(b []byte) error {
	o.Present = true // Unmarshal only calls this when the key exists
	if bytes.Equal(bytes.TrimSpace(b), []byte("null")) {
		o.Null = true
		return nil
	}
	return json.Unmarshal(b, &o.Value)
}

// MarshalJSON writes null for the null state; combined with omitzero, an absent field is left out entirely.
func (o Optional[T]) MarshalJSON() ([]byte, error) {
	if o.Null {
		return []byte("null"), nil
	}
	return json.Marshal(o.Value)
}

// IsZero is what `omitzero` consults. Absent fields have Present false.
func (o Optional[T]) IsZero() bool { return !o.Present }

// Set builds a present value, for tests and for code that constructs patches.
func Set[T any](v T) Optional[T] { return Optional[T]{Present: true, Value: v} }

// Clear builds an explicit null.
func Clear[T any]() Optional[T] { return Optional[T]{Present: true, Null: true} }

Optional[T] has three fields: Present, Null and Value. UnmarshalJSON is called by the decoder when it meets the key, so it sets Present and then checks whether the raw bytes are null. If so it sets Null and leaves Value at its zero; otherwise it decodes into Value, which gives the right errors for the wrong type. A key that never appears never calls the method, so Present stays false. The documented behavior of encoding/json is that UnmarshalJSON is called for a JSON null too when the target is a non-pointer type, which is exactly why this works; the run shows it.

The encoding side has two methods. MarshalJSON writes null for the null state and the value otherwise. IsZero returns !Present, which matters for the option below.

Set and Clear are constructors for code that builds patches, such as an API client or a test. They read better than a struct literal with three fields.

omitzero and the encoding side

Decoding is only half of it. When you encode a struct, omitempty omits a field with an "empty" value (false, 0, a nil pointer or interface, an empty slice, map or string), and it cannot omit a struct. Go 1.24 added omitzero, which omits any field whose value is the zero value of its type, or whose type has an IsZero() bool method that returns true.

The second half of the program's output shows the difference:

• A zero time.Time tagged omitempty is printed as 0001-01-01T00:00:00Z because a struct is never "empty". The same field tagged omitzero is left out. This alone removes a classic wart from API output.
• An int tagged omitzero is omitted at zero, like omitempty. A *int pointing at zero tagged omitempty is kept as 0, because the pointer is not nil.
• Optional[int] tagged omitzero is omitted when absent (through IsZero), printed as null when cleared, and printed as 0 when set to zero. That is the full set of three states, round-tripped.

The omitzero option requires Go 1.24 or later, which matters if a library you publish supports older versions. It is a separate feature from the work on a new encoding/json/v2 package, discussed in JSON v2 in Go.

Tests that pin the behavior

package main

import (
	"encoding/json"
	"testing"
)

type patch struct {
	Bio Optional[string] `json:"bio,omitzero"`
	Age Optional[int]    `json:"age,omitzero"`
}

func TestThreeStatesDecode(t *testing.T) {
	var p patch
	if err := json.Unmarshal([]byte(`{"bio":null,"age":0}`), &p); err != nil {
		t.Fatal(err)
	}
	if !p.Bio.Present || !p.Bio.Null {
		t.Errorf("bio should be an explicit null: %+v", p.Bio)
	}
	if !p.Age.Present || p.Age.Null || p.Age.Value != 0 {
		t.Errorf("age should be a present zero: %+v", p.Age)
	}
	var none patch
	json.Unmarshal([]byte(`{}`), &none)
	if none.Bio.Present || none.Age.Present {
		t.Errorf("absent keys must stay absent: %+v", none)
	}
}

func TestRoundTripKeepsEachState(t *testing.T) {
	in := patch{Bio: Clear[string](), Age: Set(0)}
	b, _ := json.Marshal(in)
	if string(b) != `{"bio":null,"age":0}` {
		t.Fatalf("got %s", b)
	}
	b, _ = json.Marshal(patch{})
	if string(b) != `{}` {
		t.Fatalf("empty patch marshaled to %s", b)
	}
}

func TestWrongTypeIsStillAnError(t *testing.T) {
	var p patch
	if err := json.Unmarshal([]byte(`{"age":"old"}`), &p); err == nil {
		t.Fatal("a string for an int should fail")
	}
}

=== RUN   TestThreeStatesDecode
--- PASS: TestThreeStatesDecode (0.00s)
=== RUN   TestRoundTripKeepsEachState
--- PASS: TestRoundTripKeepsEachState (0.00s)
=== RUN   TestWrongTypeIsStillAnError
--- PASS: TestWrongTypeIsStillAnError (0.00s)
PASS
ok  	example.test/lab	1.263s

The first test decodes {"bio":null,"age":0} and asserts that the bio is an explicit null and the age a present zero, then decodes {} and asserts that both are absent. The second marshals a patch with a cleared bio and a zero age and requires exactly {"bio":null,"age":0}, and marshals an empty patch and requires {}. The third checks that a string where an integer belongs is still an error, since a custom UnmarshalJSON that swallowed type errors would be a quiet regression.

Choosing

Use plain fields when zero is not a meaningful input, such as a name that must be non-empty, where "missing" and "empty" are both the same validation failure.

Use a pointer when you need to tell zero from missing and null does not matter, for instance an optional discount where 0 is valid and nothing else is.

Use RawMessage or Optional[T] when null means something: PATCH bodies, where null clears a field, and any API where "explicitly nothing" differs from "unspecified". Pick RawMessage for a one-off and Optional for models.

Reject null deliberately where it does not make sense. If name can never be null, the setter or validator should return a clear error for it, rather than treating it as absent.

One more thing: be careful with database/sql, which has its own family of types for the same problem (sql.NullString, sql.Null[T]). Converting between the JSON representation and the SQL one is a good place for a test that walks all three states. The decoding rules are documented in the encoding/json package, and the Go 1.24 release notes describe omitzero at go.dev/doc/go1.24.