Concurrency Bugs in Go That Your Tests Will Not Catch

A Go test suite that passes under -race is a good sign, and a lot of teams stop there. The race detector is excellent at what it does: it reports unsynchronized accesses to the same memory that actually happened during the run. It does not claim to find anything else, and the bugs that cause most concurrency incidents are in the "anything else" column. They are logic errors about order and lifetime: who does what first, who stops whom, who waits for whom. A program can be free of data races and still hang, leak, drop work or compute the wrong answer.

This roundup lists ten of these bugs, one at a time, with why tests miss it and what catches it. One has a runnable demonstration, because it is the case people find hardest to believe. The list is a companion to data races, the race detector and testing concurrent code, and it belongs to the concurrency patterns cluster. Everything ran on Go 1.25.6.

1. Check then act on an atomic

Atomics make individual operations safe, and programmers sometimes hear "atomics are thread-safe" and stop thinking. A balance with an atomic load and an atomic subtract has no data race, and still lets two goroutines both pass the check before either subtracts:

package main

import (
	"fmt"
	"sync"
	"sync/atomic"
	"time"
)

type Account struct{ balance atomic.Int64 }

func (a *Account) Withdraw(n int64) bool {
	if a.balance.Load() >= n { // check
		time.Sleep(time.Microsecond) // any work between the check and the act
		a.balance.Add(-n)            // act: another goroutine may have passed the same check
		return true
	}
	return false
}

func main() {
	var acct Account
	acct.balance.Store(500)

	var wg sync.WaitGroup
	var approved atomic.Int32
	for range 8 {
		wg.Go(func() {
			if acct.Withdraw(100) {
				approved.Add(1)
			}
		})
	}
	wg.Wait()
	fmt.Println("approved withdrawals:", approved.Load(), "(only 5 were affordable)")
	fmt.Println("final balance:", acct.balance.Load())
}

approved withdrawals: 7 (only 5 were affordable)
final balance: -200

The run was under -race and the detector printed nothing, since every access was atomic. The result is wrong: seven withdrawals were approved where only five were affordable, and the balance ended at -200. The check and the update have to be one atomic step: hold a mutex around both, or use CompareAndSwap in a loop so the subtraction only succeeds if the value is still what the check saw (compare and swap). The compare-and-swap repair retries until the update and the check refer to the same value:

func (a *Account) Withdraw(n int64) bool {
	for {
		old := a.balance.Load()
		if old < n {
			return false
		}
		if a.balance.CompareAndSwap(old, old-n) { // succeeds only if nobody changed it since the Load
			return true
		}
	}
}

If another goroutine changed the balance between the load and the swap, the swap fails and the loop re-reads. For anything with more than one field, a mutex is clearer than a CAS loop. Tests miss it because it needs two goroutines to interleave between two lines, and a unit test with one caller never produces that. A stress test with many goroutines and an invariant assertion (balance never negative) does.

2. The goroutine that never ends

A goroutine blocked forever on a channel nobody will touch again passes every test, because the test finished and the process exited. In a server it is a slow memory leak that appears over days (goroutine leaks). Catch it with a leak check at the end of tests (goleak, or comparing runtime.NumGoroutine before and after, leak detection) and by exporting the goroutine count as a metric so a steady climb is visible.

3. Shutdown that loses work

Tests that start a component and exit never exercise stopping it. Real shutdowns reveal that the consumer exits while a producer is still sending, that the database closes before in-flight requests finish, that a signal handler returns and main exits with jobs queued. The fix is a designed shutdown sequence (starting and stopping goroutines, graceful shutdown) and a test that stops the component under load and asserts that accepted work completed.

4. A copied mutex

A struct containing a sync.Mutex that is passed or returned by value gets a copy of the lock. Two copies protect nothing from each other. The code compiles and often passes tests, because the copy happens in a path tests don't stress. go vet finds it:

package main

import "sync"

type Counter struct {
	mu sync.Mutex
	n  int
}

// Value receiver: every call locks a private copy of the mutex.
func (c Counter) Inc() {
	c.mu.Lock()
	defer c.mu.Unlock()
	c.n++
}

func main() {
	var c Counter
	c.Inc()
}

# example.test/lab
# [example.test/lab]
./main.go:11:9: Inc passes lock by value: example.test/lab.Counter contains sync.Mutex

go vet reports that Inc passes a lock by value, which is the whole bug. The method also increments a copy of n, so the counter never changes at all. The rule is that types containing a mutex, a WaitGroup or anything from sync are used by pointer, and go vet in CI enforces it (struct copies).

5. Lost wakeups with a condition variable

sync.Cond.Wait requires the caller to hold the lock and to re-check the condition in a loop. Code that does if !ready { cond.Wait() } or signals before the waiter starts waiting misses the wakeup and hangs. It works in tests where the timing happens to be right. Prefer a channel, which remembers a close even if the receiver arrives late (sync.Cond).

6. Unbounded fan-out

A goroutine per item works for a hundred items in a test and takes down production at a million. Tests use small inputs. Review any for ... { go ... } for a bound, a limit or a pool (bounded parallelism).

7. Order assumptions

"The consumer starts before the producer" and "this log line appears before that one" are facts about one run. Tests that pass 99 times and fail on the hundredth depend on scheduling. Run the suite with -count=50 and -cpu 1,2,8 to vary the interleavings, and shuffle with -shuffle=on, which also exposes tests that depend on each other.

8. Waiting without a deadline

A call that waits on a network peer or another goroutine with no timeout works until the peer misbehaves. Tests use well-behaved fakes. A fake that sometimes hangs, plus a deadline in the code under test, is how you cover it (timeouts on blocking calls).

9. Context that is not passed on

A function accepts a ctx and then calls a method that doesn't take one. Cancellation silently stops at that boundary, and no test cancels mid-call. Grep for functions with a ctx parameter that call blocking functions without it, and write a test that cancels while the callee is blocked and asserts it returns promptly (context cancellation).

10. Panics in goroutines

A panic in a goroutine that has no recover ends the process, whichever goroutine started it. Tests rarely feed bad input to background goroutines, so the first panic arrives in production (recovering in goroutines).

Why these survive code review too

Concurrency bugs of this kind are hard to see in a diff because each line is correct on its own. Load is fine, Add is fine, the check is reasonable, the sleep stands for ordinary work. The bug lives in the gap between lines, and reviewers read lines. It helps to review with a different question: "what if another goroutine runs right here?" Put a finger between each pair of statements that touch shared state and ask what could change. Most of the time the answer is nothing, and the one time it is not, you have found the bug that would have taken a week of production logs to diagnose.

It also helps to keep the shared state small and visible. A type that owns its data behind methods, with the lock in the same struct, gives the reviewer one place to look. A package-level variable touched from five files gives them none (confinement and ownership).

A review habit that catches most of them

For every goroutine in a diff I ask four questions. Who starts it and who waits for it? What makes it stop? What does it share, and how is each shared variable protected? What happens to it when its work is cancelled halfway? A diff that answers all four in its structure is rarely wrong in the ways above. A diff that cannot answer one is where the bug is.

Tools that help, in the order I add them

1. go test -race ./... in CI, always. It catches the real data races.
2. go vet ./... in CI. It catches copied locks, Add called inside the goroutine for a WaitGroup, and loop-variable capture on old Go versions.
3. A leak check in TestMain or in tests that start long-lived goroutines.
4. Repeats and shuffles: -count=20 -shuffle=on on the concurrent packages, nightly if it is too slow for every push.
5. Stress tests with invariants for anything with shared state: many goroutines, a loop, an assertion at the end that cannot hold if an update was lost.
6. Production visibility: goroutine count, mutex wait time and a way to take a goroutine dump from a live process (goroutine dumps).

The race detector is a floor, and the official article on it says what it can and cannot find. The Go memory model is the reference for what orders what, and the go vet documentation lists the analyzers that run by default, including copylocks.