Skip to content
gohacks.dev
Databases

Working With SQL in Go From database sql to pgx and sqlc

A complete guide to SQL in Go: pick a driver, size the pool, scan rows, run transactions, avoid injection, handle types, migrate and test. Runnable tour.

24 min read
Summarize in AI
Flat illustration of a database beside the table it holds.

Almost every Go service talks to a SQL database, and the way it does so decides whether the service is easy to operate or an incident waiting for a bad Friday. The standard library gives you a small, stable interface. The ecosystem gives you drivers, scanners, query builders, generators and ORMs. This guide is the map: what each layer does, which choices matter, which defaults are traps, and where each topic is covered in depth. It carries one program you can run end to end, and every claim about behaviour in the cluster comes from a run on Go 1.25.6 against PostgreSQL 18, MySQL 8.4 or SQLite.

Tested with Go 1.25.6, pgx v5.11.0, PostgreSQL 18.6 and MySQL 8.4.11 in Docker, modernc.org/sqlite v1.50.0, macOS on Apple Silicon (arm64, Apple M1 Max)

It is long on purpose, and it is built so that you can leave after any section. The first section gives you the shape of a good data layer. The rest is the evidence and the details.

The short version

If you read nothing else, these ten rules cover most of what goes wrong in production.

  1. One *sql.DB (or one pgx pool) per database, created at start-up and shared. Never one per request. See how database/sql works.
  2. Bound the pool. Set MaxOpenConns from the database's connection budget, and know that the default is unlimited. See pool settings and too many connections.
  3. A context with a deadline on every call, so a slow database costs a fast error, not a pile of stuck goroutines. See query timeouts.
  4. Values are parameters, always. Names and keywords come from allowlists. See SQL injection and dynamic queries.
  5. Close what you open: rows, transactions, connections. See closing rows.
  6. Transactions use a deferred rollback, and everything inside goes through the transaction. See transactions.
  7. Know your isolation level, and retry serialization failures and deadlocks. See isolation levels and retrying.
  8. Scan by name into structs, with explicit handling of NULL. See scanning rows and NULL values.
  9. Let the database enforce invariants: constraints, unique indexes, foreign keys. The application will be wrong sometimes.
  10. Own the schema with a migration tool and test against the real database. See migrations with goose.

One program, end to end

The program below uses the native pgx interface against PostgreSQL 18. It is the shape of a small data layer: connect, create a schema, insert, scan, transact, time out and hit a constraint. Everything it prints is the output of a real run.

tour.goGo
package main

import (
	"context"
	"errors"
	"fmt"
	"log"
	"time"

	"github.com/jackc/pgx/v5"
	"github.com/jackc/pgx/v5/pgconn"
	"github.com/jackc/pgx/v5/pgxpool"
)

const dsn = "postgres://postgres:[email protected]:55432/lab?sslmode=disable"

type Account struct {
	ID      int       `db:"id"`
	Owner   string    `db:"owner"`
	Cents   int64     `db:"cents"`
	Closed  *string   `db:"closed_reason"` // NULL while the account is open
	Created time.Time `db:"created_at"`
}

func must(err error) {
	if err != nil {
		log.Fatal(err)
	}
}

// retryable: PostgreSQL asked the client to run the whole transaction again.
func retryable(err error) bool {
	var pgErr *pgconn.PgError
	return errors.As(err, &pgErr) && (pgErr.Code == "40001" || pgErr.Code == "40P01")
}

func transfer(ctx context.Context, pool *pgxpool.Pool, from, to int, cents int64) error {
	for attempt := 1; attempt <= 5; attempt++ {
		err := pgx.BeginTxFunc(ctx, pool, pgx.TxOptions{IsoLevel: pgx.Serializable}, func(tx pgx.Tx) error {
			var balance int64
			if err := tx.QueryRow(ctx, `select cents from accounts574 where id = $1`, from).Scan(&balance); err != nil {
				return err
			}
			if balance < cents {
				return fmt.Errorf("insufficient funds: have %d, need %d", balance, cents)
			}
			if _, err := tx.Exec(ctx, `update accounts574 set cents = cents - $1 where id = $2`, cents, from); err != nil {
				return err
			}
			_, err := tx.Exec(ctx, `update accounts574 set cents = cents + $1 where id = $2`, cents, to)
			return err
		})
		if err == nil || !retryable(err) {
			return err
		}
	}
	return errors.New("gave up after 5 attempts")
}

func main() {
	ctx := context.Background()

	// 1. Connect: a bounded pool, a ping, the server version.
	cfg, err := pgxpool.ParseConfig(dsn)
	must(err)
	cfg.MaxConns = 8
	cfg.MaxConnLifetime = time.Hour
	cfg.ConnConfig.ConnectTimeout = 3 * time.Second
	pool, err := pgxpool.NewWithConfig(ctx, cfg)
	must(err)
	defer pool.Close()
	must(pool.Ping(ctx))
	var ver string
	must(pool.QueryRow(ctx, `select split_part(version(), ' ', 2)`).Scan(&ver))
	fmt.Printf("1 connect:      PostgreSQL %s, pool max %d\n", ver, pool.Config().MaxConns)

	// 2. Schema: created in code here; a migration tool owns this in a real service.
	_, err = pool.Exec(ctx, `drop table if exists accounts574;
		create table accounts574 (
			id int generated always as identity primary key,
			owner text not null,
			cents bigint not null check (cents >= 0),
			closed_reason text,
			created_at timestamptz not null default now())`)
	must(err)
	fmt.Println("2 schema:       accounts574 created with a check constraint and a nullable column")

	// 3. Insert with RETURNING, parameters only.
	var id int
	for _, a := range []struct {
		owner string
		cents int64
	}{{"ada", 100000}, {"grace", 5000}} {
		must(pool.QueryRow(ctx, `insert into accounts574 (owner, cents) values ($1, $2) returning id`, a.owner, a.cents).Scan(&id))
	}
	_, err = pool.Exec(ctx, `update accounts574 set closed_reason = $1 where owner = 'grace'`, "requested by customer")
	must(err)
	fmt.Printf("3 insert:       2 rows, last id %d\n", id)

	// 4. Query many rows into structs by name, with a NULL column.
	rows, err := pool.Query(ctx, `select id, owner, cents, closed_reason, created_at from accounts574 order by id`)
	must(err)
	accounts, err := pgx.CollectRows(rows, pgx.RowToStructByName[Account])
	must(err)
	for _, a := range accounts {
		closed := "open"
		if a.Closed != nil {
			closed = "closed: " + *a.Closed
		}
		fmt.Printf("4 scan:         %-5s %7d cents, %s, timestamp set: %v\n", a.Owner, a.Cents, closed, !a.Created.IsZero())
	}

	// 5. A transaction that moves money, and one that is refused by the database rules.
	fmt.Printf("5 transfer ok:  %v\n", transfer(ctx, pool, 1, 2, 25000))
	fmt.Printf("5 transfer bad: %v\n", transfer(ctx, pool, 2, 1, 999999))
	var a1, a2 int64
	must(pool.QueryRow(ctx, `select (select cents from accounts574 where id = 1), (select cents from accounts574 where id = 2)`).Scan(&a1, &a2))
	fmt.Printf("5 balances:     ada=%d grace=%d total=%d\n", a1, a2, a1+a2)

	// 6. A deadline on a slow query: cancelled on the client and on the server.
	tctx, cancel := context.WithTimeout(ctx, 200*time.Millisecond)
	defer cancel()
	start := time.Now()
	var n int
	err = pool.QueryRow(tctx, `select 1 from pg_sleep(5)`).Scan(&n)
	fmt.Printf("6 timeout:      deadline exceeded=%v, returned in under 1s=%v\n", errors.Is(err, context.DeadlineExceeded), time.Since(start) < time.Second)

	// 7. The constraint is the last line of defence when the application is wrong.
	_, err = pool.Exec(ctx, `update accounts574 set cents = -1 where id = 1`)
	var pgErr *pgconn.PgError
	if errors.As(err, &pgErr) {
		fmt.Printf("7 constraint:   SQLSTATE %s (%s)\n", pgErr.Code, pgErr.ConstraintName)
	}

	st := pool.Stat()
	fmt.Printf("8 pool:         total=%d acquired=%d idle=%d\n", st.TotalConns(), st.AcquiredConns(), st.IdleConns())
	_, _ = pool.Exec(ctx, `drop table accounts574`)
}
Shell
go run . 2>&1
Text
1 connect:      PostgreSQL 18.6, pool max 8
2 schema:       accounts574 created with a check constraint and a nullable column
3 insert:       2 rows, last id 2
4 scan:         ada    100000 cents, open, timestamp set: true
4 scan:         grace    5000 cents, closed: requested by customer, timestamp set: true
5 transfer ok:  <nil>
5 transfer bad: insufficient funds: have 30000, need 999999
5 balances:     ada=75000 grace=30000 total=105000
6 timeout:      deadline exceeded=true, returned in under 1s=true
7 constraint:   SQLSTATE 23514 (accounts574_cents_check)
8 pool:         total=1 acquired=0 idle=1

Read the output line by line, because each line is a topic.

1, connect. The pool is created from a URL with ParseConfig, bounded at 8 connections, with a connection lifetime and a dial timeout. Ping forces the first connection, so a wrong password fails here and not on the first request. The program logs the server version, PostgreSQL 18.6, a habit that saves time in incidents. The details are in connecting to PostgreSQL, and the MySQL equivalent in connecting to MySQL.

2, schema. The accounts574 table has an identity primary key, a not null owner, a check (cents >= 0) constraint, a nullable closed_reason and a timestamptz with a default. The schema is created in code only to keep the example one file; a real service lets a migration tool own it.

3, insert. Parameters go in as $1, $2; returning id hands the generated key back in the same round trip. No string is ever joined into the SQL.

4, scan. pgx.CollectRows with RowToStructByName fills a slice of Account structs by column name, closes the rows and reports the error. The nullable column maps to a *string: nil for open accounts, a value for the closed one. The output shows the closed account's reason. See scanning rows into structs.

5, transfer. A Serializable transaction, a retry loop and a balance check. The good transfer succeeded (<nil>); the bad one returned insufficient funds: have 30000, need 999999 and rolled back, so the final line reads ada=75000 grace=30000 total=105000: no money lost, no money created. The transaction helper is the pattern from the transaction article, and the loop retries SQLSTATE 40001 and 40P01 as in retrying serialization failures.

6, timeout. A query that sleeps five seconds, given 200 ms. The error is the context's, and the call returned in under a second. The driver also cancels the query on the server, as measured in query timeouts.

7, constraint. A raw update that sets a negative balance bypasses all the Go code, and the database refuses it with SQLSTATE 23514, naming the constraint accounts574_cents_check. The application checked the balance too, and the constraint is why that check is not the last line of defence.

8, pool. After the run the pool holds idle connections and none acquired: nothing leaked.

Keep this shape in your head. The rest of the guide fills in each of its parts.

The layers

A Go data layer has five layers, and most debates are about which to include.

LayerJobChoices
DriverSpeaks the database's protocolpgx, lib/pq, go-sql-driver/mysql, modernc.org/sqlite, mattn/go-sqlite3
InterfacePool, transactions, rowsdatabase/sql or pgx native
ScannerRows into structsManual Scan, sqlx, pgx.CollectRows
Query constructionBuild SQL, including dynamic partsPlain strings, squirrel, goqu, generated code with sqlc
Higher levelSchema, relations, hooksORMs: GORM, Bun, Ent

Migrations and testing cut across all five.

You can use one choice per layer. The most common healthy stack in 2026 is pgx for PostgreSQL, pgx.CollectRows or sqlx for scanning, plain SQL or sqlc for queries, goose for migrations and no ORM. The most common stack for teams that value rapid CRUD is GORM or Bun on top of database/sql. Both work. The dispute between them is real, and this guide takes a side: for services where the data layer matters, write SQL you can read, and keep a thin Go layer. The reasons are the next section.

SQL against ORMs, and a side

An ORM promises you will not write SQL. In practice you write it anyway, through a method chain, and debug it by reading the SQL the ORM produced. The costs show up late: queries you did not intend (the N+1 pattern in nested structs and joins), implicit transactions and isolation levels you did not choose, schema and migrations coupled to struct tags, performance work that fights an abstraction, and behaviour that differs between databases in ways the documentation does not state.

The benefits are real too. For CRUD-heavy admin applications with simple relations, an ORM saves a great deal of typing and gives migrations and validation hooks. Teams without strong SQL skills get something that works. Ent's generated, typed API removes most of the string typing.

The position here is narrower than "ORMs are bad". For a service whose value is its data and its queries, which is most backends, hand-written SQL with a scanner is easier to reason about, to review, to optimise and to move between teams. sqlc improves on plain strings by checking your SQL against the schema at generation time and generating typed Go. Where you need dynamic filters, a small builder helps; the comparison is in query builders compared. If you do choose an ORM, still learn the database's isolation levels, indexes and explain, because the ORM cannot spare you.

Choosing a database

This guide is about SQL from Go, and the first question is which engine.

PostgreSQLMySQL and MariaDBSQLite
ModelServerServerLibrary, one file
ConcurrencyMany writers (MVCC)Many writers (InnoDB)One writer, many readers (WAL)
TypesRich: arrays, JSONB, ranges, enums, UUIDSimpler; JSONDynamic, advisory
Transactional DDLYesNo (implicit commits)Yes
Default isolationRead CommittedRepeatable ReadSerializable (single writer)
Typical Go driverpgxgo-sql-driver/mysqlmodernc or mattn
Choose whenDefault choice for a serviceYou inherit it, or the host provides itEmbedded, single process, tests

For a new service with no constraints, PostgreSQL is the default recommendation: it has the strongest feature set, the best tooling in Go, and behaviour that rewards learning. MySQL and MariaDB are fully serviceable and sometimes the only option on a host. SQLite is excellent when the database belongs to one process; the decision between the two drivers is in SQLite with and without cgo, with the configuration in pure Go SQLite.

Write portable SQL only when you have to. The types and operators that make PostgreSQL pleasant, arrays, JSONB, = any($1), returning, are the reason to choose it; avoiding them for portability you will never use is a poor trade.

Drivers and the pool

The Go side starts with a driver registered under a name. sql.Open("pgx", dsn) returns a *sql.DB, which is a pool, not a connection; it dials nothing until first use. The behaviours worth knowing:

  • A wrong password does not fail at Open. Call PingContext at start-up. See connecting to PostgreSQL.
  • Rows, transactions and connections hold a connection until closed. The pool's counters show which, as measured in the explainer.
  • The pool defaults are wrong for services: unlimited open connections, two idle, no lifetime. Four settings fix that: MaxOpenConns, MaxIdleConns, ConnMaxLifetime and ConnMaxIdleTime. In the cluster's lab, twenty 100 ms queries took 1040 ms through a pool of 2, 420 ms through 5 and 120 ms through 20, so the pool is a queue, and WaitCount is the metric that shows it. The sizing arguments, from the server's budget and from Little's law, are in pool settings.
  • Without a bound, a spike becomes an outage. 130 concurrent one-second queries on an unbounded pool against default PostgreSQL gave 100 successes and 30 failures with FATAL: sorry, too many clients already; the walk-through and the fixes are in too many connections.

For PostgreSQL, choose between pgx's native pool (pgxpool) and the database/sql adapter. Native gives you batches, COPY, LISTEN/NOTIFY, richer types and slightly better performance. The adapter gives you the standard API that sqlx, sqlc in database/sql mode and most ORMs expect. If you need both, stdlib.OpenDBFromPool shares one pool.

For MySQL, the driver needs parseTime=true, utf8mb4 and a connection lifetime under the server's wait_timeout, or you meet the failures described in connecting to MySQL.

Reading data

A query is three steps: send SQL with parameters, iterate rows, scan columns. Each has a failure mode.

Closing. db.Query returns Rows, which holds a connection until you close it or read to the end. An early return without defer rows.Close() leaks the connection, and with a small pool the next query waits until its context expires. In the lab for closing rows a single leaked Rows left inUse=1 and the next query failed with context deadline exceeded. After the loop, return rows.Err(): it is the only way to see an error that ended the iteration, such as a dropped connection halfway through a result. QueryRow(...).Scan closes itself.

Scanning. A manual Scan(&a, &b, &c) loop is explicit and brittle: the column list lives in two places, and swapping two columns of the same type produces wrong data and no error. Scanning by name removes that. pgx.CollectRows with RowToStructByName and sqlx.Select both map columns to db-tagged fields, close the rows and check the error; a column the struct does not have is an error with both, which is the point. The comparison, with the error messages, is in scanning rows into structs, and the sqlx features in what sqlx adds.

NULL. SQL has a value Go does not. Scanning NULL into a plain string fails with converting NULL to string is unsupported. The four options, pointers, sql.NullString and friends, the generic sql.Null[T] and COALESCE, behave differently in JSON output and in what they preserve; the table is in scanning NULL values. The best fix is in the schema: declare a column NOT NULL unless the domain needs absence.

Joins. A one-to-many join returns one row per child. Scan the flat rows and group them by parent in Go, aggregate with json_agg in PostgreSQL, or run one query per level with = any($1). Avoid N+1, one query per parent, which scales with the page size. The three approaches, with the left-join NULLs and the pagination trap, are in nested structs and joins.

Prepared statements. The lore says prepare everything. The measurement says little: 5000 single-row lookups cost 220 µs with the driver's cached prepares, 260 µs with the simple protocol and 230 µs with an explicit Prepare on a local PostgreSQL, so the round trip dominated. Prepared statements help most for complex plans run often, and they do not protect against injection by themselves; parameters do. See prepared statements.

Changing data safely

Transactions

A transaction is how a group of changes becomes one change. In Go the pattern is BeginTx, defer tx.Rollback(), statements through tx, tx.Commit(). The deferred rollback is safe after a successful commit (it returns sql.ErrTxDone and does nothing), and it makes every early return roll back. In the lab for transactions a crash between a debit and a credit left a=700 b=300 total=1000: the debit was undone. Most data layers wrap this in a helper that takes a function, so call sites cannot forget it. Keep transactions short, do nothing slow inside, and never use db where you meant tx.

Isolation levels and the anomalies they allow

The isolation level decides what concurrent transactions see. PostgreSQL's default, Read Committed, lets a transaction read the same row twice and get different answers. Repeatable Read gives a snapshot but allows write skew. Serializable prevents both by aborting one transaction. The lab for isolation levels ran two doctors checking that someone else was on call and then going off call: at Read Committed and Repeatable Read both committed and nobody was on call; at Serializable the second commit failed with could not serialize access due to read/write dependencies.

The practical policy: run at Read Committed, express invariants as constraints and single-statement updates (set n = n + 1), and use Serializable or explicit locks for the few multi-row rules that nothing else can express. MySQL's InnoDB defaults to Repeatable Read with different mechanics, so test on the database you deploy.

Retrying

At the strict levels, aborts are normal. PostgreSQL signals them with SQLSTATE 40001 (serialization failure) and 40P01 (deadlock); MySQL with errors 1213 and 1205. The client must roll back and run the whole transaction again, with a cap and a short randomised delay. In the experiment of retrying serialization failures, eight workers incrementing a counter at Serializable ended at 8 with a retry wrapper and at 2 without it, with six aborted. The function you retry must have no side effects outside the database.

Timeouts and cancellation

Every call takes a context. A 200 ms deadline cancelled a five second query on the client and the server, and PostgreSQL's pg_stat_activity showed no backend still running it. A server-side statement_timeout is the backstop for sessions the client forgot to bound. The transaction context rolls the transaction back when it ends, which protects the pool. Derive database deadlines from the request's budget, with a middleware at the edge. See query timeouts.

Security

SQL injection is still the commonest way a Go data layer fails. The rule has no exceptions: values travel as parameters, never inside the SQL text. The lab for SQL injection ran four payloads against a concatenated query and a parameterised one. A tautology returned every user, a UNION leaked the secret column, and a stacked DROP TABLE removed the table in a driver that runs multiple statements. The parameterised version returned nothing for all four, because the input never became syntax.

Placeholders carry values, not identifiers. A sort column, a table name or a direction needs an allowlist: a map from the user's word to one of your constants. Dynamic filters are built from fixed fragments plus an argument list, with IN lists generated to the right length; the patterns, and the same thing in squirrel, are in dynamic queries. Add defence in depth that does not depend on every query being right: a database role with least privilege, separate roles for migrations and reporting, and no multiStatements on MySQL.

Types that need care

Most data bugs are type bugs. Five need their own articles.

  • Timestamps. Use timestamptz for instants. In the lab for time zones, a Tokyo 09:00 written to a plain timestamp came back as 09:00 UTC, nine hours off, with no error, while timestamptz kept the instant and lost only the location. Pin the session zone, convert to UTC at the boundary and store user zones as IANA names.
  • Money. float64 gives 0.9999999999999999 for ten tenths. Use integer cents, or numeric with a decimal type, and allocate remainders explicitly when splitting: three shares of 100.00 rounded individually lose a cent, while cents gave 3334, 3333 and 3333. See money.
  • Keys. bigserial for internal keys; UUIDv7 or ULID when IDs are generated outside the database. A random UUIDv4 built a 2.09 MB index for 50000 rows against 1.52 MB for v7 and 1.09 MB for a serial. See choosing a key.
  • JSON. jsonb stores documents you query by ->> and @> with a GIN index and update with jsonb_set. It normalises what you send, sorting keys and dropping duplicates. See JSONB from Go.
  • Arrays and enums. pgx maps slices to arrays and = any($1) replaces the IN (?, ?, ?) list you would have to expand by hand. NULL and empty arrays stay distinct. See arrays and custom types.

Building queries

Plain strings are fine until queries are dynamic. Then there are three directions. A builder such as squirrel or goqu assembles text and arguments; for one filtered, paged query, squirrel produced $1-style placeholders and goqu quoted every identifier and parameterised even the limit and offset. A generator, sqlc, turns SQL files into typed functions and checks them against the schema. An ORM takes over the SQL. The trade-offs and the actual generated SQL are in query builders compared. A reasonable mix is sqlc for the fixed majority and a builder for the dynamic reports, both on the same pool.

The schema is code

A migration tool applies numbered changes in order, records what ran and supports rollback. goose, run as a library with embedded SQL files, took a database from version 0 to 3, rolled one step back, and re-applied it; each migration ran in a transaction on PostgreSQL, so a failure left the previous version. See migrations with goose.

The discipline matters more than the tool. Never edit an applied migration; add a new one. Make changes backward compatible, in the expand-and-contract style: add a nullable column, deploy code that handles both, backfill, then tighten. Know which statements lock the table (create index without concurrently, adding a not null constraint), set lock_timeout so a migration fails fast, and keep large data backfills out of the migration transaction. Run the same migrations in development, in tests and in production.

Testing database code

Database code has behaviour that only a database shows: constraints, locking, types, plans. A test strategy therefore uses the real engine for the code that talks to it and fakes for everything else.

  • Fast, portable SQL: SQLite in memory, one private database per test, with the caveats in testing against SQLite: foreign keys off by default, types advisory.
  • PostgreSQL features: a real PostgreSQL, started from the test with testcontainers, as in testing against Postgres.
  • Speed: one container per package, one transaction per test, rolled back at the end, as in rollback tests, which also explains what a rollback cannot cover.
  • Concurrency: two transactions on two connections with explicit ordering, as the isolation lab does, and the technique in deterministic interleavings.
  • Migrations: one test that applies all of them to an empty database.

The overall plan, with tiers and budgets, is in a test strategy for Go services, and the whole practice starts at the Go testing guide.

Running it in production

A data layer is also an operations concern.

Metrics. Export db.Stats() or pool.Stat(): in-use and idle connections, WaitCount and WaitDuration, closed-for-idle and closed-for-age counts. A rising wait count with a flat in-use at the limit means the pool is too small or something holds connections too long; an in-use count that never falls means a leak. Also record query duration by name, and the count of retries and timeouts.

Logs. Log the SQL name and duration, never parameter values at a level shared with others. Set application_name on every connection so pg_stat_activity tells you who owns what.

Timeouts at three layers. The client context, the server statement_timeout, and the pool's wait. Together they bound how long anything can hold a connection.

Backups and restores. Rehearse the restore, not just the backup. For SQLite use VACUUM INTO or the online backup API, not a file copy.

Capacity. Do the multiplication: instances times pool size, summed over services, under max_connections with headroom for deploys and admins. Behind many small clients, use a pooler and know which features it disables.

Failover. Connection lifetimes below your proxy and DNS timeouts mean the pool follows a failover within minutes. Retry reads on connection errors; retry writes only when they are idempotent.

Observability of the database itself. pg_stat_statements for the slowest queries, explain (analyze, buffers) for the plans, and the slow query log on MySQL. Optimise with the database's numbers, then benchmark the Go side with the methods in the benchmark guide.

A triage table for the errors you will see

When something fails at 3 a.m., start from the message. Each row names the likely cause and the article that explains it.

You seeIt usually meansRead
FATAL: sorry, too many clients already or Error 1040Pool unbounded, or too many instancesToo many connections
Requests hang, then context deadline exceeded, pool in-use at the limitA leaked Rows or Tx, or a pool that is too smallClosing rows, pool settings
converting NULL to string is unsupportedA nullable column scanned into a plain typeNULL values
missing destination name X or struct doesn't have corresponding row fieldA column and a struct out of stepScanning rows, sqlx
could not serialize access due to ... (40001) or deadlock (40P01)A normal abort at a strict levelRetries, isolation
sql: transaction has already been committed or rolled backStatement after Commit, or the context endedTransactions
database is locked (5) (SQLITE_BUSY)SQLite writers contending with no busy timeoutPure Go SQLite
unsupported Scan, storing driver.Value type []uint8 into type *time.TimeMySQL without parseTime=trueConnecting to MySQL
Binary was compiled with 'CGO_ENABLED=0'The cgo SQLite driver built without cgoSQLite drivers
A time that is hours offtimestamp instead of timestamptz, or a zone assumptionTime zones
Totals off by a centFloats or per-share roundingMoney
invalid input value for enumA value outside the enumArrays and custom types
Rows from a stranger in a resultConcatenated SQLSQL injection
A migration hangs behind other queriesA lock wait, no lock_timeoutMigrations

Three decisions, worked through

A new internal service with a PostgreSQL database, five engineers. Use pgx with pgxpool, pgx.CollectRows for scanning, plain SQL in a queries package or sqlc, goose for migrations embedded in the binary, timestamptz and identity keys, and a transaction helper with retry. Test the repository against a PostgreSQL container with a transaction per test. Reasoning: nothing here needs an ORM, and every part is replaceable.

A CRUD admin tool for a handful of tables, one engineer. An ORM is a defensible choice here, because the value is speed and the tables are simple. Use its migrations if they exist, keep an eye on the SQL it generates, and set the same pool limits and timeouts. The rules about injection, transactions and time zones apply unchanged.

A command line tool that keeps local state. SQLite through modernc, so the tool builds without a C compiler and cross-compiles, with WAL mode and a busy timeout in the DSN, one writer connection and migrations run at start-up. Backups with VACUUM INTO. No server, no pool tuning beyond that.

The pattern in all three is to make the decision explicit and to record the reasons next to the code, so the next engineer can disagree with evidence.

What changes with scale

At a hundred requests per second, defaults mostly work. At a thousand, connection arithmetic, slow queries and lock contention show up, and the metrics above decide what to do: raise or lower the pool, add an index, shorten a transaction, move a report to a replica. At ten thousand, you add a pooler, read replicas and a cache, and you start sharding hot rows (a counter updated by every request is the classic one). At each step the measure-first rule holds: the pool lab showed that a smaller pool can finish the same work sooner because the server is not thrashing, which no amount of guessing would tell you.

Two limits are worth stating plainly. Application code cannot fix a missing index; explain can find it in a minute. And replicas introduce staleness: a read after a write may not see it, so route reads that must be fresh to the primary.

Where this guide does not apply

  • NoSQL and document stores. MongoDB, DynamoDB, Redis and the rest have their own drivers and consistency models. Some ideas transfer (pools, timeouts, retries), the SQL-specific ones do not.
  • Analytics at scale. Warehouses such as BigQuery or ClickHouse are used differently: bulk loads, columnar scans, few connections.
  • Very small programs. A script that makes ten queries does not need a pool configuration or a migration tool. A database/sql handle with a context is enough.
  • ORM-centred frameworks. If your team has chosen an ORM deliberately, most of the safety rules still apply, but the code shapes differ; read your ORM's documentation for its transaction, isolation and raw SQL APIs.
  • Distributed SQL. Systems such as CockroachDB or Spanner speak the PostgreSQL or MySQL protocol and add their own rules: Serializable by default, mandatory retries, different performance models. The retry wrapper here is essential there.

A production checklist

Use this as a review list for a service's data layer.

  1. A single pool, created in main, closed on shutdown after the server drains.
  2. MaxOpenConns set from a written connection budget, with idle and lifetime settings.
  3. Ping at start-up; the server version logged.
  4. A context with a deadline on every query; a server-side statement timeout configured.
  5. No string-built SQL; allowlists for identifiers; a test with hostile input.
  6. Every Rows, Tx and Conn closed on every path; rows.Err() checked.
  7. Transactions via a helper with a deferred rollback; nothing slow inside.
  8. Isolation level chosen on purpose; retry on 40001, 40P01, 1213.
  9. Rows scanned by name; NULL handled by design; no select * in production code.
  10. Timestamps in timestamptz and UTC; money in cents or numeric; keys chosen deliberately.
  11. Constraints for every invariant that can be stated in the schema.
  12. Migrations in version control, tested from empty, never edited after release.
  13. Pool and query metrics exported, with alerts on waits and errors.
  14. Integration tests against the real database engine; concurrency tests for the risky transactions.
  15. A restore rehearsed in the last quarter.

Where to go next

Start with the article closest to the problem in front of you. For the pool and connections: how database/sql works, PostgreSQL, MySQL, SQLite and modernc SQLite, pool settings and too many connections. For reading and writing: rows, NULL, prepared statements, transactions, isolation, retries and timeouts. For safety and construction: injection, dynamic queries, builders, sqlx, struct scanning and joins. For PostgreSQL types: JSONB, arrays, keys, time and money. For the schema: goose.

The primary sources are the Go team's database access tutorial, the database/sql package reference, the pgx documentation and the PostgreSQL manual, which is the best database documentation there is. Read its chapters on concurrency control and on indexes once; most of the Go advice above is a consequence of what they say.

Share

Keep reading

Get new articles by email

We send one email when a new article is published. No spam.

Unsubscribe in one click. No spam, ever.