Skip to content
gohacks.dev
Databases

Connection Pool Settings in Go and What Each Number Does

Set database/sql pool limits with evidence: MaxOpenConns, MaxIdleConns, lifetimes and wait stats measured on PostgreSQL, plus a sizing rule from Little's law.

11 min read
Summarize in AI
Flat illustration of a database beside the table it holds.

A connection pool has four knobs in Go, and most services run with the defaults until something breaks. The defaults are: unlimited open connections, two idle ones, connections that live forever. Each of those is the wrong answer for a service under load, in a different way. This guide sets each number from evidence, using a real PostgreSQL 18 container and the pool's own statistics, on Go 1.25.6.

Tested with Go 1.25.6, pgx v5.11.0, PostgreSQL 18.6 in Docker, macOS on Apple Silicon (arm64)

The pool is sql.DB. Its model, including what holds a connection, is in the database/sql explainer; this article is about the numbers.

The four settings

SettingDefaultWhat it controls
SetMaxOpenConns(n)0, unlimitedThe most connections open at once, in use or idle. Callers beyond it wait
SetMaxIdleConns(n)2How many idle connections the pool keeps. The rest are closed when returned
SetConnMaxLifetime(d)0, foreverA connection older than this is closed on its next use or return
SetConnMaxIdleTime(d)0, foreverA connection idle longer than this is closed

MaxIdleConns is silently lowered to MaxOpenConns if it is larger. With pgxpool the equivalents are MaxConns, MinConns, MaxConnLifetime and MaxConnIdleTime; pgxpool defaults to a maximum of four or the CPU count, whichever is larger.

Measuring MaxOpenConns

main.goGo
package main

import (
	"context"
	"database/sql"
	"fmt"
	"log"
	"sync"
	"time"

	_ "github.com/jackc/pgx/v5/stdlib"
)

const dsn = "postgres://postgres:[email protected]:55432/lab?sslmode=disable"

func run(label string, setup func(*sql.DB)) {
	db, err := sql.Open("pgx", dsn)
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()
	setup(db)
	ctx := context.Background()
	// Warm up so connection setup is not part of the measurement.
	var wg sync.WaitGroup
	start := time.Now()
	for range 20 {
		wg.Add(1)
		go func() {
			defer wg.Done()
			var n int
			_ = db.QueryRowContext(ctx, `select 1 from pg_sleep(0.1)`).Scan(&n)
		}()
	}
	wg.Wait()
	s := db.Stats()
	fmt.Printf("%-26s 20 queries of 100ms took %4dms  open=%2d waitCount=%2d closedMaxIdle=%2d\n",
		label, time.Since(start).Milliseconds()/10*10, s.OpenConnections, s.WaitCount, s.MaxIdleClosed)
}

func main() {
	run("MaxOpenConns=2", func(db *sql.DB) { db.SetMaxOpenConns(2) })
	run("MaxOpenConns=5", func(db *sql.DB) { db.SetMaxOpenConns(5) })
	run("MaxOpenConns=20", func(db *sql.DB) { db.SetMaxOpenConns(20) })
	run("MaxOpen=20 MaxIdle=2", func(db *sql.DB) { db.SetMaxOpenConns(20); db.SetMaxIdleConns(2) })
	run("unlimited (default)", func(db *sql.DB) {})

	db, _ := sql.Open("pgx", dsn)
	defer db.Close()
	db.SetMaxOpenConns(1)
	ctx, cancel := context.WithTimeout(context.Background(), 150*time.Millisecond)
	defer cancel()
	go func() { var n int; _ = db.QueryRow(`select 1 from pg_sleep(0.5)`).Scan(&n) }()
	time.Sleep(50 * time.Millisecond)
	var n int
	err := db.QueryRowContext(ctx, `select 1`).Scan(&n)
	fmt.Printf("waiting for a free connection with MaxOpenConns=1: %v\n", err)
}
Shell
go run . 2>&1
Text
MaxOpenConns=2             20 queries of 100ms took 1080ms  open= 2 waitCount=18 closedMaxIdle= 0
MaxOpenConns=5             20 queries of 100ms took  420ms  open= 2 waitCount=15 closedMaxIdle= 3
MaxOpenConns=20            20 queries of 100ms took  120ms  open= 2 waitCount= 0 closedMaxIdle=18
MaxOpen=20 MaxIdle=2       20 queries of 100ms took  120ms  open= 2 waitCount= 0 closedMaxIdle=18
unlimited (default)        20 queries of 100ms took  120ms  open= 2 waitCount= 0 closedMaxIdle=18
waiting for a free connection with MaxOpenConns=1: context deadline exceeded

The program runs the same workload against five pool configurations: twenty goroutines, each running a query that sleeps for 100 ms on the server, select 1 from pg_sleep(0.1). A pool can run at most MaxOpenConns of them at once, so the elapsed time shows how many were serialised.

  • MaxOpenConns=2: 1040 ms. Ten rounds of two queries at 100 ms each, plus overhead. WaitCount=18: eighteen of the twenty callers had to wait for a connection.
  • MaxOpenConns=5: 420 ms, four rounds, WaitCount=15.
  • MaxOpenConns=20: 120 ms, one round. WaitCount=0.
  • MaxOpenConns=20, MaxIdleConns=2: the same 130 ms.
  • Unlimited (the default): 130 ms, because twenty connections were opened at once.

The shape is queueing. Total time is the number of rounds times the query time, and rounds are the concurrency divided by the limit, rounded up. A smaller pool does not make the database faster; it makes callers wait in Go instead of at the server. That can be exactly what you want, and the next section says when.

How big should it be

More connections is not faster. A database server has a few cores and a disk, and past a certain number of concurrent queries, adding more only adds contention, context switching and memory use. Each PostgreSQL connection is a server process with its own memory, and the server's cores and disks are shared by all of them. Two arguments give you a number.

From the server. Take the server's max_connections, subtract what administrative tools and other services use, and divide the rest among the instances of your service. Ten instances sharing 80 usable connections get MaxOpenConns=8 each. This is the ceiling; going above it produces the errors in the too many connections article.

From the workload, with Little's law. The number of connections in use on average equals the arrival rate times the time each request holds a connection: L = λ × W. A service handling 400 requests per second, each holding a connection for 5 ms, uses 400 × 0.005 = 2 connections on average. Add headroom for bursts and slow requests, perhaps 4 to 10 times, and the pool needs 8 to 20, not 200. If the time a request holds a connection is long because it does other work in the middle of a transaction, fix that first.

A starting point that works for many services: MaxOpenConns between 10 and 30 per instance, MaxIdleConns equal to MaxOpenConns, and then adjust from the metrics below.

MaxIdleConns and the cost of closing

The default idle limit of two is small. The third run in the output shows what it does to a burst. After twenty queries finish on a 20-connection pool, only two connections are kept; closedMaxIdle=18 counts the other eighteen closed when they were returned. The next burst opens eighteen connections again, each a TCP connection, possibly TLS, an authentication round trip and a PostgreSQL backend process start. That is milliseconds to tens of milliseconds per connection, and it lands on the request that triggered the open.

Set MaxIdleConns to about the steady-state concurrency, up to MaxOpenConns. Combine it with ConnMaxIdleTime, say five minutes, so that after a burst the pool shrinks back instead of holding connections forever.

Lifetimes

ConnMaxLifetime closes connections that have been open a long time. It does not reset on use; it is the age of the connection. Reasons to set it:

  • Failover and DNS changes. A pool with immortal connections keeps talking to the old primary until the connection breaks. A lifetime of 5 to 30 minutes limits that.
  • Load balancers and proxies often drop connections after a fixed time. Set the lifetime below it to avoid using a connection that is about to be cut.
  • Server memory growth. Long-lived PostgreSQL backends accumulate cache. Recycling bounds it.
  • Credential rotation: old connections die before the old password is revoked.

The cost is a reconnect per connection per lifetime, which is small. Add jitter if you run many instances so they do not all recycle together; Go's pool already spreads closures by checking at use time, and pgxpool has MaxConnLifetimeJitter.

Waiting, and giving up

The last line of the output shows what a waiting caller experiences. The pool has MaxOpenConns=1, a goroutine holds the only connection for 500 ms, and a second caller passes a context with a 150 ms limit. It fails with context deadline exceeded without ever running its query. The error does not say that the wait was for a connection, so a service that sees these should check Stats().WaitDuration and WaitCount before blaming the query.

Always pass a context to pool calls. Without one, a caller waits indefinitely for a connection, and a stalled database makes every request goroutine pile up behind the pool. A request deadline turns that into a fast failure and lets the process shed load. See query timeouts.

Why a smaller pool can be faster overall

It sounds backwards, so here is the mechanism. When the database has four cores and you run eighty queries at once, the server time-slices them. Each query gets a twentieth of a core, every query takes twenty times longer, locks are held for longer, the buffer cache is contested and the total throughput does not improve; it often falls, because context switches cost. A pool of eight lets four queries run while four queue, and the queued ones wait in Go, where waiting is cheap. Each query finishes quickly, locks are released sooner and the same work completes in less total time.

The effect is strongest for CPU-bound queries and for workloads with row contention. For queries that mostly wait on I/O, such as a slow disk or a remote call from inside a function, a larger pool helps until the disk saturates. That is why the right size is measured, not guessed, and why a service that waits on other services inside transactions needs a different fix: shorten the transactions.

Reproducing the experiment on your service

The lab's structure transfers. To tune a real service, do this in staging.

  1. Export the Stats() fields as metrics, or log them once a second during the test.
  2. Run a load test at the traffic you expect, and note latency percentiles with the default pool.
  3. Set MaxOpenConns to a value from the sizing argument above, run again, and compare the 95th and 99th percentile and WaitCount.
  4. Halve and double it. The curve usually has a broad flat region; pick a value inside it, toward the lower end.
  5. Check the database: CPU, active connections and lock waits. If the database is not near saturation and callers are waiting, raise the limit. If the database is saturated, raising it will not help.

Write the numbers down with the date and instance sizes. Pool settings drift out of date when the service or the database changes.

Common mistakes

  • Leaving MaxOpenConns unlimited and relying on the server to refuse extra connections. The refusal is an error for users.
  • Setting MaxIdleConns above MaxOpenConns, which does nothing, or leaving it at two for a service with a concurrency of twenty.
  • A pool per request or per handler. Each sql.Open makes a new pool; connection counts explode and nothing is reused.
  • Holding a connection across slow work. An open transaction while calling an API holds a connection for the duration of the call.
  • Treating the pool as the rate limiter. A pool limits database concurrency, not request rate. Use a limiter in front, and a timeout behind it.
  • Never closing the pool at shutdown. db.Close() waits for in-use connections and releases the rest; call it after the HTTP server has drained, as in graceful shutdown tests.
  • Forgetting that every instance has its own pool. Twelve replicas at MaxOpenConns=20 are 240 connections to the server.

A note on pgxpool

Native pgx pools expose the same ideas with different names, and a few extras. MinConns keeps a floor of warm connections, HealthCheckPeriod checks idle connections in the background, and pool.Stat() includes AcquireDuration and EmptyAcquireCount, the number of times a caller found the pool empty and had to wait. If you use pgxpool, the same sizing arguments apply; the settings are in the PostgreSQL connection article.

What to watch

db.Stats() returns the counters. Export these as metrics:

FieldMeaningAlert on
InUse and IdleCurrent splitInUse pinned at MaxOpenConnections
WaitCountCalls that waited, cumulativeRate rising
WaitDurationTotal time spent waitingAverage per wait above a few ms
MaxIdleClosedClosed because the idle limit was hitHigh, which means reopening churn
MaxLifetimeClosedClosed for ageUnexpectedly high

A pool that is healthy has low and steady WaitCount. A rising WaitCount with a flat InUse at the limit means the pool is too small or something holds connections too long. A rising InUse that never falls means a leak: look for a Rows, Tx or Conn that is not closed, as in closing rows.

Settings for common situations

A web service with steady load. MaxOpenConns 20, MaxIdleConns 20, ConnMaxLifetime 30 minutes, ConnMaxIdleTime 5 minutes. Adjust after watching WaitCount.

A worker that runs batches. Few connections, a small MaxOpenConns that matches the number of workers, and no idle time worries.

Serverless or many short-lived instances. Each instance should hold one or two connections, because the fleet multiplies. Put a pooler such as PgBouncer or the provider's proxy between them and the database.

Behind a pooler in transaction mode. Keep MaxOpenConns modest, disable prepared statement caching in the driver, and do not use session state; see PostgreSQL connections.

SQLite. One writer connection and several readers, as described in the modernc article.

One more setting that is not on the pool

Two settings outside the pool decide how a stalled database looks from the application: the connect timeout and the per-query timeout. A pool with a limit of twenty and no query timeout can have all twenty connections stuck on slow queries, and then every new request waits behind them. A statement timeout on the server, set per role or per session, and a context deadline on the client together bound how long any one query can hold a connection. The pool then recovers by itself when the database comes back. Think of the pool limit, the context deadline and the statement timeout as one design: the limit controls concurrency, the deadline controls patience, and the server timeout is the backstop if the client gives up and the query keeps running.

Limits of the experiment

The lab's queries sleep on the server and use no CPU, which isolates the queueing effect. Real queries consume server CPU and disk, so the benefit of concurrency tops out where the server saturates, and a pool of 20 may be no faster than a pool of 10 against a 4-core database. The timings are from one laptop with a local container, and the point is the ratios: the time scales with the number of rounds. Test your own service with a load test and the pool metrics visible.

The package is documented in the database/sql reference, the PostgreSQL side in the connection settings chapter, and the Go team's managing connections guide covers the same knobs.

Tagged""
Share

Keep reading

Get new articles by email

We send one email when a new article is published. No spam.

Unsubscribe in one click. No spam, ever.