Write Clients for Your Go API With Retries and Timeouts

An API without a client library gets many clients, written quickly, each with its own idea of timeouts, retries and error handling. Some never time out. Some retry a payment POST on a connection reset and charge twice. Some read the response body into memory without a cap. Shipping an official client for the language your users write in is how you replace all of that with one tested piece of code.

This article designs a client library in Go for a simple orders API: constructor options, a typed error, retries with the right rules, pagination as an iterator and testing without waiting for real delays. Most of it applies to any HTTP client you write for a service you call often, including the internal ones. The design questions overlap with SDK design, and the retry rules are the client side of what timeouts and retries across services describes. If the client is generated from a spec, see generated OpenAPI clients. It ran on Go 1.25.6.

What a client should and should not do

It should hide the plumbing: building URLs, setting headers, encoding bodies, decoding responses and mapping errors. It should have safe defaults: a timeout, a user agent, a retry policy that cannot cause harm. And it should leave the caller in control of the things only the caller knows: the deadline of this particular call, whether to cancel and whether a failed create may be repeated.

It should not hide failure. A library that logs and swallows errors, or that retries forever, makes the caller's system impossible to reason about. It should not decide for the caller how long to wait: that comes from the context.Context argument on every method. And it should not be a framework. Keep it to a struct, a handful of methods and some options.

The client

Here is the whole library: a Client with functional options, one do method that all calls go through, an error type and a few typed operations.

package main

import (
	"bytes"
	"context"
	"encoding/json"
	"errors"
	"fmt"
	"io"
	"iter"
	"math/rand/v2"
	"net/http"
	"strconv"
	"time"
)

// APIError is what callers get for a non-2xx answer: enough to branch on and enough to report.
type APIError struct {
	Status    int
	Code      string
	Message   string
	RequestID string
}

func (e *APIError) Error() string {
	return fmt.Sprintf("api error: %d %s: %s (request %s)", e.Status, e.Code, e.Message, e.RequestID)
}

type Client struct {
	base       string
	http       *http.Client
	userAgent  string
	maxRetries int
	baseDelay  time.Duration
	maxDelay   time.Duration
	sleep      func(context.Context, time.Duration) error // replaceable so tests do not wait
	jitter     func(time.Duration) time.Duration
}

type Option func(*Client)

func WithHTTPClient(h *http.Client) Option { return func(c *Client) { c.http = h } }
func WithMaxRetries(n int) Option          { return func(c *Client) { c.maxRetries = n } }
func WithSleep(f func(context.Context, time.Duration) error) Option {
	return func(c *Client) { c.sleep = f }
}
func WithJitter(f func(time.Duration) time.Duration) Option { return func(c *Client) { c.jitter = f } }

func New(base string, opts ...Option) *Client {
	c := &Client{
		base:       base,
		http:       &http.Client{Timeout: 10 * time.Second}, // a backstop; per-call deadlines come from the context
		userAgent:  "orders-go/1.0",
		maxRetries: 3,
		baseDelay:  200 * time.Millisecond,
		maxDelay:   5 * time.Second,
		sleep: func(ctx context.Context, d time.Duration) error {
			t := time.NewTimer(d)
			defer t.Stop()
			select {
			case <-t.C:
				return nil
			case <-ctx.Done():
				return ctx.Err()
			}
		},
		// full jitter: a random delay between zero and the exponential ceiling
		jitter: func(d time.Duration) time.Duration { return time.Duration(rand.Int64N(int64(d) + 1)) },
	}
	for _, o := range opts {
		o(c)
	}
	return c
}

// retryable decides from the outcome. safe means repeating the request cannot do the work twice:
// the method is idempotent, or the caller sent an idempotency key.
func retryable(safe bool, status int, err error) bool {
	if err != nil { // the request did not complete: a reset, a refused connection, a timeout
		return safe && !errors.Is(err, context.Canceled) && !errors.Is(err, context.DeadlineExceeded)
	}
	switch status {
	case http.StatusTooManyRequests:
		return true // the server told us to slow down, and nothing was processed
	case http.StatusBadGateway, http.StatusServiceUnavailable, http.StatusGatewayTimeout, http.StatusInternalServerError:
		return safe
	}
	return false
}

// do sends one logical request, with retries. A POST is retried only when the caller supplies an idempotency key.
func (c *Client) do(ctx context.Context, method, path string, in, out any, idemKey string) error {
	var payload []byte
	if in != nil {
		var err error
		if payload, err = json.Marshal(in); err != nil {
			return err
		}
	}
	for attempt := 0; ; attempt++ {
		req, rerr := http.NewRequestWithContext(ctx, method, c.base+path, bytes.NewReader(payload))
		if rerr != nil {
			return rerr
		}
		var err error
		req.Header.Set("User-Agent", c.userAgent)
		req.Header.Set("Accept", "application/json")
		if in != nil {
			req.Header.Set("Content-Type", "application/json")
		}
		if idemKey != "" {
			req.Header.Set("Idempotency-Key", idemKey)
		}

		resp, terr := c.http.Do(req)
		err = terr
		status := 0
		if terr == nil {
			status = resp.StatusCode
			body, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
			resp.Body.Close()
			if status >= 200 && status < 300 {
				if out == nil {
					return nil
				}
				return json.Unmarshal(body, out)
			}
			e := &APIError{Status: status, RequestID: resp.Header.Get("X-Request-Id")}
			var parsed struct{ Code, Message string }
			json.Unmarshal(body, &parsed)
			e.Code, e.Message = parsed.Code, parsed.Message
			err = e
		}

		safe := method == http.MethodGet || method == http.MethodHead || method == http.MethodPut || method == http.MethodDelete || idemKey != ""
		canRetry := retryable(safe, status, terr)
		if !canRetry || attempt >= c.maxRetries {
			return err
		}
		delay := c.delay(attempt, resp)
		if serr := c.sleep(ctx, delay); serr != nil {
			return errors.Join(err, serr)
		}
	}
}

// delay honours Retry-After when present and otherwise backs off exponentially with jitter.
func (c *Client) delay(attempt int, resp *http.Response) time.Duration {
	if resp != nil {
		if ra := resp.Header.Get("Retry-After"); ra != "" {
			if secs, err := strconv.Atoi(ra); err == nil {
				return min(time.Duration(secs)*time.Second, c.maxDelay)
			}
		}
	}
	ceiling := min(c.baseDelay<<attempt, c.maxDelay)
	return c.jitter(ceiling)
}

type Order struct {
	ID     string `json:"id"`
	Status string `json:"status"`
}

func (c *Client) GetOrder(ctx context.Context, id string) (*Order, error) {
	var o Order
	if err := c.do(ctx, http.MethodGet, "/orders/"+id, nil, &o, ""); err != nil {
		return nil, err
	}
	return &o, nil
}

func (c *Client) CreateOrder(ctx context.Context, item, idemKey string) (*Order, error) {
	var o Order
	if err := c.do(ctx, http.MethodPost, "/orders", map[string]string{"item": item}, &o, idemKey); err != nil {
		return nil, err
	}
	return &o, nil
}

// Orders returns an iterator over every page, so callers write a plain range loop.
func (c *Client) Orders(ctx context.Context) iter.Seq2[Order, error] {
	return func(yield func(Order, error) bool) {
		cursor := ""
		for {
			var page struct {
				Data []Order `json:"data"`
				Next string  `json:"next"`
			}
			if err := c.do(ctx, http.MethodGet, "/orders?cursor="+cursor, nil, &page, ""); err != nil {
				yield(Order{}, err)
				return
			}
			for _, o := range page.Data {
				if !yield(o, nil) {
					return
				}
			}
			if page.Next == "" {
				return
			}
			cursor = page.Next
		}
	}
}

The pieces, in the order a request meets them.

Construction. New(base, opts...) takes a base URL and functional options (the options pattern). The defaults are chosen to be safe: a 10 second client timeout as a backstop (the real deadline comes from the context), three retries, a 200 ms base delay and a 5 second cap. The hooks WithSleep and WithJitter exist for tests, and are also the way a user can plug in their own backoff.

One request path. All operations call do, so retries, headers and error mapping exist once. The request body is marshalled once and re-wrapped for each attempt, because a request body can only be read once.

Headers. User-Agent identifies the library and version, which is how you learn from server logs who is still on an old release. Accept declares JSON. Idempotency-Key is set when the caller supplies one.

Bounded reads. The response body is read through io.LimitReader at 1 MiB. A server bug or an attacker on the path cannot make the client allocate without limit, and the body is always closed so the connection returns to the pool.

A typed error. Any non-2xx response becomes an *APIError with the HTTP status, the machine code, the message and the X-Request-Id. Callers use errors.As to branch on status or code, and support can find the server-side log line from the request id in a bug report.

The retry rules

This is the part that is easy to get dangerously wrong. The rule that matters is never repeat a request that might have done the work, unless repeating is safe. The code encodes it in one function, retryable.

• Safe means idempotent or keyed. GET, HEAD, PUT and DELETE are idempotent by definition in HTTP. A POST becomes safe when the caller provides an Idempotency-Key, because the server will return the first result for a repeated key (idempotency keys).
• Connection-level failures (reset, refused, a timeout waiting for headers) are retried only when safe, because the request may or may not have been processed. Cancellation and deadline errors from the caller's context are never retried: the caller has decided to stop.
• 429 Too Many Requests is retried for any method. The server rejected the request before processing it, and told us to come back later.
• 500, 502, 503 and 504 are retried only when safe.
• Everything else, including 400, 401, 403, 404, 409 and 422, is final. Retrying a request that was wrong makes it wrong again.

The delay comes from Retry-After when the server sent one (capped at the client's maximum), and otherwise from exponential backoff with full jitter: the ceiling doubles with each attempt (100 ms, 200 ms, 400 ms...) up to the cap, and the actual wait is a random value below it. Jitter matters because a thousand clients that failed together and wait the same fixed time will retry together. The sleep is interruptible by the context, so a cancelled call does not sit out its backoff.

Running it

The program starts scripted servers that answer with a given sequence of statuses, records the sleeps instead of waiting and fixes the jitter at half the ceiling so the output is exact.

package main

import (
	"context"
	"errors"
	"fmt"
	"net/http"
	"net/http/httptest"
	"sync/atomic"
	"time"
)

func main() {
	var hits atomic.Int64
	script := func(statuses ...int) http.Handler {
		var i atomic.Int64
		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			hits.Add(1)
			n := int(i.Add(1)) - 1
			status := statuses[len(statuses)-1]
			if n < len(statuses) {
				status = statuses[n]
			}
			w.Header().Set("X-Request-Id", fmt.Sprintf("req-%d", hits.Load()))
			switch status {
			case 429:
				w.Header().Set("Retry-After", "2")
				w.WriteHeader(429)
			case 200, 201:
				w.WriteHeader(status)
				fmt.Fprint(w, `{"id":"ord_1","status":"paid"}`)
			default:
				w.WriteHeader(status)
				fmt.Fprintf(w, `{"code":"upstream_%d","message":"try later"}`, status)
			}
		})
	}

	// The sleeps are recorded instead of waited, and the jitter is fixed at half the ceiling, so the output is exact.
	var slept []time.Duration
	opts := []Option{
		WithSleep(func(_ context.Context, d time.Duration) error { slept = append(slept, d); return nil }),
		WithJitter(func(d time.Duration) time.Duration { return d / 2 }),
	}
	run := func(label string, h http.Handler, call func(*Client) (*Order, error)) {
		hits.Store(0)
		slept = nil
		srv := httptest.NewServer(h)
		defer srv.Close()
		o, err := call(New(srv.URL, opts...))
		fmt.Printf("%-34s attempts=%d sleeps=%v ", label, hits.Load(), slept)
		var apiErr *APIError
		switch {
		case errors.As(err, &apiErr):
			fmt.Printf("-> APIError status=%d code=%s request=%s\n", apiErr.Status, apiErr.Code, apiErr.RequestID)
		case err != nil:
			fmt.Printf("-> %v\n", err)
		default:
			fmt.Printf("-> %+v\n", *o)
		}
	}
	get := func(c *Client) (*Order, error) { return c.GetOrder(context.Background(), "ord_1") }
	post := func(key string) func(*Client) (*Order, error) {
		return func(c *Client) (*Order, error) { return c.CreateOrder(context.Background(), "book", key) }
	}

	run("GET, 503 twice then success", script(503, 503, 200), get)
	run("GET, 429 with Retry-After: 2", script(429, 200), get)
	run("GET, 404 is final", script(404), get)
	run("GET, always 503 (gives up)", script(503), get)
	run("POST without a key, 503", script(503, 200), post(""))
	run("POST with a key, 503 then success", script(503, 200), post("key-123"))
	run("POST, 429 is safe to retry", script(429, 201), post(""))

	// Pagination as an iterator.
	page := 0
	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		page++
		switch r.URL.Query().Get("cursor") {
		case "":
			fmt.Fprint(w, `{"data":[{"id":"a","status":"paid"},{"id":"b","status":"paid"}],"next":"p2"}`)
		default:
			fmt.Fprint(w, `{"data":[{"id":"c","status":"shipped"}],"next":""}`)
		}
	}))
	defer srv.Close()
	n := 0
	for o, err := range New(srv.URL).Orders(context.Background()) {
		if err != nil {
			panic(err)
		}
		n++
		fmt.Print(o.ID, " ")
	}
	fmt.Printf("\nlisted %d orders over %d requests\n", n, page)
}

GET, 503 twice then success        attempts=3 sleeps=[100ms 200ms] -> {ID:ord_1 Status:paid}
GET, 429 with Retry-After: 2       attempts=2 sleeps=[2s] -> {ID:ord_1 Status:paid}
GET, 404 is final                  attempts=1 sleeps=[] -> APIError status=404 code=upstream_404 request=req-1
GET, always 503 (gives up)         attempts=4 sleeps=[100ms 200ms 400ms] -> APIError status=503 code=upstream_503 request=req-4
POST without a key, 503            attempts=1 sleeps=[] -> APIError status=503 code=upstream_503 request=req-1
POST with a key, 503 then success  attempts=2 sleeps=[100ms] -> {ID:ord_1 Status:paid}
POST, 429 is safe to retry         attempts=2 sleeps=[2s] -> {ID:ord_1 Status:paid}
a b c 
listed 3 orders over 2 requests

The first line is the baseline retry: two 503s and then a 200 means three attempts, with sleeps of 100 ms and 200 ms (half of the 200 ms and 400 ms ceilings). The second shows the Retry-After: 2 header overriding the backoff: one sleep of exactly 2 seconds. The 404 is final after one attempt. A server that never recovers produces four attempts (the first plus three retries), three sleeps that double, and an APIError for the last 503 with its request id, req-4, which identifies the final attempt.

The POST lines show the idempotency rules. Without a key, a 503 ends the call after one attempt. With a key, the same failure is retried and succeeds on the second try. A 429 is retried even without a key.

A bug that the table caught

My first version retried nothing. Every row of that output showed one attempt. The cause was a classification mistake: the code turned a non-2xx response into an *APIError first, then asked retryable whether the error was retryable, and the function rejected any error that was an APIError, because it was written to treat "an error response from the server" as final. The status code, which is what should drive the decision, was no longer in the right place.

The fix was to separate the transport error (what http.Client.Do returned) from the HTTP outcome (the status), classify using those two, and create the APIError only for the caller. A table of scripted statuses is the cheapest way to prevent this class of bug, and it takes minutes to write.

Pagination with iterators

Listing endpoints return pages, and callers want to say "for each order". Since Go 1.23, a function can return an iter.Seq2[Order, error] that works in a for range loop. The Orders method fetches a page, yields each item, follows the cursor and stops when there is none, or when the loop body breaks (the yield function returns false). A request error is yielded as a value and ends the iteration. The caller's code is a plain loop:

for order, err := range client.Orders(ctx) {
	if err != nil {
		return err
	}
	process(order)
}

The last line of the lab output shows it listing three orders in two requests. The caller never sees cursors. This composes with the context for cancellation and with early exit, and it keeps memory flat for long lists. See range over functions for how iterators work.

The method surface

Deciding what the methods look like matters more than the plumbing, because that surface is what you will be stuck with.

One method per operation, named after the operation. GetOrder, CreateOrder, CancelOrder. Resist a generic Do(method, path, body) as the public API: it moves all the knowledge of paths and shapes onto the caller and makes every typo a runtime error.

Context first, then required arguments, then an options struct or variadic options for the rest. CreateOrder(ctx, item, opts ...CreateOption) leaves room for new optional parameters without breaking callers, which a long list of positional arguments does not.

Return domain types, not raw responses. Callers want an *Order. If they sometimes need headers (rate limit remaining, request id), expose them through a small metadata value returned alongside, or through the error for failures, and not by returning *http.Response.

Make the dangerous thing explicit. The idempotency key is a required argument of the create method, not an option hidden in a config struct. Forcing the caller to think about it at the call site is the strongest nudge towards doing it right. If you want a convenience, provide CreateOrderOnce, which generates a key and documents that retries inside it are safe but a repeat call is a new order.

Do not leak your wire types. The library's Order struct is a promise to callers. If the wire format has fields they should not touch, or names that will change, convert at the boundary.

Errors callers actually need

Most callers want to ask four questions of a failure, and the library should make each easy:

1. Is it the caller's fault (a 4xx) or the server's (a 5xx or a network failure)?
2. Is it worth retrying later, at a higher level than the library's own retries?
3. Which specific condition is it, so I can handle "insufficient funds" differently from "invalid card"?
4. What do I tell support?

The APIError answers the first with its status, the third with its code and the fourth with the request id. For the second and for common conditions, add helper functions such as IsNotFound(err), IsRateLimited(err) and IsRetryable(err) built on errors.As, so callers do not repeat the status-code comparisons. Define exported sentinel codes as constants (CodeInsufficientFunds) so switch statements read well and typos are compile errors.

Wrap transport errors with context but keep them unwrappable: fmt.Errorf("get order %s: %w", id, err) lets callers use errors.Is(err, context.DeadlineExceeded) and errors.As for net.Error to distinguish a timeout from a refused connection. Never turn a typed error into a string. The Go blog's discussion of working with errors covers the mechanics.

Documentation and release

A client library lives or dies on its examples. Put a runnable Example function in the package for each main flow (create with an idempotency key, list with the iterator, handle a rate limit), so go test checks that they compile and pkg.go.dev renders them. Keep a short README with installation, the one-screen quick start and the retry policy stated in plain terms: which methods retry, how many times, what the delays are, and how to turn it off.

Release with semantic versions and a changelog. A breaking change to the library is a major version with the new import path (/v2), as the Go module system expects. Keep the user agent in sync with the module version using build info, so your server logs show how many callers are on each release and you know when it is safe to drop support for an old API behaviour.

Finally, give users a way to see what the client is doing. A Logger option that takes a *slog.Logger and logs each retry at debug level with the attempt number, status and delay answers most "why was my call slow?" tickets without a debugger. Make the default silent, and take care that the logger never receives request bodies or authorisation headers (logging without leaking secrets).

Concerns beyond the basics

Connection reuse. Keep one http.Client per target and share it. Always read the response body to the end or close it so the connection returns to the pool. A client that creates a new http.Client per call loses keep-alive and leaks sockets.

Timeouts. The 10 second backstop catches a hung server when the caller forgot to set a deadline. For most calls the context deadline is the real control, and the client should document it as the first argument of every method. See the client timeout article.

Observability hooks. Let users pass their own http.Client, with a RoundTripper wrapping the default transport, to add metrics, tracing (OpenTelemetry) and logging without the library depending on any of them.

Authentication. Accept a token source (a function that returns the current token) instead of a fixed string, so refresh can happen without rebuilding the client.

Concurrency. The client is safe for concurrent use because it holds only immutable configuration and an http.Client. Say so in the docs.

Versioning and compatibility. Tag releases with semantic versions, send the version in the user agent, and keep the public surface small so you can change what is behind it (backward compatibility).

Testing. Test the retry logic with scripted servers as above, test error mapping with canned bodies, and run a contract test against a staging server in CI to catch drift between the library and the API (contract testing).

The standard library reference for what the client builds on is net/http, and RFC 9110 defines the semantics of idempotent methods and Retry-After.