Call Web APIs From Go With Typed Clients and Context
Build a typed Go client for a JSON API: context on every call, one generic request helper, an APIError you can test with errors.Is, and paging as an iterator.
Sooner or later every Go service calls somebody else's API, and the first version is always an http.Get with a hand-built URL, a json.NewDecoder and an if err != nil that checks the wrong thing. The second version is the same code in twelve places. The fix is not a framework but a small, boring client type: one struct per remote API, one place that knows how to talk HTTP, and methods whose names are the API's operations.
Tested with Go 1.25.6, macOS on Apple Silicon (arm64)
This project builds one against a fake bookmarks API, and tests it the way you would test a real one. It uses the client defaults from the HTTP client article, the retry wrapper from retries and backoff can sit underneath it unchanged, and the shape is the consumer-side mirror of the server in a production HTTP server. Everything ran on Go 1.25.6 with the standard library only.
What a good client looks like
Five properties, in order of how often their absence hurts:
- Context first. Every method takes
ctx context.Contextas its first argument and puts it on the request. That gives callers deadlines and cancellation, and it lets a handler's disconnect cancel the downstream call. - The
http.Clientis injected. The constructor takes an option to supply one, defaulting to a client with a timeout. Production passes a transport tuned as in transport settings; tests pass thehttptestserver's client. - One request helper. URL building, auth headers, JSON encoding, status checking, error decoding and body limits are written once. If the API changes how it signals errors, one function changes.
- Errors are values. A non-2xx response becomes a typed error that carries what the API said, and callers match on meaning (
errors.Is(err, ErrNotFound)), not on strings or on raw status codes. - Pagination is hidden. A method that lists things returns something you can range over, and the client fetches the next page when the loop asks for it.
The client
package main
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"iter"
"net/http"
"net/url"
"strconv"
"time"
)
type Bookmark struct {
ID int `json:"id"`
URL string `json:"url"`
Title string `json:"title"`
}
type NewBookmark struct {
URL string `json:"url"`
Title string `json:"title"`
}
type page struct {
Items []Bookmark `json:"items"`
NextCursor string `json:"next_cursor"`
}
// ErrNotFound matches any APIError with status 404.
var ErrNotFound = errors.New("bookmarks: not found")
// APIError is what the server said when it refused a request.
type APIError struct {
Status int
Code string
Message string
RequestID string
}
func (e *APIError) Error() string {
return fmt.Sprintf("bookmarks API: %d %s: %s (request %s)", e.Status, e.Code, e.Message, e.RequestID)
}
func (e *APIError) Is(target error) bool {
return target == ErrNotFound && e.Status == http.StatusNotFound
}
type Client struct {
base *url.URL
hc *http.Client
token string
}
type Option func(*Client)
func WithHTTPClient(hc *http.Client) Option { return func(c *Client) { c.hc = hc } }
func WithToken(t string) Option { return func(c *Client) { c.token = t } }
func New(baseURL string, opts ...Option) (*Client, error) {
u, err := url.Parse(baseURL)
if err != nil || u.Scheme == "" || u.Host == "" {
return nil, fmt.Errorf("bookmarks: invalid base URL %q", baseURL)
}
c := &Client{base: u, hc: &http.Client{Timeout: 10 * time.Second}}
for _, o := range opts {
o(c)
}
return c, nil
}
const maxResponse = 4 << 20
// do is the only function that touches net/http. T is the type of the successful response body.
func do[T any](ctx context.Context, c *Client, method, path string, query url.Values, in any) (T, error) {
var zero T
var body io.Reader
if in != nil {
b, err := json.Marshal(in)
if err != nil {
return zero, fmt.Errorf("bookmarks: encoding request: %w", err)
}
body = bytes.NewReader(b)
}
u := c.base.JoinPath(path)
u.RawQuery = query.Encode()
req, err := http.NewRequestWithContext(ctx, method, u.String(), body)
if err != nil {
return zero, err
}
req.Header.Set("Accept", "application/json")
req.Header.Set("User-Agent", "gohacks-bookmarks-client/1.0")
if in != nil {
req.Header.Set("Content-Type", "application/json")
}
if c.token != "" {
req.Header.Set("Authorization", "Bearer "+c.token)
}
res, err := c.hc.Do(req)
if err != nil {
return zero, fmt.Errorf("bookmarks: %s %s: %w", method, path, err)
}
defer res.Body.Close()
raw, err := io.ReadAll(io.LimitReader(res.Body, maxResponse))
if err != nil {
return zero, fmt.Errorf("bookmarks: reading response: %w", err)
}
if res.StatusCode < 200 || res.StatusCode > 299 {
return zero, decodeError(res, raw)
}
if res.StatusCode == http.StatusNoContent || len(raw) == 0 {
return zero, nil
}
if err := json.Unmarshal(raw, &zero); err != nil {
return zero, fmt.Errorf("bookmarks: decoding response: %w", err)
}
return zero, nil
}
// decodeError never fails: a gateway's HTML page still yields a useful APIError.
func decodeError(res *http.Response, raw []byte) error {
e := &APIError{Status: res.StatusCode, Code: http.StatusText(res.StatusCode), RequestID: res.Header.Get("X-Request-Id")}
var env struct {
Error struct {
Code string `json:"code"`
Message string `json:"message"`
} `json:"error"`
}
if json.Unmarshal(raw, &env) == nil && env.Error.Code != "" {
e.Code, e.Message = env.Error.Code, env.Error.Message
} else {
if len(raw) > 120 {
raw = raw[:120]
}
e.Message = string(bytes.TrimSpace(raw))
}
return e
}
func (c *Client) Get(ctx context.Context, id int) (Bookmark, error) {
return do[Bookmark](ctx, c, http.MethodGet, "bookmarks/"+strconv.Itoa(id), nil, nil)
}
func (c *Client) Create(ctx context.Context, in NewBookmark) (Bookmark, error) {
return do[Bookmark](ctx, c, http.MethodPost, "bookmarks", nil, in)
}
func (c *Client) Delete(ctx context.Context, id int) error {
_, err := do[struct{}](ctx, c, http.MethodDelete, "bookmarks/"+strconv.Itoa(id), nil, nil)
return err
}
// List returns every bookmark, fetching pages of the given size as the caller iterates.
func (c *Client) List(ctx context.Context, pageSize int) iter.Seq2[Bookmark, error] {
return func(yield func(Bookmark, error) bool) {
cursor := ""
for {
q := url.Values{"limit": {strconv.Itoa(pageSize)}}
if cursor != "" {
q.Set("cursor", cursor)
}
p, err := do[page](ctx, c, http.MethodGet, "bookmarks", q, nil)
if err != nil {
yield(Bookmark{}, err)
return
}
for _, b := range p.Items {
if !yield(b, nil) {
return // the caller broke out of the loop; do not fetch more
}
}
if p.NextCursor == "" {
return
}
cursor = p.NextCursor
}
}
}Most of the file is the do function, which is the point. It is generic in the response type, so do[Bookmark] decodes into a Bookmark and do[struct{}] is how a DELETE with no body says it expects nothing. Go does not allow type parameters on methods, which is why do is a function taking the client, and the public methods stay ordinary. The body is read through io.LimitReader, so a misbehaving server cannot make the client allocate without limit.
Two parts of decodeError are worth a second look. It never returns a decoding error of its own: if the body is not the expected JSON shape, say a load balancer's HTML page for a 502, the APIError carries the first 120 bytes as its message. A client whose error path can itself fail with invalid character '<' looking for beginning of value turns a clear outage into a confusing one. And the Is method lets any 404 match ErrNotFound, so callers need no knowledge of status codes.
A fake server and a tour of the behavior
The test double is a real HTTP server, built with httptest, that keeps seven bookmarks in memory, checks a bearer token, pages by cursor and returns the same JSON error shape the client expects.
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"net/http/httptest"
"strconv"
"sync"
"sync/atomic"
"time"
)
type fakeAPI struct {
mu sync.Mutex
items []Bookmark
nextID int
requests atomic.Int64
}
func (f *fakeAPI) fail(w http.ResponseWriter, status int, code, msg string) {
w.Header().Set("X-Request-Id", "req-"+strconv.Itoa(int(f.requests.Load())))
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status)
json.NewEncoder(w).Encode(map[string]any{"error": map[string]string{"code": code, "message": msg}})
}
func (f *fakeAPI) ServeHTTP(w http.ResponseWriter, r *http.Request) {
f.requests.Add(1)
if r.Header.Get("Authorization") != "Bearer good-token" {
f.fail(w, http.StatusUnauthorized, "unauthorized", "missing or invalid token")
return
}
f.mu.Lock()
defer f.mu.Unlock()
switch {
case r.Method == "GET" && r.URL.Path == "/v1/bookmarks":
limit, _ := strconv.Atoi(r.URL.Query().Get("limit"))
start, _ := strconv.Atoi(r.URL.Query().Get("cursor"))
end := min(start+limit, len(f.items))
out := map[string]any{"items": f.items[start:end]}
if end < len(f.items) {
out["next_cursor"] = strconv.Itoa(end)
}
json.NewEncoder(w).Encode(out)
case r.Method == "POST" && r.URL.Path == "/v1/bookmarks":
var in NewBookmark
if json.NewDecoder(r.Body).Decode(&in) != nil || in.URL == "" {
f.fail(w, http.StatusUnprocessableEntity, "invalid_bookmark", "url is required")
return
}
f.nextID++
b := Bookmark{ID: f.nextID, URL: in.URL, Title: in.Title}
f.items = append(f.items, b)
w.WriteHeader(http.StatusCreated)
json.NewEncoder(w).Encode(b)
case r.Method == "GET" || r.Method == "DELETE":
id, _ := strconv.Atoi(r.URL.Path[len("/v1/bookmarks/"):])
for i, b := range f.items {
if b.ID == id {
if r.Method == "DELETE" {
f.items = append(f.items[:i], f.items[i+1:]...)
w.WriteHeader(http.StatusNoContent)
return
}
json.NewEncoder(w).Encode(b)
return
}
}
f.fail(w, http.StatusNotFound, "not_found", fmt.Sprintf("bookmark %d does not exist", id))
default:
http.Error(w, "<html>502 Bad Gateway from the load balancer</html>", http.StatusBadGateway)
}
}
func main() {
api := &fakeAPI{}
for i := 1; i <= 7; i++ {
api.nextID++
api.items = append(api.items, Bookmark{ID: i, URL: fmt.Sprintf("https://example.test/%d", i), Title: fmt.Sprintf("Page %d", i)})
}
srv := httptest.NewServer(api)
defer srv.Close()
c, _ := New(srv.URL+"/v1", WithToken("good-token"), WithHTTPClient(srv.Client()))
ctx := context.Background()
b, err := c.Get(ctx, 3)
fmt.Printf("Get(3): %+v err=%v\n", b, err)
_, err = c.Get(ctx, 99)
var apiErr *APIError
fmt.Printf("Get(99): errors.Is(ErrNotFound)=%v errors.As=%v\n %v\n", errors.Is(err, ErrNotFound), errors.As(err, &apiErr), err)
created, err := c.Create(ctx, NewBookmark{URL: "https://go.dev/blog", Title: "The Go Blog"})
fmt.Printf("Create: %+v err=%v\n", created, err)
_, err = c.Create(ctx, NewBookmark{Title: "no url"})
fmt.Printf("Create invalid: %v\n", err)
api.requests.Store(0)
n := 0
for b, err := range c.List(ctx, 3) {
if err != nil {
fmt.Println("list error:", err)
break
}
n++
_ = b
}
fmt.Printf("List(pageSize 3): %d bookmarks in %d requests\n", n, api.requests.Load())
api.requests.Store(0)
for b, err := range c.List(ctx, 3) {
if err != nil || b.ID == 2 {
break // stop early
}
}
fmt.Printf("List stopped at the second item: %d request(s) made\n", api.requests.Load())
fmt.Printf("Delete(1): err=%v\n", c.Delete(ctx, 1))
bad, _ := New(srv.URL+"/v1", WithToken("wrong"), WithHTTPClient(srv.Client()))
_, err = bad.Get(ctx, 1)
fmt.Printf("wrong token: %v\n", err)
gw, _ := New(srv.URL+"/v1", WithToken("good-token"), WithHTTPClient(srv.Client()))
_, err = do[Bookmark](ctx, gw, "PATCH", "bookmarks/1", nil, nil)
fmt.Printf("HTML error page: %v\n", err)
short, cancel := context.WithTimeout(ctx, time.Nanosecond)
defer cancel()
time.Sleep(time.Millisecond)
_, err = c.Get(short, 2)
fmt.Printf("expired context: errors.Is(DeadlineExceeded)=%v\n", errors.Is(err, context.DeadlineExceeded))
}Get(3): {ID:3 URL:https://example.test/3 Title:Page 3} err=<nil>
Get(99): errors.Is(ErrNotFound)=true errors.As=true
bookmarks API: 404 not_found: bookmark 99 does not exist (request req-2)
Create: {ID:8 URL:https://go.dev/blog Title:The Go Blog} err=<nil>
Create invalid: bookmarks API: 422 invalid_bookmark: url is required (request req-4)
List(pageSize 3): 8 bookmarks in 3 requests
List stopped at the second item: 1 request(s) made
Delete(1): err=<nil>
wrong token: bookmarks API: 401 unauthorized: missing or invalid token (request req-3)
HTML error page: bookmarks API: 502 Bad Gateway: <html>502 Bad Gateway from the load balancer</html> (request )
expired context: errors.Is(DeadlineExceeded)=trueRead the output as a list of guarantees the client makes. Get on a missing ID produces an error that satisfies both errors.Is(err, ErrNotFound) and errors.As into *APIError, and the printed message already contains the request ID the server sent, which is the single most useful string when you phone a vendor's support. Listing the eight bookmarks (the original seven plus the one just created) with a page size of three took three requests, and breaking out of the loop at the second item made only one request, because the iterator checks yield's return value before fetching more: laziness is part of the contract, and you can see it in the request counter. The gateway's HTML body surfaced as a readable APIError, with status 502 and the text of the page, and an expired context was reported as context.DeadlineExceeded, since %w kept the chain intact through the wrapping in do.
Decisions I would defend
Return the library's types, not map[string]any. The cost is declaring the structs. The benefit is that a field rename in the API breaks compilation or a test in one file, instead of producing a silent zero value in production. Generate the structs when the API publishes a spec, as in generating a client from OpenAPI, and keep a hand-written client when the API is small or the spec is unreliable.
Do not retry inside every method. Keep retries in a RoundTripper under the client, where the policy applies to every call and can see idempotency. The client above is built so that wrapping its http.Client's transport needs no change here.
Do not hide the HTTP client behind an interface you invent. Tests should use httptest with the real client, because the wire behavior (status codes, headers, paging) is what can break. Define an interface at the point where your code consumes the client, with only the methods that consumer needs; the type above will satisfy it without knowing.
Version the user agent and the client. The User-Agent header with a version is what lets the API's owner find your traffic in their logs, and what lets you find which of your services still runs the old client.
Unit tests for the parts that can break quietly
The tour above exercises the happy and sad paths against a server. Some behavior is easier to pin with small tests that need no server at all: how decodeError treats odd bodies, what JoinPath does with a base URL that has a prefix, and which headers a request carries. These run in milliseconds and are the ones that stop a refactor from changing the wire format by accident.
package main
import (
"context"
"errors"
"io"
"net/http"
"strings"
"testing"
)
type fakeTransport func(*http.Request) (*http.Response, error)
func (f fakeTransport) RoundTrip(r *http.Request) (*http.Response, error) { return f(r) }
func respond(status int, body string) *http.Response {
return &http.Response{StatusCode: status, Body: io.NopCloser(strings.NewReader(body)), Header: http.Header{}}
}
func TestDecodeError(t *testing.T) {
cases := []struct {
name, body string
status int
code, msg string
}{
{"json envelope", `{"error":{"code":"quota","message":"slow down"}}`, 429, "quota", "slow down"},
{"html page", "<html>bad gateway</html>", 502, "Bad Gateway", "<html>bad gateway</html>"},
{"empty body", "", 503, "Service Unavailable", ""},
{"json without our shape", `{"detail":"nope"}`, 400, "Bad Request", `{"detail":"nope"}`},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
err := decodeError(respond(tc.status, tc.body), []byte(tc.body))
var api *APIError
if !errors.As(err, &api) {
t.Fatalf("not an APIError: %v", err)
}
if api.Status != tc.status || api.Code != tc.code || api.Message != tc.msg {
t.Errorf("got %+v", api)
}
})
}
}
func TestRequestShape(t *testing.T) {
var got *http.Request
hc := &http.Client{Transport: fakeTransport(func(r *http.Request) (*http.Response, error) {
got = r
return respond(200, `{"id":5,"url":"u","title":"t"}`), nil
})}
c, err := New("https://api.example.test/v1/", WithToken("tok"), WithHTTPClient(hc))
if err != nil {
t.Fatal(err)
}
if _, err := c.Create(context.Background(), NewBookmark{URL: "u", Title: "t"}); err != nil {
t.Fatal(err)
}
checks := map[string]string{
"Authorization": "Bearer tok",
"Content-Type": "application/json",
"Accept": "application/json",
}
for k, want := range checks {
if v := got.Header.Get(k); v != want {
t.Errorf("%s = %q, want %q", k, v, want)
}
}
if got.URL.String() != "https://api.example.test/v1/bookmarks" || got.Method != "POST" {
t.Errorf("request went to %s %s", got.Method, got.URL)
}
}
func TestNewRejectsBadURL(t *testing.T) {
if _, err := New("not a url"); err == nil {
t.Error("expected an error for a base URL with no scheme or host")
}
}=== RUN TestDecodeError
=== RUN TestDecodeError/json_envelope
=== RUN TestDecodeError/html_page
=== RUN TestDecodeError/empty_body
=== RUN TestDecodeError/json_without_our_shape
--- PASS: TestDecodeError (0.00s)
--- PASS: TestDecodeError/json_envelope (0.00s)
--- PASS: TestDecodeError/html_page (0.00s)
--- PASS: TestDecodeError/empty_body (0.00s)
--- PASS: TestDecodeError/json_without_our_shape (0.00s)
=== RUN TestRequestShape
--- PASS: TestRequestShape (0.00s)
=== RUN TestNewRejectsBadURL
--- PASS: TestNewRejectsBadURL (0.00s)
PASS
ok example.test/lab 1.306sTable tests with named subtests make a failure point at the exact case, and the fake transport is four lines. The request-shape test is the one I value most: it records the request that would have gone out and compares headers and URL, so a change to User-Agent or a double slash in the path (a classic JoinPath surprise when the base ends in /v1/) fails immediately. The same technique is used in testing HTTP handlers without a server from the other side.
Auth belongs in the transport
The client above adds the bearer token in do, which is fine for a static key. When tokens expire, the logic grows: refresh before expiry, refresh once on a 401, and do not stampede the token endpoint when ten requests notice at the same time. That is transport behavior, not API behavior, so put it in a RoundTripper and leave do alone. The official golang.org/x/oauth2 package does this for OAuth2 flows: its oauth2.NewClient returns an *http.Client whose transport attaches and refreshes tokens. Pass that client through WithHTTPClient and the bookmarks client needs no auth code at all.
For a custom scheme, such as a signed header, the pattern is the same: a type with a RoundTrip method that clones the request, adds the header, and calls the next transport. Cloning matters because a RoundTripper must not mutate the caller's request. Keep secrets out of the client struct's String or %+v output, so a log line that prints the client cannot leak the token; a small token type with a String() that returns "[redacted]" is enough.
Be a tolerant reader
One setting that looks like diligence is a mistake in client code: DisallowUnknownFields on the decoder. It makes your client fail the day the API adds a field, which a well-behaved API does without a version bump. Servers should be strict about what they accept, as in rejecting unknown fields; clients should ignore what they do not understand. Decode into the fields you use and let the rest pass. If you need to preserve unknown fields for a round trip (read, modify, write back), capture the original object as json.RawMessage and merge your changes into it, or the write will silently drop data the server added.
The opposite also applies to enums and states. If the API returns "status":"archived" and your code only knows active and deleted, treat the unknown value as a case to handle, not a panic. Switches over server-provided strings should have a default branch that logs and degrades.
Paging styles you will meet
The iterator hides how the server pages, but you still have to implement the right style, and APIs differ. Cursor paging, used above, hands back an opaque token for the next page; it is stable under concurrent writes and the easiest to consume, and the client must treat the token as a black box and never build one. Offset paging (?offset=40&limit=20) is simple and breaks when rows are inserted while you page: items shift and you see one twice or miss one. If the API offers only offsets, deduplicate by ID when it matters. Link header paging, which GitHub's API uses, puts the next URL in a Link: <...>; rel="next" header; the client follows it as given, which again keeps URL construction on the server. The server side of all three is in pagination in Go APIs.
Whatever the style, make the page size a parameter with a sane default, and stop on an empty page as well as on a missing cursor, since a bug in the server that returns the same cursor forever would otherwise make the loop endless. A guard of a maximum page count (a few thousand) turns that failure into an error.
What to log, and what never to
A client is the place where you learn about a vendor's bad day, so give it enough logging to say so afterwards. Per request, at debug level: method, path (not the query if it can hold personal data), status, duration, the request ID from the response, and the attempt number if there is a retry layer. At warn level: any response that was not 2xx. Count outcomes by status class in a metric; a rising share of 5xx or 429 from one dependency is the first sign of trouble, ahead of any user report.
Never log the Authorization header, request or response bodies by default (they carry personal data and secrets), or the full URL if tokens can ride in the query string. If you need bodies to debug, enable it behind a flag for a single request and redact known fields. A handful of integrations I have seen leaked credentials through a debug log that was left on after an incident.
SDK or hand-written client
Many vendors ship a Go SDK. Use it when it is maintained, accepts an *http.Client so you can control timeouts and retries, and returns errors you can inspect. Be wary of ones that create their own client internally, panic on bad input, or pull in a large dependency tree for a few endpoints. Writing the twenty lines you need against the REST API is often the smaller risk, and the typed client in this article is about the size of a small SDK's core anyway. When the vendor publishes an OpenAPI document, generating the types and the methods is the middle path, covered in generating a typed client, and the resulting code can still sit behind an interface of your own.
Before it ships
A few additions are worth the lines. Add an idempotency key to the Create call so a retried POST cannot create two bookmarks, as described in idempotency keys. Add a rate limiter if the API publishes a limit, so the client queues rather than collects 429 responses: the token bucket from rate limiting fits inside do. Log request ID, method, path, status and duration at debug level, never the token. And keep a contract test, running against a staging copy of the real API on a schedule, that exercises each method once; it will find the day the vendor changes a field long before a customer does. The iter.Seq2 pattern used for pagination is described in the Go team's range over function article, and the details of errors.Is with custom Is methods are in the errors package documentation.
