API Gateways for Go Services and Whether to Build One

Once a system has more than two or three services behind it, someone asks for an API gateway. One address for clients, one place for TLS and authentication, one place to apply rate limits and log requests. The idea is sound. The decision that follows is where the arguments are: use a managed gateway from your cloud, run an open-source one such as Envoy, Traefik, nginx or Kong, or write a small one in Go because Go makes reverse proxies embarrassingly easy.

This article lays out what a gateway should do, what it should leave alone, the numbers for the cost of a hop in Go, and how to choose. If you decide to write one, building a small gateway with filters is the next article, and the reverse proxy basics cover the standard library piece it uses. It ran on Go 1.25.6.

What a gateway is for

A gateway is the single entry point that sits between external clients and your services. The jobs that belong there are the ones that are the same for every service and are better done once:

• Routing. /v1/orders/* goes to the orders service, /v1/users/* to users.
• TLS termination. Certificates in one place.
• Authentication. Verifying a token or API key, rejecting bad ones, and passing identity downstream in a trusted header.
• Rate limiting and quotas per client, before the request costs anything.
• Observability. Request ID injection, access logs, metrics, trace context.
• Request hygiene. Size limits, header normalisation, CORS, compression.
• Resilience at the edge. Timeouts, a retry policy for safe methods, circuit breaking per upstream.

The jobs that do not belong there:

• Business logic. The moment the gateway knows what an order is, it has become a service with a worse deployment story.
• Aggregation with real logic. Combining five backend calls into one response is a legitimate pattern, but it is a "backend for frontend" service with an owner, not a gateway plugin.
• Authorisation that depends on data. "Can this user see order 42?" needs the orders service's data. The gateway can check coarse scopes; the service decides objects.

What a hop costs

The standard library's httputil.ReverseProxy makes the proxy part trivial, so the useful question is what it costs. The benchmark below sends requests through three setups with parallel clients: straight to a tiny backend, through a plain reverse proxy, and through a proxy that checks an API key and applies a per-key rate limiter first.

package main

import (
	"io"
	"net/http"
	"net/http/httptest"
	"net/http/httputil"
	"net/url"
	"sync"
	"testing"
	"time"

	"golang.org/x/time/rate"
)

func backend() *httptest.Server {
	return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Content-Type", "application/json")
		io.WriteString(w, `{"ok":true}`)
	}))
}

// gateway is the smallest useful one: route by prefix, check a key, limit per key, forward.
func gateway(target *url.URL, withFilters bool) http.Handler {
	proxy := httputil.NewSingleHostReverseProxy(target)
	// The default transport keeps 2 idle connections per host. Under load a gateway then opens and closes
	// a connection for most requests and runs out of ephemeral ports.
	proxy.Transport = &http.Transport{MaxIdleConnsPerHost: 64, IdleConnTimeout: 90 * time.Second}
	if !withFilters {
		return proxy
	}
	var mu sync.Mutex
	limiters := map[string]*rate.Limiter{}
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		key := r.Header.Get("X-API-Key")
		if key != "k-123" {
			http.Error(w, "unauthorized", http.StatusUnauthorized)
			return
		}
		mu.Lock()
		l, ok := limiters[key]
		if !ok {
			l = rate.NewLimiter(rate.Inf, 1)
			limiters[key] = l
		}
		mu.Unlock()
		if !l.Allow() {
			http.Error(w, "slow down", http.StatusTooManyRequests)
			return
		}
		proxy.ServeHTTP(w, r)
	})
}

func run(b *testing.B, url string, header bool) {
	client := &http.Client{Transport: &http.Transport{MaxIdleConnsPerHost: 64}}
	b.ReportAllocs()
	b.ResetTimer()
	b.RunParallel(func(pb *testing.PB) {
		for pb.Next() {
			req, _ := http.NewRequest(http.MethodGet, url, nil)
			if header {
				req.Header.Set("X-API-Key", "k-123")
			}
			resp, err := client.Do(req)
			if err != nil {
				b.Fatal(err)
			}
			io.Copy(io.Discard, resp.Body)
			resp.Body.Close()
		}
	})
}

func BenchmarkDirect(b *testing.B) {
	be := backend()
	defer be.Close()
	run(b, be.URL, false)
}

func BenchmarkGatewayPlainProxy(b *testing.B) {
	be := backend()
	defer be.Close()
	u, _ := url.Parse(be.URL)
	gw := httptest.NewServer(gateway(u, false))
	defer gw.Close()
	run(b, gw.URL, false)
}

func BenchmarkGatewayWithAuthAndLimit(b *testing.B) {
	be := backend()
	defer be.Close()
	u, _ := url.Parse(be.URL)
	gw := httptest.NewServer(gateway(u, true))
	defer gw.Close()
	run(b, gw.URL, true)
}

package main

func main() {}

go test -run='^$' -bench=. -benchmem -count=1 . | grep -E '^(goos|goarch|cpu|Benchmark|PASS|ok)'

goos: darwin
goarch: arm64
cpu: Apple M1 Max
BenchmarkDirect-10                     	   74900	     14866 ns/op	    5755 B/op	      65 allocs/op
BenchmarkGatewayPlainProxy-10          	   26916	     46081 ns/op	   46703 B/op	     142 allocs/op
BenchmarkGatewayWithAuthAndLimit-10    	   27126	     46057 ns/op	   47637 B/op	     152 allocs/op
PASS
ok  	example.test/lab	4.963s

On an Apple M1 Max, the direct call took about 14.9 microseconds per operation and the plain proxy about 46. The authenticated and limited gateway came out at about 46, which is the same within noise: an in-memory key comparison and a token bucket check are nothing next to the second HTTP exchange. Two runs gave stable numbers within a few percent. Allocations grow from 65 to about 142 per request because the proxy copies headers and body through a second connection.

Read these numbers for what they are. They are loopback figures with no TLS and a backend that answers instantly. On a real network, the dominant costs are the extra round trip (hundreds of microseconds within a data centre) and the connection handling. A gateway hop is cheap in CPU and real in latency. The reason to care is not that Go is slow at proxying, which it is not, but that every request now depends on one more thing being up.

A bug the benchmark found

My first version of the benchmark failed partway through with thousands of dial tcp ...: can't assign requested address errors. The proxy was built with the default transport, which keeps at most two idle connections per host. With sixty-four parallel clients it opened and closed a connection for most requests, burned through the ephemeral port range and started failing.

The fix is in the code above: give the proxy its own http.Transport with a MaxIdleConnsPerHost large enough for your concurrency and an IdleConnTimeout. This is the most common production fault in hand-written gateways, and it looks like network trouble. If you build one, load test it before the first launch, and look for connection churn in ss -s or the transport's metrics. The same tuning applies to HTTP clients in general.

Buy, configure or build

Use a managed gateway (API Gateway on AWS, Apigee, Azure API Management, Cloud Endpoints) when your needs are standard, you are already on that cloud, and you value not running anything. You pay per request, you accept the vendor's limits on timeouts and payload sizes, and you give up custom behaviour. Check the maximum request duration and body size before committing; they surprise people.

Configure an open-source proxy (Envoy, Traefik, nginx, HAProxy, Kong, Caddy) when you need power without writing a proxy. These handle HTTP/2, gRPC, retries, mTLS and health checking, and thousands of people have found their bugs for you. The cost is learning the configuration language and running it, including upgrades. Envoy in particular is the base of most service meshes, so skills transfer.

Build in Go when your requirements are small and specific and you want them in your language and your release process: API key lookup against your own database, per-tenant routing, a particular header contract, request signing. A Go gateway of a few hundred lines is understandable by the whole team, deploys like any other service, and is covered by your usual tests and observability. The cost is that you now own the proxy's edge cases: connection pooling as above, hop-by-hop headers, streaming and flush behaviour, WebSocket upgrades, HTTP/2 specifics, graceful shutdown, and security issues in the HTTP stack that you must patch on the language's release schedule.

A rough decision procedure:

1. If a managed gateway meets your constraints, start there.
2. If you need more control, configure Envoy, Traefik or nginx and see how far it goes.
3. If what you need is a small amount of custom logic in the request path, write it in Go. Use ReverseProxy with its Rewrite hook and a short chain of middleware, as in the build article.
4. If you find yourself implementing retries, load balancing, health checks and circuit breaking from scratch, stop and reconsider option 2.

Operational points that apply to all of them

The gateway is a single point of failure by position. Run at least two instances in different failure domains, health check them from the load balancer (health checks), and make deploys boring: rolling, with connection draining.

Timeouts need a budget. The gateway's upstream timeout must be shorter than the client's, and the services' own timeouts shorter than the gateway's. Otherwise the gateway holds a connection for a request the client has abandoned. Timeouts, retries and deadlines across calls walks through the budget.

Retries multiply. A gateway that retries a failed call to a service that also retries multiplies traffic exactly when the service is struggling. Retry only idempotent methods, cap attempts, and add jitter, as in retry with backoff.

Do not trust inbound identity headers. The gateway must strip X-User-Id and similar headers from incoming requests before setting its own. Otherwise a client can send them directly. Services should accept identity only from the gateway's network or a signed assertion.

Forwarded headers need a policy. X-Forwarded-For is a list that every proxy appends to, and the first entries are whatever the client claimed. Decide how many trusted proxies sit in front of you and read the address from the right side of the list. This is the source of many broken IP-based rate limits.

Log at the edge, with redaction. Access logs at the gateway are the audit trail. Keep tokens and query-string secrets out of them; see API logging without leaking secrets.

When you do not need one

A monolith with one deployment needs a load balancer and a router, not a gateway. Two internal services that call each other can use plain HTTP clients with sensible timeouts. A gateway pays off when external clients meet several backends and you want shared policy applied once. If you cannot name three jobs it will do that are the same for every service, wait. The reverse proxy documentation is in the net/http/httputil package, and Envoy's own architecture overview is a useful read before you choose between configuring and building.