Skip to content
gohacks.dev
Performance

Profile a Running Go Service Safely With the pprof Handlers

Expose net/http/pprof safely: the import that publishes it on DefaultServeMux, a token protected admin listener, and what a goroutine dump reveals

9 min read
Summarize in AI
Flat illustration of a flame graph with one hot path.

The net/http/pprof package turns a running Go service into something you can profile with one command: go tool pprof http://host:6060/debug/pprof/profile?seconds=30. It is the standard route for CPU, heap, goroutine, mutex and block profiles from production, and one of the most common ways Go services leak information and offer an attacker a free denial of service. This article shows how the handlers get registered (and why a single import is enough to publish them), how to serve them on a separate protected listener, what an exposed endpoint reveals, and how long profiles interact with server timeouts. Each claim is a request against an httptest server.

Tested with Go 1.25.6 on Apple M1 Max (10 cores, 32 GB), macOS, httptest servers on loopback

It is the production companion to CPU profiling and heap profiles, and it is the access layer for continuous profiling. For a broader view of the standard endpoint, see the pprof endpoint article; for server hardening, HTTP server timeouts.

The import registers the handlers

The package registers its handlers in an init function, on http.DefaultServeMux:

/debug/pprof/ (index), /debug/pprof/cmdline, /debug/pprof/profile, /debug/pprof/symbol and /debug/pprof/trace, and the named profiles (heap, goroutine, allocs, block, mutex, threadcreate) via the index. It does not matter whether you import the package with a blank identifier or by name. The side effect is the same.

If your service serves http.DefaultServeMux (a bare http.ListenAndServe(addr, nil), or a handler that falls back to it), the profiler is now on your public port. Many services get here by adding the import "temporarily" to debug something.

The lab

The program starts several httptest servers and makes requests to each:

main.goGo
package main

import (
	"fmt"
	"io"
	"net/http"
	"net/http/httptest"
	"net/http/pprof"
	"strings"
	"time"
)

// Importing net/http/pprof, even with a blank identifier, registers its handlers on http.DefaultServeMux.

func get(url, token string) (int, []byte) {
	req, _ := http.NewRequest("GET", url, nil)
	if token != "" {
		req.Header.Set("Authorization", "Bearer "+token)
	}
	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		return 0, []byte(err.Error())
	}
	defer resp.Body.Close()
	b, _ := io.ReadAll(resp.Body)
	return resp.StatusCode, b
}

// adminMux registers the handlers on a mux you control, behind a token.
func adminMux(secret string) http.Handler {
	mux := http.NewServeMux()
	mux.HandleFunc("/debug/pprof/", pprof.Index)
	mux.HandleFunc("/debug/pprof/cmdline", pprof.Cmdline)
	mux.HandleFunc("/debug/pprof/profile", pprof.Profile)
	mux.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
	mux.HandleFunc("/debug/pprof/trace", pprof.Trace)
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		if r.Header.Get("Authorization") != "Bearer "+secret {
			http.Error(w, "unauthorized", http.StatusUnauthorized)
			return
		}
		mux.ServeHTTP(w, r)
	})
}

func main() {
	// 1. The accident: a service that serves http.DefaultServeMux exposes the profiler.
	accident := httptest.NewServer(http.DefaultServeMux)
	code, body := get(accident.URL+"/debug/pprof/", "")
	fmt.Println("1. DefaultServeMux after the import:", code, "| lists heap:", strings.Contains(string(body), "heap"))

	// 2. A service with its own mux does not have the endpoints.
	own := http.NewServeMux()
	own.HandleFunc("/hello", func(w http.ResponseWriter, r *http.Request) { fmt.Fprint(w, "ok") })
	public := httptest.NewServer(own)
	code, _ = get(public.URL+"/debug/pprof/", "")
	hello, _ := get(public.URL+"/hello", "")
	fmt.Println("2. public server with its own mux: /debug/pprof/ is", code, "| /hello is", hello)

	// 3. An admin listener with a token.
	admin := httptest.NewServer(adminMux("s3cret"))
	code, _ = get(admin.URL+"/debug/pprof/", "")
	fmt.Println("3. admin without a token:", code)
	code, _ = get(admin.URL+"/debug/pprof/", "s3cret")
	fmt.Println("   admin with the token:", code)

	// 4. What a goroutine dump gives away.
	code, body = get(accident.URL+"/debug/pprof/goroutine?debug=1", "")
	var leaks []string
	for _, l := range strings.Split(string(body), "\n") {
		if strings.Contains(l, "main.") || strings.Contains(l, ".go:") {
			leaks = append(leaks, strings.TrimSpace(l))
		}
	}
	fmt.Println("4. goroutine dump:", code, "|", len(leaks), "lines naming functions or source files, for example:")
	for _, l := range leaks[:2] {
		if i := strings.LastIndex(l, "/"); i > 0 {
			l = "..." + l[i:]
		}
		fmt.Println("     ", l)
	}

	// 5. A heap profile downloaded over HTTP is a gzip file.
	code, body = get(accident.URL+"/debug/pprof/heap", "")
	fmt.Printf("5. heap profile: %d, %d bytes, gzip magic %x\n", code, len(body), body[:2])

	// 6. Profiles block for their duration. In the Go 1.25 source used here the handler extends the write deadline, so a short WriteTimeout does not cut a longer profile.
	slow := httptest.NewUnstartedServer(adminMux("s3cret"))
	slow.Config.WriteTimeout = time.Second
	slow.Start()
	start := time.Now()
	code, body = get(slow.URL+"/debug/pprof/profile?seconds=2", "s3cret")
	fmt.Printf("6. 2 s profile on a server with a 1 s WriteTimeout: %d after %d s, %d bytes\n", code, int(time.Since(start).Round(time.Second)/time.Second), len(body))
}
Shell
set -e
go run .
Text
1. DefaultServeMux after the import: 200 | lists heap: true
2. public server with its own mux: /debug/pprof/ is 404 | /hello is 200
3. admin without a token: 401
   admin with the token: 200
4. goroutine dump: 200 | 57 lines naming functions or source files, for example:
      .../netpoll.go:351
      .../fd_poll_runtime.go:84
5. heap profile: 200, 1170 bytes, gzip magic 1f8b
6. 2 s profile on a server with a 1 s WriteTimeout: 200 after 2 s, 211 bytes

1. The accident

A server serving http.DefaultServeMux, in a program that imports net/http/pprof, answered GET /debug/pprof/ with 200 and an index page that lists heap. Nothing else in the program mentions pprof. Anyone who can reach that port can download a CPU profile, a heap profile or a full goroutine dump. The first of those also makes the server busy for the duration of the profile, which is a cheap denial of service for anyone who asks for ?seconds=30 in a loop.

2. Your own mux

A service that builds its own http.NewServeMux() and registers only its routes does not have the endpoints, even though the package is imported in the same binary. In the lab, /debug/pprof/ on such a server returned 404 and /hello returned 200. This is the simplest defence: never serve the default mux. Use your own mux or a router, and the import registers handlers on a mux nobody listens on.

(The test in the lab uses a specific route for /hello. A catch all "/" pattern would answer 200 for everything, which I discovered in my first version and which hides the point.)

3. A separate listener with authentication

The pattern for production is a second listener that is not reachable from the internet:

  1. Listen on 127.0.0.1:6060 or an internal interface, not :6060.
  2. Register the handlers on a dedicated mux, explicitly: pprof.Index, pprof.Profile, pprof.Symbol, pprof.Trace, pprof.Cmdline.
  3. Wrap the mux in authentication (a bearer token, mTLS, or an identity aware proxy).

The lab's admin mux does steps 2 and 3. A request without the token returned 401, with the token 200. Reaching a loopback port from a laptop usually means kubectl port-forward or an SSH tunnel, which also gives you an access log. On Kubernetes a ClusterIP service, a network policy that only admits the profiling collector, or a sidecar are the equivalents of "internal interface".

A token in the URL (?token=...) ends up in logs. A header is better. If an identity aware proxy fronts the port, the application check is the second layer.

What an exposed endpoint gives away

The heap and CPU profiles contain function names and source file paths. The goroutine dump is the most revealing. The lab fetched /debug/pprof/goroutine?debug=1 from the unprotected server and found 57 lines that name functions or source files, for example .../netpoll.go:351 and .../fd_poll_runtime.go:84. From a text dump an observer learns:

  • The package and function names of your service, hence its structure and dependencies.
  • Library versions through file paths in the module cache.
  • What each goroutine is doing, including blocked states and some arguments.
  • Sometimes the contents of arguments on a debug=2 dump, which prints full stacks with values.

/debug/pprof/cmdline prints the process's command line, which can contain secrets passed as flags. None of that is a credential on its own, and each piece makes the next step of an attack easier. The same data is what you want on a trusted network, so the rule is where to serve it, not whether.

Downloading and using a profile

A heap profile fetched over HTTP returned 200 with 1135 bytes and the gzip magic bytes 1f8b. It is a compressed protobuf; go tool pprof reads it directly from the URL or from a saved file:

go tool pprof http://127.0.0.1:6060/debug/pprof/heap or curl -o heap.pb.gz ... followed by go tool pprof heap.pb.gz.

Pass the binary as well when you want line level -list (go tool pprof ./service heap.pb.gz), and use -http=:8081 for the web UI. Add ?gc=1 to the heap URL to force a collection before the snapshot, as explained in the heap article.

Timeouts and long profiles

The CPU profile endpoint blocks for seconds (default 30) while it samples, then writes the result. That interacts with server timeouts: a WriteTimeout shorter than the profile would normally cut the response. Earlier releases of the package rejected such a request with a 400 and the message profile duration exceeds server's WriteTimeout; I did not run an older toolchain, so treat that as the documented behaviour of those releases. In the Go 1.25 source used here the handler instead extends the write deadline by the profile duration, which the lab confirmed: a 2 second profile from a server with a 1 second WriteTimeout returned 200 after 2 seconds with 210 bytes. If you run an older toolchain, check which behaviour you have, because a profile that silently fails is not a good moment to learn it.

Two more details. Only one CPU profile can run at a time in a process, so a second request while one is running fails. And the client needs a generous timeout: go tool pprof waits for the whole duration, and a proxy with a 30 second idle timeout in front of the port will cut a 60 second profile.

A checklist for production

  • Do not serve http.DefaultServeMux. Use your own mux for the public listener.
  • Serve the profiler on a separate listener bound to an internal address.
  • Authenticate it (token header, mTLS or an identity aware proxy).
  • Register only the handlers you need. trace and profile consume CPU, so they deserve the most protection.
  • Do not log full URLs with credentials.
  • Keep access to the port off the internet in network policy.
  • Test it: a request to the public port for /debug/pprof/ must return 404, and a request to the admin port without credentials must return 401. This is an easy test to put in CI.

Checklist

  • The import registers on the default mux, with or without a blank identifier.
  • A service with its own mux does not expose the handlers.
  • Admin listener plus bearer token returned 401 and 200 as designed.
  • Goroutine dumps leak structure, so treat the endpoint as sensitive.
  • Check your Go version's write deadline behaviour before relying on long profiles.

The handlers are documented in the net/http/pprof package and the profiling workflow in the Go diagnostics guide.

Share

Keep reading

Get new articles by email

We send one email when a new article is published. No spam.

Unsubscribe in one click. No spam, ever.