Build a Link Preview Service in Go Without SSRF Holes
Build a link preview service in Go that fetches a URL and reads its Open Graph tags, with a dialer that blocks private addresses after DNS to stop SSRF.
Paste a link into a chat app and a card appears with a title, a description and a picture. Behind that card is a server that fetched the page on your behalf, and that is the dangerous part. A link preview service takes a URL from a stranger and makes your infrastructure open a connection to it. If the stranger types http://169.254.169.254/latest/meta-data/ or http://localhost:6379/, a naive implementation connects to the cloud metadata service or your own Redis, from inside your network, and may return what it finds. That is server side request forgery (SSRF), and it is the reason link previews have a reputation as a place where breaches start.
Tested with Go 1.25.6 and golang.org/x/net v0.48.0, macOS on Apple Silicon (arm64)
This project builds the feature properly. It fetches a page, extracts Open Graph metadata, and refuses the destinations it must never touch, in a way that survives redirects, tricky hostnames and DNS tricks. It belongs to the web cluster that ends in a production HTTP server, follows the client settings from the HTTP client article, and the general class of bug is covered in SSRF in Go. Everything ran on Go 1.25.6 with golang.org/x/net v0.48.0 for the HTML tokenizer.
Why checking the URL is not enough
The first defence people write is a check on the URL string: parse it, look at the host, reject localhost and anything that starts with 10.. It fails in at least five ways.
- The host can be a name that resolves to an internal address.
internal.attacker.exampleis anArecord pointing at10.0.0.5, and no string check on the name sees that. - The check and the connection resolve the name separately, so an attacker with a DNS server answers a public address to your check and
127.0.0.1to the dial moments later. This is DNS rebinding, and it is a real technique, not a theory. - A redirect from a harmless public URL to an internal one arrives after your check.
- Addresses have many spellings:
2130706433and0x7f000001are both127.0.0.1, and::ffff:127.0.0.1is an IPv6 form of the same address. - You can forget a whole range, such as
169.254.0.0/16, home of the cloud metadata endpoint, or100.64.0.0/10, used for carrier-grade NAT and by some providers internally.
All five have the same cure: do not judge the URL, judge the address the operating system is about to connect to. In Go that moment is net.Dialer.Control. It is called after name resolution, for every connection attempt, with the numeric host:port that is about to be dialed, and returning an error aborts the connection. Redirects, rebinding and unusual spellings all end up as a numeric address there.
The guard
package main
import (
"errors"
"fmt"
"net"
"net/http"
"net/netip"
"syscall"
"time"
)
// Prefixes that netip's helper methods do not cover but that still reach internal networks.
var extraBlocked = []netip.Prefix{
netip.MustParsePrefix("100.64.0.0/10"), // carrier-grade NAT, also used by some cloud providers internally
netip.MustParsePrefix("192.0.0.0/24"), // IETF protocol assignments
netip.MustParsePrefix("198.18.0.0/15"), // benchmarking
netip.MustParsePrefix("240.0.0.0/4"), // reserved
netip.MustParsePrefix("64:ff9b::/96"), // NAT64: an IPv4 address in disguise
}
// blocked reports whether an address is one the previewer must never connect to.
func blocked(a netip.Addr) bool {
a = a.Unmap() // ::ffff:127.0.0.1 is 127.0.0.1
if a.IsLoopback() || a.IsPrivate() || a.IsLinkLocalUnicast() || a.IsLinkLocalMulticast() ||
a.IsInterfaceLocalMulticast() || a.IsMulticast() || a.IsUnspecified() {
return true
}
for _, p := range extraBlocked {
if p.Contains(a) {
return true
}
}
return false
}
// safeDialer checks the address the OS is about to connect to, after DNS has been resolved.
// allow is an escape hatch for tests; production passes nil.
func safeDialer(allow func(netip.AddrPort) bool) *net.Dialer {
return &net.Dialer{
Timeout: 3 * time.Second,
Control: func(network, address string, _ syscall.RawConn) error {
ap, err := netip.ParseAddrPort(address)
if err != nil {
return fmt.Errorf("unparseable address %q", address)
}
if blocked(ap.Addr()) && (allow == nil || !allow(ap)) {
return fmt.Errorf("address %s is not allowed", ap.Addr())
}
return nil
},
}
}
func safeClient(allow func(netip.AddrPort) bool, timeout time.Duration) *http.Client {
return &http.Client{
Timeout: timeout,
Transport: &http.Transport{
Proxy: nil, // a proxy would make the connection on our behalf, past the check
DialContext: safeDialer(allow).DialContext,
DisableKeepAlives: true,
ResponseHeaderTimeout: timeout,
MaxResponseHeaderBytes: 32 << 10,
},
CheckRedirect: func(req *http.Request, via []*http.Request) error {
if len(via) >= 3 {
return errors.New("too many redirects")
}
if req.URL.Scheme != "http" && req.URL.Scheme != "https" {
return fmt.Errorf("redirect to %q refused", req.URL.Scheme)
}
return nil
},
}
}The function blocked is an allow-by-exclusion list, and its first line matters most. Unmap turns an IPv4-mapped IPv6 address back into its IPv4 form, so ::ffff:127.0.0.1 cannot sneak past a check that only knew IPv4 loopback. The netip.Addr helper methods cover loopback, private (10/8, 172.16/12, 192.168/16, fc00::/7), link-local (which includes 169.254.169.254), multicast and unspecified. The extra prefixes add ranges the helpers deliberately leave out: carrier-grade NAT, reserved space, and the NAT64 prefix, where an IPv4 address is embedded in IPv6 and a gateway translates it.
Three decisions in safeClient are as important as the dialer. Proxy: nil matters because the default transport honors HTTPS_PROXY; with a proxy, the TCP connection goes to the proxy, which is allowed, and the proxy connects onward to the internal address with no check from us. DisableKeepAlives makes every request dial afresh, so the guard runs for every request and a pooled connection cannot carry a request to a different place than the check approved. And CheckRedirect limits the chain to three hops and refuses non-HTTP schemes; each hop is dialed through the same guarded transport, so a redirect to an internal address fails exactly like a direct request.
Fetching and parsing
package main
import (
"context"
"errors"
"fmt"
"io"
"mime"
"net/http"
"net/url"
"strings"
"unicode/utf8"
"golang.org/x/net/html"
)
const maxBody = 1 << 20
type Preview struct {
URL string `json:"url"`
Title string `json:"title"`
Description string `json:"description,omitempty"`
Image string `json:"image,omitempty"`
SiteName string `json:"site_name,omitempty"`
}
type Fetcher struct{ client *http.Client }
func (f *Fetcher) Fetch(ctx context.Context, raw string) (*Preview, error) {
u, err := url.Parse(raw)
if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Hostname() == "" {
return nil, errors.New("only absolute http and https URLs can be previewed")
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil)
if err != nil {
return nil, err
}
req.Header.Set("User-Agent", "gohacks-linkpreview/1.0")
req.Header.Set("Accept", "text/html,application/xhtml+xml")
res, err := f.client.Do(req)
if err != nil {
return nil, fmt.Errorf("fetching %s: %w", u.Host, err)
}
defer res.Body.Close()
if res.StatusCode != http.StatusOK {
return nil, fmt.Errorf("the page answered %s", res.Status)
}
if mt, _, _ := mime.ParseMediaType(res.Header.Get("Content-Type")); mt != "text/html" && mt != "application/xhtml+xml" {
return nil, fmt.Errorf("not an HTML page (%q)", res.Header.Get("Content-Type"))
}
p, err := parseHead(io.LimitReader(res.Body, maxBody), res.Request.URL)
if err != nil {
return nil, err
}
p.URL = res.Request.URL.String()
return p, nil
}
// parseHead reads meta tags from the head of a document and stops at </head>.
func parseHead(r io.Reader, base *url.URL) (*Preview, error) {
z := html.NewTokenizer(r)
meta := map[string]string{}
var title string
inTitle := false
loop:
for {
switch z.Next() {
case html.ErrorToken:
break loop
case html.StartTagToken, html.SelfClosingTagToken:
t := z.Token()
switch t.Data {
case "title":
inTitle = true
case "meta":
var key, val string
for _, a := range t.Attr {
switch a.Key {
case "property", "name":
key = strings.ToLower(a.Val)
case "content":
val = a.Val
}
}
if key != "" {
if _, seen := meta[key]; !seen {
meta[key] = val
}
}
case "body":
break loop
}
case html.TextToken:
if inTitle && title == "" {
title = strings.TrimSpace(string(z.Text()))
}
case html.EndTagToken:
t := z.Token()
if t.Data == "title" {
inTitle = false
}
if t.Data == "head" {
break loop
}
}
}
first := func(keys ...string) string {
for _, k := range keys {
if v := strings.TrimSpace(meta[k]); v != "" {
return v
}
}
return ""
}
pageTitle := first("og:title", "twitter:title")
if pageTitle == "" {
pageTitle = title
}
p := &Preview{
Title: clip(pageTitle, 120),
Description: clip(first("og:description", "twitter:description", "description"), 280),
SiteName: clip(first("og:site_name"), 60),
}
if img := first("og:image", "twitter:image"); img != "" {
if iu, err := base.Parse(img); err == nil && (iu.Scheme == "http" || iu.Scheme == "https") {
p.Image = iu.String() // relative image paths resolve against the final URL
}
}
if p.Title == "" {
return nil, errors.New("the page has no title")
}
return p, nil
}
// clip shortens text to n runes without cutting a character in half.
func clip(s string, n int) string {
if utf8.RuneCountInString(s) <= n {
return s
}
r := []rune(s)
return string(r[:n-1]) + "…"
}The parser reads only as far as it needs: it tokenizes the document, collects <meta> tags and the <title>, and stops at </head> or <body>, so a hostile page cannot make the service tokenize a gigabyte of markup (the io.LimitReader bound of 1 MiB is a second guard). The og: properties come first, then Twitter's equivalents, then the plain <title> and description. The first value seen for a key wins, which stops a page from overriding a title with a later duplicate. The image URL is resolved against the final URL of the response, after redirects, and only http and https results are kept, so a javascript: or data: image never reaches your front end.
Everything returned is clipped by rune count, not bytes, so a multi-byte character is never cut in half. This is the part that is easy to overlook: the preview comes from an untrusted page and ends up rendered in your product, so treat every field as untrusted text, escape it on output, and never insert it as HTML. If the front end shows the image, serve it through your own image proxy, which also has to run behind this same guard.
Trying to break it
The program prints the address classification first, and then runs the fetcher against a "site" server and an "internal" server, both of which are really on loopback. The guard's test hook allows exactly the site's address and port, which stands in for a public host, and leaves the internal server blocked.
package main
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"net/netip"
"strings"
"time"
)
func main() {
fmt.Println("which addresses are blocked:")
for _, s := range []string{"8.8.8.8", "2606:4700:4700::1111", "127.0.0.1", "10.0.0.5", "172.16.3.4", "192.168.1.1", "169.254.169.254",
"100.64.0.1", "0.0.0.0", "::1", "fe80::1", "fc00::1", "::ffff:127.0.0.1", "64:ff9b::7f00:1"} {
fmt.Printf(" %-24s blocked=%v\n", s, blocked(netip.MustParseAddr(s)))
}
internal := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, "<html><head><title>Internal admin</title></head></html>")
}))
defer internal.Close()
var site *httptest.Server
site = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/article":
w.Header().Set("Content-Type", "text/html; charset=utf-8")
fmt.Fprint(w, `<!doctype html><html><head><title>Plain title</title>
<meta property="og:title" content="Go 1.25 released">
<meta property="og:description" content="The release notes, in short.">
<meta property="og:image" content="/img/cover.png">
<meta property="og:site_name" content="Example News">
</head><body><h1>Hello</h1></body></html>`)
case "/plain":
w.Header().Set("Content-Type", "text/html")
fmt.Fprint(w, `<html><head><title> Only a title </title></head><body></body></html>`)
case "/bounce":
http.Redirect(w, r, internal.URL+"/admin", http.StatusFound)
case "/image":
w.Header().Set("Content-Type", "image/png")
fmt.Fprint(w, "not html")
case "/slow":
time.Sleep(time.Second)
case "/huge":
w.Header().Set("Content-Type", "text/html")
fmt.Fprint(w, "<html><head><title>")
fmt.Fprint(w, strings.Repeat("x", 3<<20))
}
}))
defer site.Close()
siteAddr := netip.MustParseAddrPort(strings.TrimPrefix(site.URL, "http://"))
allow := func(ap netip.AddrPort) bool { return ap == siteAddr }
guarded := &Fetcher{client: safeClient(allow, 400*time.Millisecond)}
open := &Fetcher{client: safeClient(func(netip.AddrPort) bool { return true }, 400*time.Millisecond)}
port := func(srv *httptest.Server) string { return strings.Split(srv.Listener.Addr().String(), ":")[1] }
mask := strings.NewReplacer(site.Listener.Addr().String(), "SITE", internal.Listener.Addr().String(), "INTERNAL")
show := func(label string, f *Fetcher, target string) {
p, err := f.Fetch(context.Background(), target)
if err != nil {
fmt.Printf(" %-26s refused: %s\n", label, mask.Replace(err.Error()))
return
}
fmt.Printf(" %-26s %s\n", label, mask.Replace(fmt.Sprintf("%+v", *p)))
}
fmt.Println("previews:")
show("article with Open Graph", guarded, site.URL+"/article")
show("title only", guarded, site.URL+"/plain")
show("redirect to internal host", guarded, site.URL+"/bounce")
show("same redirect, no guard", open, site.URL+"/bounce")
show("internal host directly", guarded, internal.URL+"/admin")
show("not HTML", guarded, site.URL+"/image")
show("slow server", guarded, site.URL+"/slow")
show("3 MB title", guarded, site.URL+"/huge")
show("file scheme", guarded, "file:///etc/passwd")
show("localhost by name", guarded, "http://localhost:"+port(internal)+"/")
show("decimal IP for 127.0.0.1", guarded, "http://2130706433:"+port(internal)+"/")
}which addresses are blocked:
8.8.8.8 blocked=false
2606:4700:4700::1111 blocked=false
127.0.0.1 blocked=true
10.0.0.5 blocked=true
172.16.3.4 blocked=true
192.168.1.1 blocked=true
169.254.169.254 blocked=true
100.64.0.1 blocked=true
0.0.0.0 blocked=true
::1 blocked=true
fe80::1 blocked=true
fc00::1 blocked=true
::ffff:127.0.0.1 blocked=true
64:ff9b::7f00:1 blocked=true
previews:
article with Open Graph {URL:http://SITE/article Title:Go 1.25 released Description:The release notes, in short. Image:http://SITE/img/cover.png SiteName:Example News}
title only {URL:http://SITE/plain Title:Only a title Description: Image: SiteName:}
redirect to internal host refused: fetching SITE: Get "http://INTERNAL/admin": dial tcp INTERNAL: address 127.0.0.1 is not allowed
same redirect, no guard {URL:http://INTERNAL/admin Title:Internal admin Description: Image: SiteName:}
internal host directly refused: fetching INTERNAL: Get "http://INTERNAL/admin": dial tcp INTERNAL: address 127.0.0.1 is not allowed
not HTML refused: not an HTML page ("image/png")
slow server refused: fetching SITE: Get "http://SITE/slow": net/http: timeout awaiting response headers (Client.Timeout exceeded while awaiting headers)
3 MB title {URL:http://SITE/huge Title:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx… Description: Image: SiteName:}
file scheme refused: only absolute http and https URLs can be previewed
localhost by name refused: fetching localhost:52641: Get "http://localhost:52641/": dial tcp [::1]:52641: address ::1 is not allowed
decimal IP for 127.0.0.1 refused: fetching 2130706433:52641: Get "http://2130706433:52641/": dial tcp INTERNAL: address 127.0.0.1 is not allowedThe classification table is the first thing to read. Public addresses pass, and everything else the guard knows about is refused, including the two forms that tend to slip through reviews: ::ffff:127.0.0.1 and the NAT64 spelling of loopback.
In the preview block, the article with Open Graph tags gives the og: title in preference to the page <title>, a description, a site name and an image whose relative path was resolved against the page address. The title-only page falls back to the trimmed <title>. The redirect row is the key one. The site is allowed, so the first request goes through, but it answers 302 to the internal server, and the dialer refuses that second connection: the error names the address, 127.0.0.1 is not allowed. Run the same redirect through a fetcher with the guard opened and it succeeds and returns the internal page's title, which is what an attacker would have received.
The remaining rows each close one hole. The internal host is refused when asked directly. A response of the wrong type (image/png) is rejected before parsing. The slow server hits the client timeout, so one slow target cannot hold a worker. The 3 MB title is read only up to the 1 MiB limit and then clipped to 120 characters. The file:/// URL fails the scheme check before any network access. And localhost, resolved by the system to ::1, and the decimal form 2130706433, which the system resolver turns into 127.0.0.1, are both stopped at the dialer, because the dialer sees addresses and not the spellings that led to them.
Tests for the parts that must not regress
The demo above is a tour; the regression guard is a table test. Two tables carry most of the value: the address classifier, because a new range added to the deny list or removed by a refactor is a security change, and the head parser, because the hostile inputs are known.
package main
import (
"net/netip"
"net/url"
"strings"
"testing"
)
func TestBlockedAddresses(t *testing.T) {
for addr, want := range map[string]bool{
"1.1.1.1": false, "93.184.216.34": false, "2606:4700:4700::1111": false,
"127.0.0.1": true, "127.255.255.254": true, "10.1.2.3": true, "172.31.255.255": true, "172.32.0.1": false,
"192.168.0.1": true, "169.254.169.254": true, "100.127.255.255": true, "100.128.0.1": false,
"0.0.0.0": true, "224.0.0.1": true, "255.255.255.255": true,
"::1": true, "::": true, "fe80::1": true, "fd00::1": true, "ff02::1": true,
"::ffff:10.0.0.1": true, "::ffff:8.8.8.8": false,
} {
if got := blocked(netip.MustParseAddr(addr)); got != want {
t.Errorf("blocked(%s) = %v, want %v", addr, got, want)
}
}
}
func TestParseHead(t *testing.T) {
base, _ := url.Parse("https://example.org/news/story")
cases := []struct {
name, doc string
title string
image string
wantErr bool
}{
{"og beats title", `<head><title>T</title><meta property="og:title" content="OG"></head>`, "OG", "", false},
{"first duplicate wins", `<head><meta property="og:title" content="one"><meta property="og:title" content="two"></head>`, "one", "", false},
{"relative image resolves", `<head><title>T</title><meta property="og:image" content="../img/a.png"></head>`, "T", "https://example.org/img/a.png", false},
{"script image dropped", `<head><title>T</title><meta property="og:image" content="javascript:alert(1)"></head>`, "T", "", false},
{"no title is an error", `<head><meta name="description" content="x"></head>`, "", "", true},
{"stops at body", `<head><title>T</title></head><body><meta property="og:title" content="late"></body>`, "T", "", false},
{"long title is clipped by runes", `<head><title>` + strings.Repeat("é", 300) + `</title></head>`, strings.Repeat("é", 119) + "…", "", false},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
p, err := parseHead(strings.NewReader(tc.doc), base)
if (err != nil) != tc.wantErr {
t.Fatalf("err = %v, wantErr %v", err, tc.wantErr)
}
if err == nil && (p.Title != tc.title || p.Image != tc.image) {
t.Errorf("title %q image %q, want %q %q", p.Title, p.Image, tc.title, tc.image)
}
})
}
}=== RUN TestBlockedAddresses
--- PASS: TestBlockedAddresses (0.00s)
=== RUN TestParseHead
=== RUN TestParseHead/og_beats_title
=== RUN TestParseHead/first_duplicate_wins
=== RUN TestParseHead/relative_image_resolves
=== RUN TestParseHead/script_image_dropped
=== RUN TestParseHead/no_title_is_an_error
=== RUN TestParseHead/stops_at_body
=== RUN TestParseHead/long_title_is_clipped_by_runes
--- PASS: TestParseHead (0.00s)
--- PASS: TestParseHead/og_beats_title (0.00s)
--- PASS: TestParseHead/first_duplicate_wins (0.00s)
--- PASS: TestParseHead/relative_image_resolves (0.00s)
--- PASS: TestParseHead/script_image_dropped (0.00s)
--- PASS: TestParseHead/no_title_is_an_error (0.00s)
--- PASS: TestParseHead/stops_at_body (0.00s)
--- PASS: TestParseHead/long_title_is_clipped_by_runes (0.00s)
PASS
ok example.test/lab 1.267sThe address table includes the boundaries: 172.31.255.255 is the last private address and 172.32.0.1 the first public one, and the same for 100.127.255.255 and 100.128.0.1. Off-by-one mistakes in a deny list are typically at the edges, so test the edges. The parser cases cover the four hostile moves from earlier: a duplicate tag meant to override, a script URL in an image field, content after <body> that must not count, and a title long enough to need clipping, measured in runes so the accented characters survive.
Other designs worth knowing
The dial-time guard is the best general answer when users can supply any URL. There are two stronger options when your situation permits. If previews are only needed for a known set of sites (your own docs, a few partners), use an allowlist of hostnames in addition to the dialer, since an attacker cannot point an allowlisted name at an internal address unless they control its DNS. And if the fetcher must reach arbitrary sites at scale, run it behind a dedicated egress proxy that enforces the policy centrally. Stripe's open source Smokescreen is the best known example: every outbound request from every service goes through it, and the deny rules live in one reviewed place. I have not run it here, so I will not describe its configuration beyond saying the approach scales better than copying a guard into every service.
Either way, treat the guard as code that deserves a regular review. The set of addresses worth blocking changes (new reserved ranges, new cloud metadata endpoints), and an old deny list is a false comfort.
Operating it safely
The guard is necessary and not sufficient. A few more habits complete it.
Run the fetcher where it can do little harm. Put it in a separate service or container with no credentials, no access to your databases and egress rules that allow only the public internet. If the guard has a bug, network policy is the second line. On cloud platforms, also require the newer metadata service version that needs a request header, which most SSRF payloads cannot add.
Rate limit and cache. Previews are expensive and abusable. Limit requests per user, cache results by final URL for hours, and de-duplicate in-flight fetches with singleflight, as in singleflight for duplicate work. A cache also hides your fetcher's IP from the sites being previewed, which they will appreciate.
Respect the target. Send a clear User-Agent with a contact URL, honor robots.txt for anything beyond a single user-initiated fetch, and do not follow links on the page.
Do not return raw responses, and do not echo errors. The refusal messages above are for logs. The API response should say only "this link can't be previewed". Detailed errors let an attacker map your internal network by trying addresses and reading the difference between "connection refused" and "not allowed", a technique called blind SSRF.
Re-check on the image proxy. If the front end loads og:image directly from the page, a malicious page controls what your users' browsers fetch. Proxy images through a second guarded fetcher, validate the content type and size, and re-encode.
What to watch for in production
Log every refusal with the requesting user, the target host and the reason, and alert on bursts. A person who tries 169.254.169.254 once is probing; someone who walks a dozen internal addresses in a minute is scanning your network through your product, and the refusal log is the only place you will see it. Count outcomes by category (refused address, timeout, non-HTML, too large, parse failure) and look at the ratios over time: a sudden rise in timeouts may be one slow site, but a rise in refused addresses means somebody found the feature.
Keep a short, tested list of URLs that must always be refused, and run it in CI against the real client, not only against the blocked function. The dialer wiring is the part that can quietly regress: a refactor that swaps the transport for the default one makes every address test still pass while the guard is gone. A test that starts a loopback server, does not allow it, and expects a refusal is the check that fails when that happens, and it is the first test I would write.
For the broader class of mistakes, OWASP's SSRF prevention cheat sheet lists the defences by situation, and the Go documentation for net.Dialer.Control describes the hook used here. The Open Graph fields are specified at ogp.me.
