Backtestshugorun-20260930-140149#15301

Issue #15301 Merged fix PR # Run 30 Sep 2026 · run-20260930-140149 gohugoio/hugo

Improve/document security.http.urls override behavior

Better

Our patch was judged better than the merged fix.

In this judgment, Patch B = our agent, Patch A = the merged fix.

The issue asks to improve or document how a custom `security.http.urls` disables the dial-time resolved-address check (`CheckAllowedHTTPAddress`).

Patch B addresses this directly:
- It documents the behavior in security.md: the default value enables resolved-address validation, any deviation from the default disables it, and custom patterns should be narrow and anchored.
- It adds a clarifying code comment.
- It adds a `reason` to `AccessDeniedError`, so a dial-time denial now says why it happened and notes that a custom urls setting disables the check. Before, the error was confusing because the URL itself had passed the allowlist.
- Its tests are meaningful. They check that the new message appears for address denials and not for URL-text denials, that changing other http settings keeps the check, and that a custom urls value disables it. The message assertions would fail before the change.

Patch A also adds similar documentation for `http.urls`. However, most of the patch is a new `security.http.proxyFromEnvironment` feature that the issue does not request. It sets `base.Proxy = nil` by default, which silently stops honoring HTTP_PROXY/HTTPS_PROXY for all users. That is a notable behavior change and will break proxied corporate or CI environments. It also brings in a new `httpproxy` dependency, a custom `RoundTripper`, context-key plumbing and a hand-rolled `canonicalAddr` mirror of net/http internals, all of which add maintenance risk. The proxy test is solid, but it tests the extra feature rather than the issue's ask.

Patch A's scope expansion adds real risk and obscures the documentation fix. Patch B is focused, low-risk, and improves both the docs and the user-facing error. B is the better fix.

The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.

Solve

24m 20s20 turns

Wall-clock time in the container

Full test suite

No new failures

3,754 passed · 100 skipped · 3,856 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

49vs 167 merged

Changed lines, ours vs merged

In d6e6f9e500eebdeae8e28de830fff2e3bfc7d534 (unreleased) we added a dial-time URL validation to catch URLs that's valid according to the whitelist but that resolves to a loopback, private or link-local IP addresses. That check is skipped it if the user has a custom security.http.urls setup, as it would not make sense to block local development servers.

See CheckAllowedHTTPAddress.

This has been reported by @Reload3d as a potential security issue, and while I think the current behaviour is the pragmatic correct approach, I'm creating this issue for visibility and to see if this can be improved further (e.g. better documentation).

The two patches

Our patch+48 −1
config/security/securityConfig.goindex f7cc157b8..a46b9dda1 100644@@ -228,7 +228,10 @@ func (c Config) CheckAllowedHTTPAddress(network, address string) error { 	// Only enforced under the default hardened allowlist. If the user has 	// customized security.http.urls they have opted into whatever hosts they 	// listed, including internal ones (e.g. a local dev server), so we do not-	// second-guess the resolved address.+	// second-guess the resolved address. Note that any value that differs+	// from the default, even one that only adds a host to the default+	// patterns, disables this check, so users who customize+	// security.http.urls should keep their patterns narrow. 	if !slices.Equal(c.HTTP.URLs.patternsStrings, DefaultConfig.HTTP.URLs.patternsStrings) { 		return nil 	}@@ -237,6 +240,7 @@ func (c Config) CheckAllowedHTTPAddress(network, address string) error { 			name:     name, 			path:     "security.http.urls", 			policies: c.ToTOML(),+			reason:   "the URL's host resolved to a non-public (e.g. loopback, private or link-local) address, which the default policy blocks; a custom security.http.urls disables this check", 		} 	} 	host, _, err := net.SplitHostPort(address)@@ -463,9 +467,13 @@ type AccessDeniedError struct { 	path     string 	name     string 	policies string+	reason   string // optional extra explanation }  func (e *AccessDeniedError) Error() string {+	if e.reason != "" {+		return fmt.Sprintf("access denied: %q is not whitelisted in policy %q (%s); the current security configuration is:\n\n%s\n\n", e.name, e.path, e.reason, e.policies)+	} 	return fmt.Sprintf("access denied: %q is not whitelisted in policy %q; the current security configuration is:\n\n%s\n\n", e.name, e.path, e.policies) } config/security/securityConfig_test.goindex 4ac059a67..7fb1a09c3 100644@@ -296,6 +296,44 @@ func TestCheckAllowedHTTPAddress(t *testing.T) { 	} } +// The dial-time denial of non-public addresses only applies when+// security.http.urls is left at its default. Setting a custom+// security.http.urls intentionally (and as documented) disables it; the+// error message should explain this, since the URL itself passed the allowlist.+func TestCheckAllowedHTTPAddressCustomURLs(t *testing.T) {+	t.Parallel()+	c := qt.New(t)++	decode := func(tomlConfig string) Config {+		cfg, err := config.FromConfigString(tomlConfig, "toml")+		c.Assert(err, qt.IsNil)+		pc, err := DecodeConfig(cfg)+		c.Assert(err, qt.IsNil)+		return pc+	}++	// Default config: denied, with an explanatory message.+	err := DefaultConfig.CheckAllowedHTTPAddress("tcp", "169.254.169.254:80")+	c.Assert(err, qt.IsNotNil)+	c.Assert(err, qt.ErrorMatches, `(?s).*is not whitelisted in policy "security\.http\.urls".*`)+	c.Assert(err.Error(), qt.Contains, "resolved to a non-public")+	c.Assert(err.Error(), qt.Contains, "custom security.http.urls")++	// URL text denial should not carry the dial-time explanation.+	err = DefaultConfig.CheckAllowedHTTPURL("http://localhost/")+	c.Assert(err, qt.IsNotNil)+	c.Assert(err.Error(), qt.Not(qt.Contains), "resolved to a non-public")++	// Customizing other http settings keeps the default urls and the check.+	pc := decode("[security.http]\nmethods = ['(?i)GET']")+	c.Assert(pc.CheckAllowedHTTPAddress("tcp", "127.0.0.1:80"), qt.IsNotNil)++	// A custom security.http.urls disables the check.+	pc = decode("[security.http]\nurls = ['^http://localhost:1313/']")+	c.Assert(pc.CheckAllowedHTTPAddress("tcp", "127.0.0.1:1313"), qt.IsNil)+	c.Assert(pc.CheckAllowedHTTPAddress("tcp", "169.254.169.254:80"), qt.IsNil)+}+ func TestCheckAllowedHTTPURLAtInPathIssue14825(t *testing.T) { 	t.Parallel() 	c := qt.New(t)docs/content/en/configuration/security.mdindex 9efbc0148..cd5bab250 100644@@ -36,6 +36,7 @@ This is the default security configuration:  `http.urls` : (`[]string`) A slice of [regular expressions](g) matching the URLs that the `resources.GetRemote` function is allowed to access.+: With the default value, Hugo also checks the resolved IP address at connection time and refuses loopback, private, link-local (e.g., the cloud metadata endpoint `169.254.169.254`), and other non-public addresses, so a public-looking hostname that resolves to an internal address is blocked. Setting a value that differs from the default disables this resolved-address check (to allow, for example, local development servers), leaving the allowlist as the only safeguard. Keep custom patterns as narrow as possible, anchor them (e.g., `^https://api\.example\.com/`), and avoid broad patterns such as `.*` when building untrusted content.  `node.permissions.disable` : {{< new-in 0.161.0 />}} 
Merged fix · PR #+163 −4
config/security/securityConfig.goindex f7cc157b8..a2708288a 100644@@ -128,6 +128,12 @@ type HTTP struct { 	// URLs to allow in remote HTTP (resources.Get, getJSON, getCSV). 	URLs Whitelist `json:"urls"` +	// Whether to honor the HTTP_PROXY, HTTPS_PROXY and NO_PROXY environment+	// variables for remote HTTP. Off by default, as a proxy hides the+	// resolved destination address from the security policy.+	// <docsmeta>{ "newIn": "0.166.0" }</docsmeta>+	ProxyFromEnvironment bool `json:"proxyFromEnvironment"`+ 	// HTTP methods to allow. 	Methods Whitelist `json:"methods"` config/security/securityConfig_test.goindex 4ac059a67..e6f8c68e3 100644@@ -135,7 +135,7 @@ func TestToTOML(t *testing.T) { 	got := DefaultConfig.ToTOML()  	c.Assert(got, qt.Equals,-		"[security]\n  allowContent = ['! ^text/html$', '! ^text/org$']\n  enableInlineShortcodes = false\n\n  [security.exec]\n    allow = ['^(dart-)?sass$', '^go$', '^git$', '^node$', '^postcss$']\n    osEnv = ['(?i)^((HTTPS?|NO)_PROXY|PATH(EXT)?|APPDATA|TE?MP|TERM|GO\\w+|(XDG_CONFIG_)?HOME|USERPROFILE|SSH_AUTH_SOCK|DISPLAY|LANG|SYSTEMDRIVE|PROGRAMDATA)$']\n\n  [security.funcs]\n    getenv = ['^HUGO_', '^CI$']\n\n  [security.http]\n    methods = ['(?i)GET|POST']\n    urls = ['(?i)^https?://[a-z0-9]', '! (?i)^https?://\\d+\\.', '! (?i)localhost', '! (?i)^https?://[^/?#]*@']\n\n  [security.node]\n    [security.node.permissions]\n      allowAddons = ['tailwindcss']\n      allowChildProcess = ['tailwindcss']\n      allowRead = ['.']\n      allowWorker = ['tailwindcss']\n      allowWrite = []\n      disable = false",+		"[security]\n  allowContent = ['! ^text/html$', '! ^text/org$']\n  enableInlineShortcodes = false\n\n  [security.exec]\n    allow = ['^(dart-)?sass$', '^go$', '^git$', '^node$', '^postcss$']\n    osEnv = ['(?i)^((HTTPS?|NO)_PROXY|PATH(EXT)?|APPDATA|TE?MP|TERM|GO\\w+|(XDG_CONFIG_)?HOME|USERPROFILE|SSH_AUTH_SOCK|DISPLAY|LANG|SYSTEMDRIVE|PROGRAMDATA)$']\n\n  [security.funcs]\n    getenv = ['^HUGO_', '^CI$']\n\n  [security.http]\n    methods = ['(?i)GET|POST']\n    proxyFromEnvironment = false\n    urls = ['(?i)^https?://[a-z0-9]', '! (?i)^https?://\\d+\\.', '! (?i)localhost', '! (?i)^https?://[^/?#]*@']\n\n  [security.node]\n    [security.node.permissions]\n      allowAddons = ['tailwindcss']\n      allowChildProcess = ['tailwindcss']\n      allowRead = ['.']\n      allowWorker = ['tailwindcss']\n      allowWrite = []\n      disable = false", 	) } @@ -156,6 +156,7 @@ func TestDecodeConfigDefault(t *testing.T) { 	c.Assert(pc.HTTP.Methods.Accept("get"), qt.IsTrue) 	c.Assert(pc.HTTP.Methods.Accept("DELETE"), qt.IsFalse) 	c.Assert(pc.HTTP.MediaTypes.Accept("application/msword"), qt.IsFalse)+	c.Assert(pc.HTTP.ProxyFromEnvironment, qt.IsFalse)  	c.Assert(pc.Exec.OsEnv.Accept("PATH"), qt.IsTrue) 	c.Assert(pc.Exec.OsEnv.Accept("GOROOT"), qt.IsTrue)docs/content/en/configuration/security.mdindex 9efbc0148..54ebb4521 100644@@ -34,9 +34,15 @@ This is the default security configuration: `http.mediaTypes` : (`[]string`) Applicable to the `resources.GetRemote` function, a slice of [regular expressions](g) matching the `Content-Type` in HTTP responses that Hugo trusts, bypassing file content analysis for media type detection. +`http.proxyFromEnvironment`+: {{< new-in 0.166.0 />}}+: (`bool`) Whether the `resources.GetRemote` function honors the `HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY` environment variables. Default is `false`. When a proxy is used, Hugo connects to the proxy rather than the destination, so the resolved address validation described under `http.urls` does not apply.+ `http.urls` : (`[]string`) A slice of [regular expressions](g) matching the URLs that the `resources.GetRemote` function is allowed to access. +  The default allowlist denies URLs with an IP address or `localhost` as the host name. In addition, with the default allowlist, Hugo validates the resolved address when connecting and rejects connections to loopback, private, link-local, and other non-public addresses, e.g. a host name that resolves to a cloud metadata endpoint. This validation is disabled if you override `http.urls`, as the override may intentionally allow access to hosts on your local network, such as a development server.+ `node.permissions.disable` : {{< new-in 0.161.0 />}} : (`bool`) Whether to disable the Node.js [permission model][]. When `false`, Hugo runs Node.js tools with the `--permission` flag, restricting their file system and resource access to what is explicitly allowed below. Default is `false`.docs/data/docs.yamlindex b392c6a71..bf63fb6b0 100644@@ -2240,6 +2240,7 @@ config:       mediaTypes: null       methods:       - (?i)GET|POST+      proxyFromEnvironment: false       urls:       - (?i)^https?://[a-z0-9]       - '! (?i)^https?://\d+\.'resources/resource_factories/create/create.goindex a34f9fc7b..1570c2648 100644@@ -16,9 +16,11 @@ package create  import (+	"context" 	"errors" 	"net" 	"net/http"+	"net/url" 	"os" 	"path" 	"path/filepath"@@ -26,6 +28,8 @@ import ( 	"syscall" 	"time" +	"golang.org/x/net/http/httpproxy"+ 	"github.com/bep/helpers/contexthelpers"  	"github.com/bep/logg"@@ -65,6 +69,7 @@ type contextKey uint8  const ( 	contextKeyResourceID contextKey = iota+	contextKeyProxyURL )  // New creates a new Client with the given specification.@@ -154,17 +159,80 @@ func New(rs *resources.Spec) *Client { // hook that validates the resolved destination address against the security // policy, so a hostname resolving to an internal address cannot be used to // reach internal endpoints. See CheckAllowedHTTPAddress.+//+// Proxies are only used when security.http.proxyFromEnvironment is enabled.+// A proxied connection is dialed to the proxy, not the destination, so the+// address check is skipped for that dial; the user has opted into trusting+// the proxy with the destination. The proxy decision is carried per request+// in its context, so a direct dial is always validated, even to the proxy's+// own address. func newSecureBaseTransport(sec security.Config) http.RoundTripper { 	base := http.DefaultTransport.(*http.Transport).Clone()-	d := &net.Dialer{+	secureDialer := &net.Dialer{ 		Timeout:   30 * time.Second, 		KeepAlive: 30 * time.Second, 		Control: func(network, address string, _ syscall.RawConn) error { 			return sec.CheckAllowedHTTPAddress(network, address) 		}, 	}-	base.DialContext = d.DialContext-	return base+	base.Proxy = nil+	base.DialContext = secureDialer.DialContext++	if !sec.HTTP.ProxyFromEnvironment {+		return base+	}++	proxyFromCtx := func(ctx context.Context) *url.URL {+		u, _ := ctx.Value(contextKeyProxyURL).(*url.URL)+		return u+	}+	base.Proxy = func(req *http.Request) (*url.URL, error) {+		return proxyFromCtx(req.Context()), nil+	}+	proxyDialer := &net.Dialer{Timeout: secureDialer.Timeout, KeepAlive: secureDialer.KeepAlive}+	base.DialContext = func(ctx context.Context, network, addr string) (net.Conn, error) {+		if u := proxyFromCtx(ctx); u != nil && canonicalAddr(u) == addr {+			return proxyDialer.DialContext(ctx, network, addr)+		}+		return secureDialer.DialContext(ctx, network, addr)+	}+	return &proxyTransport{base: base, proxyFunc: httpproxy.FromEnvironment().ProxyFunc()}+}++// proxyTransport resolves the proxy for each request up front and stores it+// in the request context, where both the Proxy and DialContext hooks of base+// can see it.+type proxyTransport struct {+	base      *http.Transport+	proxyFunc func(*url.URL) (*url.URL, error)+}++func (t *proxyTransport) RoundTrip(req *http.Request) (*http.Response, error) {+	u, err := t.proxyFunc(req.URL)+	if err != nil {+		return nil, err+	}+	if u != nil {+		req = req.WithContext(context.WithValue(req.Context(), contextKeyProxyURL, u))+	}+	return t.base.RoundTrip(req)+}++// canonicalAddr mirrors net/http's canonicalAddr: the host:port the transport+// dials for u, with the port defaulted from the scheme.+func canonicalAddr(u *url.URL) string {+	port := u.Port()+	if port == "" {+		switch u.Scheme {+		case "http":+			port = "80"+		case "https":+			port = "443"+		case "socks5", "socks5h":+			port = "1080"+		}+	}+	return net.JoinHostPort(u.Hostname(), port) }  // Copy copies r to the new targetPath.resources/resource_factories/create/remote_test.goindex f9830c842..d7bb98361 100644@@ -14,8 +14,10 @@ package create  import (+	"io" 	"net/http" 	"net/http/httptest"+	"sync/atomic" 	"testing"  	qt "github.com/frankban/quicktest"@@ -173,3 +175,78 @@ urls = ['.*'] 	c.Assert(err, qt.IsNil) 	resp.Body.Close() }++// Proxies from the environment are ignored unless+// security.http.proxyFromEnvironment is enabled. When enabled, the dial-time+// address check does not apply to the proxy itself. See issue 15302.+func TestSecureBaseTransportProxyFromEnvironment(t *testing.T) {+	c := qt.New(t)++	var proxyHits, targetHits atomic.Int32+	proxy := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {+		proxyHits.Add(1)+		w.Write([]byte("via proxy"))+	}))+	t.Cleanup(proxy.Close)+	target := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {+		targetHits.Add(1)+		w.Write([]byte("direct"))+	}))+	t.Cleanup(target.Close)++	t.Setenv("HTTP_PROXY", proxy.URL)+	t.Setenv("NO_PROXY", "")+	t.Setenv("no_proxy", "")++	decode := func(toml string) security.Config {+		sec, err := security.DecodeConfig(config.FromTOMLConfigString(toml))+		c.Assert(err, qt.IsNil)+		return sec+	}+	fetch := func(sec security.Config, u string) (string, error) {+		req, err := http.NewRequest("GET", u, nil)+		c.Assert(err, qt.IsNil)+		resp, err := newSecureBaseTransport(sec).RoundTrip(req)+		if err != nil {+			return "", err+		}+		defer resp.Body.Close()+		b, err := io.ReadAll(resp.Body)+		c.Assert(err, qt.IsNil)+		return string(b), nil+	}++	// Default: proxy ignored, fetched directly.+	body, err := fetch(decode(`+[security.http]+urls = ['.*']+`), target.URL)+	c.Assert(err, qt.IsNil)+	c.Assert(body, qt.Equals, "direct")+	c.Assert(proxyHits.Load(), qt.Equals, int32(0))+	c.Assert(targetHits.Load(), qt.Equals, int32(1))++	// Enabled: the loopback proxy is dialed even under the default hardened+	// policy, and the destination is never dialed directly.+	body, err = fetch(decode(`+[security.http]+proxyFromEnvironment = true+`), "http://example.com/")+	c.Assert(err, qt.IsNil)+	c.Assert(body, qt.Equals, "via proxy")+	c.Assert(proxyHits.Load(), qt.Equals, int32(1))+	c.Assert(targetHits.Load(), qt.Equals, int32(1))++	// Enabled, but loopback destinations are never proxied: the address+	// check still applies to the direct dial, also when the destination is+	// the proxy's own address.+	for _, u := range []string{target.URL, proxy.URL} {+		_, err = fetch(decode(`+[security.http]+proxyFromEnvironment = true+`), u)+		c.Assert(security.IsAccessDenied(err), qt.IsTrue, qt.Commentf(u))+	}+	c.Assert(proxyHits.Load(), qt.Equals, int32(1))+	c.Assert(targetHits.Load(), qt.Equals, int32(1))+} 

What the agent reported

I kept the current behaviour (a custom security.http.urls still turns off the resolved-address check) and made it visible in the error message, the docs and the code comment. The new test fails without the change and passes with it, but only in my offline go test runs. ./bt-test never ran any tests: it failed three times with [setup failed] because the test box can't download Go modules (lookup proxy.golang.org … server misbehaving, later Forbidden).

What was wrong. Nothing was broken. CheckAllowedHTTPAddress blocks connections where an allowed URL resolves to a loopback, private or link-local address, but only when security.http.urls is the default. That exception wasn't documented. And when the check did block something, the error just said "169.254.169.254" is not whitelisted in policy "security.http.urls". That's confusing, because the URL itself had passed the allowlist.

What I changed

  • config/security/securityConfig.go:
  • AccessDeniedError has a new optional reason. Only the blocked-connection error sets it: "the URL's host resolved to a non-public (e.g. loopback, private or link-local) address, which the default policy blocks; a custom security.http.urls disables this check".
  • Every other denial message (exec, getenv, URL text, methods, allowContent) is unchanged.
  • The code comment now says that any value that differs from the default, even one that only adds a host, turns the check off, so custom patterns should stay narrow.
  • docs/content/en/configuration/security.md: the http.urls entry now explains:
  • With the default value, Hugo refuses to connect when an allowed URL's host resolves to a non-public address (e.g. 169.254.169.254).
  • Changing the setting turns that check off, leaving the allowlist as the only protection.
  • Keep patterns anchored and narrow, and avoid .* when building untrusted content.
  • config/security/securityConfig_test.go: new test TestCheckAllowedHTTPAddressCustomURLs. It checks that the default config blocks 169.254.169.254:80 with the new explanation, and that URL-text denials don't get it. It also checks that changing only http.methods keeps the check, while a custom urls = ['^http://localhost:1313/'] lets 127.0.0.1:1313 and 169.254.169.254:80 through.

Test evidence. I ran go test directly with GOMODCACHE=/bt/cache/go-mod GOPROXY=off, a module cache I found on the machine:

  • Before the change: the new test fails with no substring match found … want: "resolved to a non-public".
  • After the change: ok github.com/gohugoio/hugo/config/security (all tests), and ok github.com/gohugoio/hugo/hugolib for -run TestSecurity, the other tests that match on the error text.
  • resources/resource_factories/create: TestGetRemoteHead and TestGetRemoteResponseHeaders fail both with and without my change, because they fetch https://gohugo.io and there's no network (Forbidden).

Requirements

AskOutcome
Document the override behaviourDone: the docs page and the code comment.
Make it more visibleDone: the blocked-connection error explains the check and the override, covered by the new test.
"See if this can be improved further"Kept the behaviour as the maintainer wanted, because blocking private addresses under any custom list would break local dev servers. The new test pins the override (a custom urls lets private addresses through; changing only methods does not).
No unrelated changesThe reviewer found no unrelated changes in these three files; I fixed its wording points and renamed the test. AGENTS.md and CLAUDE.md were already deleted in the working tree before I started, and I left them alone.

Nothing is committed.