Backtestshugorun-20260922-141813#15301

Issue #15301 Merged fix PR # Run 22 Sep 2026 · run-20260922-141813 gohugoio/hugo

Improve/document security.http.urls override behavior

Better

Our patch was judged better than the merged fix.

In this judgment, Patch B = our agent, Patch A = the merged fix.

The issue is explicitly a documentation/visibility ticket: the maintainer states the current urls-override skip is 'the pragmatic correct approach' and asks to 'see if this can be improved further (e.g. better documentation)'. It does not request a behavior change, and it is scoped specifically to the security.http.urls override. Patch B answers that ask precisely — a dedicated, well-organized 'Remote resource address validation' section that explains the resolved-address check, exactly which ranges are rejected (loopback/private/link-local/CGNAT/reserved), and why the check is deliberately skipped on override with the user assuming responsibility, plus a cross-link from the http.urls entry. It is minimal, on-target, and low-risk. Patch A documents the same behavior more briefly but pivots into a substantial new feature — a `security.http.proxyFromEnvironment` config field, a proxyTransport wrapper, and a per-request-context proxy decision — that addresses a *different* vector (env-proxy bypass) the issue never raises. That is a real but adjacent hardening that adds public API surface and changes the default (env proxies now off), which cuts against both the issue's stated scope and the maintainer's explicit position that current behavior is correct; per the scope criterion this counts against A. B's weakness is that its test is a characterization test of pre-existing behavior (it would pass with or without the docs change, so it 'counts for little'), whereas A's proxy test is a genuine fail-before/pass-after test. But that stronger test only exercises A's out-of-scope feature, so it does little to resolve *this* issue. Weighing correctness against what the issue actually wants (documentation of an intentionally-correct behavior) plus scope discipline, B is the better-fitting fix; A is impressive engineering aimed partly at a different problem.

The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.

Solve

15m14 turns

Wall-clock time in the container

Full test suite

No new failures

3,750 passed · 100 skipped · 3,856 total

Contamination audit

Unknown

See the audit notes for this run

Patch size

38vs 167 merged

Changed lines, ours vs merged

In d6e6f9e500eebdeae8e28de830fff2e3bfc7d534 (unreleased) we added a dial-time URL validation to catch URLs that's valid according to the whitelist but that resolves to a loopback, private or link-local IP addresses. That check is skipped it if the user has a custom security.http.urls setup, as it would not make sense to block local development servers.

See CheckAllowedHTTPAddress.

This has been reported by @Reload3d as a potential security issue, and while I think the current behaviour is the pragmatic correct approach, I'm creating this issue for visibility and to see if this can be improved further (e.g. better documentation).

The two patches

Our patch+37 −1
config/security/securityConfig_test.goindex 4ac059a67..812c66ee5 100644@@ -296,6 +296,34 @@ func TestCheckAllowedHTTPAddress(t *testing.T) { 	} } +// When the user customizes security.http.urls they opt into whatever hosts+// they listed (including internal ones such as a local dev server), so the+// resolved-address check must be skipped and CheckAllowedHTTPAddress must+// return nil even for addresses the default hardened config denies.+// See issue 15301.+func TestCheckAllowedHTTPAddressSkippedWhenURLsCustomizedIssue15301(t *testing.T) {+	t.Parallel()+	c := qt.New(t)++	tomlConfig := `+[security.http]+urls = ['.*']+`+	cfg, err := config.FromConfigString(tomlConfig, "toml")+	c.Assert(err, qt.IsNil)+	pc, err := DecodeConfig(cfg)+	c.Assert(err, qt.IsNil)++	for _, addr := range []string{+		"127.0.0.1:80",+		"10.0.0.1:8080",+		"169.254.169.254:80", // Cloud metadata.+		"[::1]:80",+	} {+		c.Assert(pc.CheckAllowedHTTPAddress("tcp", addr), qt.IsNil, qt.Commentf(addr))+	}+}+ func TestCheckAllowedHTTPURLAtInPathIssue14825(t *testing.T) { 	t.Parallel() 	c := qt.New(t)docs/content/en/configuration/security.mdindex 9efbc0148..987a29c05 100644@@ -35,7 +35,7 @@ This is the default security configuration: : (`[]string`) Applicable to the `resources.GetRemote` function, a slice of [regular expressions](g) matching the `Content-Type` in HTTP responses that Hugo trusts, bypassing file content analysis for media type detection.  `http.urls`-: (`[]string`) A slice of [regular expressions](g) matching the URLs that the `resources.GetRemote` function is allowed to access.+: (`[]string`) A slice of [regular expressions](g) matching the URLs that the `resources.GetRemote` function is allowed to access. When left at its default value, Hugo applies an additional [connection-time address check](#remote-resource-address-validation).  `node.permissions.disable` : {{< new-in 0.161.0 />}}@@ -61,6 +61,14 @@ This is the default security configuration: : {{< new-in 0.161.0 />}} : (`[]string`) A slice of file system paths that Node.js tools are allowed to write (`--allow-fs-write`). Paths are relative to the working directory; `"."` means the working directory itself. Use `"*"` to allow all paths. +## Remote resource address validation++The `security.http.urls` allowlist matches only the text of a URL. On its own it cannot see the IP address that a host name ultimately resolves to, so an allowed host name could still resolve to a loopback, private, or link-local address---including the cloud metadata endpoint---and let the [`resources.GetRemote`][] function reach an internal service.++To close this gap, when `security.http.urls` is left at its default value Hugo performs an additional check at connection time: it inspects the resolved destination IP address and denies the request unless the address is publicly routable. Loopback, private, link-local, shared (CGNAT), reserved, and documentation or test ranges are all rejected.++This connection-time check applies _only_ when you use the default `security.http.urls` value. If you customize `security.http.urls`, Hugo assumes you have deliberately opted in to the hosts you listed---including internal ones such as a local development server---and skips the resolved-address check. In that case you are responsible for ensuring that the URLs you allow do not expose internal services.+ ## Negation rules  {{< new-in 0.161.0 />}} 
Merged fix · PR #+163 −4
config/security/securityConfig.goindex f7cc157b8..a2708288a 100644@@ -128,6 +128,12 @@ type HTTP struct { 	// URLs to allow in remote HTTP (resources.Get, getJSON, getCSV). 	URLs Whitelist `json:"urls"` +	// Whether to honor the HTTP_PROXY, HTTPS_PROXY and NO_PROXY environment+	// variables for remote HTTP. Off by default, as a proxy hides the+	// resolved destination address from the security policy.+	// <docsmeta>{ "newIn": "0.166.0" }</docsmeta>+	ProxyFromEnvironment bool `json:"proxyFromEnvironment"`+ 	// HTTP methods to allow. 	Methods Whitelist `json:"methods"` config/security/securityConfig_test.goindex 4ac059a67..e6f8c68e3 100644@@ -135,7 +135,7 @@ func TestToTOML(t *testing.T) { 	got := DefaultConfig.ToTOML()  	c.Assert(got, qt.Equals,-		"[security]\n  allowContent = ['! ^text/html$', '! ^text/org$']\n  enableInlineShortcodes = false\n\n  [security.exec]\n    allow = ['^(dart-)?sass$', '^go$', '^git$', '^node$', '^postcss$']\n    osEnv = ['(?i)^((HTTPS?|NO)_PROXY|PATH(EXT)?|APPDATA|TE?MP|TERM|GO\\w+|(XDG_CONFIG_)?HOME|USERPROFILE|SSH_AUTH_SOCK|DISPLAY|LANG|SYSTEMDRIVE|PROGRAMDATA)$']\n\n  [security.funcs]\n    getenv = ['^HUGO_', '^CI$']\n\n  [security.http]\n    methods = ['(?i)GET|POST']\n    urls = ['(?i)^https?://[a-z0-9]', '! (?i)^https?://\\d+\\.', '! (?i)localhost', '! (?i)^https?://[^/?#]*@']\n\n  [security.node]\n    [security.node.permissions]\n      allowAddons = ['tailwindcss']\n      allowChildProcess = ['tailwindcss']\n      allowRead = ['.']\n      allowWorker = ['tailwindcss']\n      allowWrite = []\n      disable = false",+		"[security]\n  allowContent = ['! ^text/html$', '! ^text/org$']\n  enableInlineShortcodes = false\n\n  [security.exec]\n    allow = ['^(dart-)?sass$', '^go$', '^git$', '^node$', '^postcss$']\n    osEnv = ['(?i)^((HTTPS?|NO)_PROXY|PATH(EXT)?|APPDATA|TE?MP|TERM|GO\\w+|(XDG_CONFIG_)?HOME|USERPROFILE|SSH_AUTH_SOCK|DISPLAY|LANG|SYSTEMDRIVE|PROGRAMDATA)$']\n\n  [security.funcs]\n    getenv = ['^HUGO_', '^CI$']\n\n  [security.http]\n    methods = ['(?i)GET|POST']\n    proxyFromEnvironment = false\n    urls = ['(?i)^https?://[a-z0-9]', '! (?i)^https?://\\d+\\.', '! (?i)localhost', '! (?i)^https?://[^/?#]*@']\n\n  [security.node]\n    [security.node.permissions]\n      allowAddons = ['tailwindcss']\n      allowChildProcess = ['tailwindcss']\n      allowRead = ['.']\n      allowWorker = ['tailwindcss']\n      allowWrite = []\n      disable = false", 	) } @@ -156,6 +156,7 @@ func TestDecodeConfigDefault(t *testing.T) { 	c.Assert(pc.HTTP.Methods.Accept("get"), qt.IsTrue) 	c.Assert(pc.HTTP.Methods.Accept("DELETE"), qt.IsFalse) 	c.Assert(pc.HTTP.MediaTypes.Accept("application/msword"), qt.IsFalse)+	c.Assert(pc.HTTP.ProxyFromEnvironment, qt.IsFalse)  	c.Assert(pc.Exec.OsEnv.Accept("PATH"), qt.IsTrue) 	c.Assert(pc.Exec.OsEnv.Accept("GOROOT"), qt.IsTrue)docs/content/en/configuration/security.mdindex 9efbc0148..54ebb4521 100644@@ -34,9 +34,15 @@ This is the default security configuration: `http.mediaTypes` : (`[]string`) Applicable to the `resources.GetRemote` function, a slice of [regular expressions](g) matching the `Content-Type` in HTTP responses that Hugo trusts, bypassing file content analysis for media type detection. +`http.proxyFromEnvironment`+: {{< new-in 0.166.0 />}}+: (`bool`) Whether the `resources.GetRemote` function honors the `HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY` environment variables. Default is `false`. When a proxy is used, Hugo connects to the proxy rather than the destination, so the resolved address validation described under `http.urls` does not apply.+ `http.urls` : (`[]string`) A slice of [regular expressions](g) matching the URLs that the `resources.GetRemote` function is allowed to access. +  The default allowlist denies URLs with an IP address or `localhost` as the host name. In addition, with the default allowlist, Hugo validates the resolved address when connecting and rejects connections to loopback, private, link-local, and other non-public addresses, e.g. a host name that resolves to a cloud metadata endpoint. This validation is disabled if you override `http.urls`, as the override may intentionally allow access to hosts on your local network, such as a development server.+ `node.permissions.disable` : {{< new-in 0.161.0 />}} : (`bool`) Whether to disable the Node.js [permission model][]. When `false`, Hugo runs Node.js tools with the `--permission` flag, restricting their file system and resource access to what is explicitly allowed below. Default is `false`.docs/data/docs.yamlindex b392c6a71..bf63fb6b0 100644@@ -2240,6 +2240,7 @@ config:       mediaTypes: null       methods:       - (?i)GET|POST+      proxyFromEnvironment: false       urls:       - (?i)^https?://[a-z0-9]       - '! (?i)^https?://\d+\.'resources/resource_factories/create/create.goindex a34f9fc7b..1570c2648 100644@@ -16,9 +16,11 @@ package create  import (+	"context" 	"errors" 	"net" 	"net/http"+	"net/url" 	"os" 	"path" 	"path/filepath"@@ -26,6 +28,8 @@ import ( 	"syscall" 	"time" +	"golang.org/x/net/http/httpproxy"+ 	"github.com/bep/helpers/contexthelpers"  	"github.com/bep/logg"@@ -65,6 +69,7 @@ type contextKey uint8  const ( 	contextKeyResourceID contextKey = iota+	contextKeyProxyURL )  // New creates a new Client with the given specification.@@ -154,17 +159,80 @@ func New(rs *resources.Spec) *Client { // hook that validates the resolved destination address against the security // policy, so a hostname resolving to an internal address cannot be used to // reach internal endpoints. See CheckAllowedHTTPAddress.+//+// Proxies are only used when security.http.proxyFromEnvironment is enabled.+// A proxied connection is dialed to the proxy, not the destination, so the+// address check is skipped for that dial; the user has opted into trusting+// the proxy with the destination. The proxy decision is carried per request+// in its context, so a direct dial is always validated, even to the proxy's+// own address. func newSecureBaseTransport(sec security.Config) http.RoundTripper { 	base := http.DefaultTransport.(*http.Transport).Clone()-	d := &net.Dialer{+	secureDialer := &net.Dialer{ 		Timeout:   30 * time.Second, 		KeepAlive: 30 * time.Second, 		Control: func(network, address string, _ syscall.RawConn) error { 			return sec.CheckAllowedHTTPAddress(network, address) 		}, 	}-	base.DialContext = d.DialContext-	return base+	base.Proxy = nil+	base.DialContext = secureDialer.DialContext++	if !sec.HTTP.ProxyFromEnvironment {+		return base+	}++	proxyFromCtx := func(ctx context.Context) *url.URL {+		u, _ := ctx.Value(contextKeyProxyURL).(*url.URL)+		return u+	}+	base.Proxy = func(req *http.Request) (*url.URL, error) {+		return proxyFromCtx(req.Context()), nil+	}+	proxyDialer := &net.Dialer{Timeout: secureDialer.Timeout, KeepAlive: secureDialer.KeepAlive}+	base.DialContext = func(ctx context.Context, network, addr string) (net.Conn, error) {+		if u := proxyFromCtx(ctx); u != nil && canonicalAddr(u) == addr {+			return proxyDialer.DialContext(ctx, network, addr)+		}+		return secureDialer.DialContext(ctx, network, addr)+	}+	return &proxyTransport{base: base, proxyFunc: httpproxy.FromEnvironment().ProxyFunc()}+}++// proxyTransport resolves the proxy for each request up front and stores it+// in the request context, where both the Proxy and DialContext hooks of base+// can see it.+type proxyTransport struct {+	base      *http.Transport+	proxyFunc func(*url.URL) (*url.URL, error)+}++func (t *proxyTransport) RoundTrip(req *http.Request) (*http.Response, error) {+	u, err := t.proxyFunc(req.URL)+	if err != nil {+		return nil, err+	}+	if u != nil {+		req = req.WithContext(context.WithValue(req.Context(), contextKeyProxyURL, u))+	}+	return t.base.RoundTrip(req)+}++// canonicalAddr mirrors net/http's canonicalAddr: the host:port the transport+// dials for u, with the port defaulted from the scheme.+func canonicalAddr(u *url.URL) string {+	port := u.Port()+	if port == "" {+		switch u.Scheme {+		case "http":+			port = "80"+		case "https":+			port = "443"+		case "socks5", "socks5h":+			port = "1080"+		}+	}+	return net.JoinHostPort(u.Hostname(), port) }  // Copy copies r to the new targetPath.resources/resource_factories/create/remote_test.goindex f9830c842..d7bb98361 100644@@ -14,8 +14,10 @@ package create  import (+	"io" 	"net/http" 	"net/http/httptest"+	"sync/atomic" 	"testing"  	qt "github.com/frankban/quicktest"@@ -173,3 +175,78 @@ urls = ['.*'] 	c.Assert(err, qt.IsNil) 	resp.Body.Close() }++// Proxies from the environment are ignored unless+// security.http.proxyFromEnvironment is enabled. When enabled, the dial-time+// address check does not apply to the proxy itself. See issue 15302.+func TestSecureBaseTransportProxyFromEnvironment(t *testing.T) {+	c := qt.New(t)++	var proxyHits, targetHits atomic.Int32+	proxy := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {+		proxyHits.Add(1)+		w.Write([]byte("via proxy"))+	}))+	t.Cleanup(proxy.Close)+	target := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {+		targetHits.Add(1)+		w.Write([]byte("direct"))+	}))+	t.Cleanup(target.Close)++	t.Setenv("HTTP_PROXY", proxy.URL)+	t.Setenv("NO_PROXY", "")+	t.Setenv("no_proxy", "")++	decode := func(toml string) security.Config {+		sec, err := security.DecodeConfig(config.FromTOMLConfigString(toml))+		c.Assert(err, qt.IsNil)+		return sec+	}+	fetch := func(sec security.Config, u string) (string, error) {+		req, err := http.NewRequest("GET", u, nil)+		c.Assert(err, qt.IsNil)+		resp, err := newSecureBaseTransport(sec).RoundTrip(req)+		if err != nil {+			return "", err+		}+		defer resp.Body.Close()+		b, err := io.ReadAll(resp.Body)+		c.Assert(err, qt.IsNil)+		return string(b), nil+	}++	// Default: proxy ignored, fetched directly.+	body, err := fetch(decode(`+[security.http]+urls = ['.*']+`), target.URL)+	c.Assert(err, qt.IsNil)+	c.Assert(body, qt.Equals, "direct")+	c.Assert(proxyHits.Load(), qt.Equals, int32(0))+	c.Assert(targetHits.Load(), qt.Equals, int32(1))++	// Enabled: the loopback proxy is dialed even under the default hardened+	// policy, and the destination is never dialed directly.+	body, err = fetch(decode(`+[security.http]+proxyFromEnvironment = true+`), "http://example.com/")+	c.Assert(err, qt.IsNil)+	c.Assert(body, qt.Equals, "via proxy")+	c.Assert(proxyHits.Load(), qt.Equals, int32(1))+	c.Assert(targetHits.Load(), qt.Equals, int32(1))++	// Enabled, but loopback destinations are never proxied: the address+	// check still applies to the direct dial, also when the destination is+	// the proxy's own address.+	for _, u := range []string{target.URL, proxy.URL} {+		_, err = fetch(decode(`+[security.http]+proxyFromEnvironment = true+`), u)+		c.Assert(security.IsAccessDenied(err), qt.IsTrue, qt.Commentf(u))+	}+	c.Assert(proxyHits.Load(), qt.Equals, int32(1))+	c.Assert(targetHits.Load(), qt.Equals, int32(1))+} 

What the agent reported

Done. Here's the summary.

What the report asked for

Issue 15301 is a visibility/documentation ticket, not a behavior-change request. An unreleased commit added a dial-time IP validation (CheckAllowedHTTPAddress in config/security/securityConfig.go) that denies URLs which pass the security.http.urls text allowlist but resolve to loopback/private/link-local IPs. That check is deliberately skipped when the user customizes security.http.urls (so local dev servers aren't blocked). The maintainer considers the current behavior the pragmatic correct approach and explicitly does not want it changed — the ask is "better documentation."

What was wrong

Investigation (via explorer) confirmed the behavior was correct and working, but entirely undocumented for end users: docs/content/en/configuration/security.md never mentioned the connection-time address check or the customize-to-opt-out behavior, and the skip-when-customized branch (securityConfig.go:232) had no dedicated test (existing TestCheckAllowedHTTPAddress only covers the default-config enforced path).

What I changed

  1. docs/content/en/configuration/security.md — expanded the http.urls entry with a cross-reference and added a new "Remote resource address validation" section that documents (a) why the URL-text allowlist alone can be bypassed by a hostname resolving to an internal IP, (b) the connection-time IP check applied under the default config, and (c) that customizing security.http.urls opts out of that check and shifts responsibility to the user.
  2. config/security/securityConfig_test.go — added TestCheckAllowedHTTPAddressSkippedWhenURLsCustomizedIssue15301, a regression test that decodes a config with urls = ['.*'] and asserts CheckAllowedHTTPAddress returns nil for internal addresses (127.0.0.1, 10.0.0.1, 169.254.169.254, [::1]) that the default config denies — locking in the documented override contract.

No production code was changed (behavior must stay as-is, per the report).

Test evidence

  • ./bt-test ./config/security/... -run 'TestCheckAllowedHTTPAddress' → EXIT=0. The new test and the pre-existing TestCheckAllowedHTTPAddress both pass.
  • Reviewer confirmed the doc accurately describes the code, the anchor/reference links resolve, the test genuinely proves the bypass (addresses chosen are ones the default config denies), and nothing unrelated changed. I applied its one wording-precision fix (the check rejects several global-unicast-but-not-public ranges, so the doc now says "publicly routable" and enumerates the rejected ranges rather than equating public with "global unicast").

Honest note on the test: because this is a documentation-only ticket with no behavior change, the added test is a characterization/regression test — it passes both before and after, since it codifies existing (now-documented) behavior. Per the harness rules I'm disclosing this rather than presenting it as a fail-before/pass-after test; there is no behavioral fix for such a test to gate. I judged it worth shipping because it exercises the exact scenario the report describes and guards the documented contract against future silent regression.

Requirements checklist

  • "Better documentation" of the security.http.urls override behavior → Implemented. New docs section + cross-reference in security.md, verified accurate against the code by the reviewer.
  • Visibility of the skip-when-customized behavior (the reported concern) → Implemented + test-guarded. Documented for end users and covered by the new regression test.
  • Change the skip behavior itself → Intentionally skipped. The report states the behavior is the pragmatic correct approach and should not change; evidence: securityConfig.go is unmodified in the diff (git diff shows only the docs and test files).