Backtestshugorun-20260917-015623#15301
Improve/document security.http.urls override behavior
The merged fix was judged better than our patch.
In this judgment, Patch A = our agent, Patch B = the merged fix.
Both patches document the http.urls override behavior well, and neither changes the override-skip itself (correct, since the maintainer calls that behavior pragmatically right). The issue explicitly invites 'see if this can be improved further,' and Patch B answers that: it identifies and closes a genuine bypass of the *same* SSRF-to-internal threat class the issue is about. Because newSecureBaseTransport clones http.DefaultTransport (which honors ProxyFromEnvironment), a set HTTP_PROXY silently routes GetRemote through the proxy, so CheckAllowedHTTPAddress only validates the proxy dial and never the true destination — a real hole the dial-time check left open. Patch B disables env proxies by default (base.Proxy=nil, all dials validated), gates them behind a new documented, versioned proxyFromEnvironment flag, and carries the proxy decision per-request in context so direct dials — even to the proxy's own address — are still validated. It backs this with substantive tests that fail-before/pass-after: default ignores the proxy and fetches directly, enabled dials the loopback proxy under the hardened policy, and enabled-but-direct destinations (including the proxy's own IP) are still denied. It also updates the serialization tests (ToTOML/docs.yaml) consistently. Patch A is clean, minimal, and exactly scoped, and its docs are arguably slightly richer prose, but it is documentation-only: it adds nothing to the codebase's actual defenses and no tests. Weighing correctness first, B resolves more of the root security concern with verifiable tests; its costs are real but bounded — larger scope and a behavioral change (proxies off by default, potentially breaking corporate builds) — which narrows the margin but does not overcome B's substantive, tested security improvement in the same threat domain.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
4m 52s1 turn
Wall-clock time in the container
Full test suite
No new failures
3,749 passed · 100 skipped · 3,855 total
Contamination audit
Unknown
See the audit notes for this run
Patch size
8vs 167 merged
Changed lines, ours vs merged
The issue
View on GitHubIn d6e6f9e500eebdeae8e28de830fff2e3bfc7d534 (unreleased) we added a dial-time URL validation to catch URLs that's valid according to the whitelist but that resolves to a loopback, private or link-local IP addresses. That check is skipped it if the user has a custom security.http.urls setup, as it would not make sense to block local development servers.
See CheckAllowedHTTPAddress.
This has been reported by @Reload3d as a potential security issue, and while I think the current behaviour is the pragmatic correct approach, I'm creating this issue for visibility and to see if this can be improved further (e.g. better documentation).
The two patches
docs/content/en/configuration/security.mdindex 9efbc0148..56eaabe58 100644@@ -35,7 +35,7 @@ This is the default security configuration: : (`[]string`) Applicable to the `resources.GetRemote` function, a slice of [regular expressions](g) matching the `Content-Type` in HTTP responses that Hugo trusts, bypassing file content analysis for media type detection. `http.urls`-: (`[]string`) A slice of [regular expressions](g) matching the URLs that the `resources.GetRemote` function is allowed to access.+: (`[]string`) A slice of [regular expressions](g) matching the URLs that the `resources.GetRemote` function is allowed to access. When left at its default value, Hugo also performs an additional [remote address validation](#remote-address-validation) check at connection time; customizing this setting disables that check. `node.permissions.disable` : {{< new-in 0.161.0 />}}@@ -61,6 +61,12 @@ This is the default security configuration: : {{< new-in 0.161.0 />}} : (`[]string`) A slice of file system paths that Node.js tools are allowed to write (`--allow-fs-write`). Paths are relative to the working directory; `"."` means the working directory itself. Use `"*"` to allow all paths. +## Remote address validation++When `http.urls` is left at its default value, Hugo performs an additional security check when the `resources.GetRemote` function connects to a remote host. Because the `http.urls` allowlist matches only the URL text, a hostname that passes the allowlist could still resolve, through DNS, to an internal address. To prevent this class of [server-side request forgery](https://owasp.org/www-community/attacks/Server_Side_Request_Forgery), Hugo inspects the resolved connection address at dial time and rejects any request that targets a loopback, private, link-local, or otherwise non-public address, including the cloud provider metadata endpoint.++This check is only active while `http.urls` is left at its default value. If you customize `http.urls`, Hugo assumes that you have deliberately opted into the hosts that you listed, including any internal ones such as a local development server, and the resolved-address check is skipped. If you need `resources.GetRemote` to reach an internal address, add the relevant pattern to `http.urls`.+ ## Negation rules {{< new-in 0.161.0 />}}
config/security/securityConfig.goindex f7cc157b8..a2708288a 100644@@ -128,6 +128,12 @@ type HTTP struct { // URLs to allow in remote HTTP (resources.Get, getJSON, getCSV). URLs Whitelist `json:"urls"` + // Whether to honor the HTTP_PROXY, HTTPS_PROXY and NO_PROXY environment+ // variables for remote HTTP. Off by default, as a proxy hides the+ // resolved destination address from the security policy.+ // <docsmeta>{ "newIn": "0.166.0" }</docsmeta>+ ProxyFromEnvironment bool `json:"proxyFromEnvironment"`+ // HTTP methods to allow. Methods Whitelist `json:"methods"` config/security/securityConfig_test.goindex 4ac059a67..e6f8c68e3 100644@@ -135,7 +135,7 @@ func TestToTOML(t *testing.T) { got := DefaultConfig.ToTOML() c.Assert(got, qt.Equals,- "[security]\n allowContent = ['! ^text/html$', '! ^text/org$']\n enableInlineShortcodes = false\n\n [security.exec]\n allow = ['^(dart-)?sass$', '^go$', '^git$', '^node$', '^postcss$']\n osEnv = ['(?i)^((HTTPS?|NO)_PROXY|PATH(EXT)?|APPDATA|TE?MP|TERM|GO\\w+|(XDG_CONFIG_)?HOME|USERPROFILE|SSH_AUTH_SOCK|DISPLAY|LANG|SYSTEMDRIVE|PROGRAMDATA)$']\n\n [security.funcs]\n getenv = ['^HUGO_', '^CI$']\n\n [security.http]\n methods = ['(?i)GET|POST']\n urls = ['(?i)^https?://[a-z0-9]', '! (?i)^https?://\\d+\\.', '! (?i)localhost', '! (?i)^https?://[^/?#]*@']\n\n [security.node]\n [security.node.permissions]\n allowAddons = ['tailwindcss']\n allowChildProcess = ['tailwindcss']\n allowRead = ['.']\n allowWorker = ['tailwindcss']\n allowWrite = []\n disable = false",+ "[security]\n allowContent = ['! ^text/html$', '! ^text/org$']\n enableInlineShortcodes = false\n\n [security.exec]\n allow = ['^(dart-)?sass$', '^go$', '^git$', '^node$', '^postcss$']\n osEnv = ['(?i)^((HTTPS?|NO)_PROXY|PATH(EXT)?|APPDATA|TE?MP|TERM|GO\\w+|(XDG_CONFIG_)?HOME|USERPROFILE|SSH_AUTH_SOCK|DISPLAY|LANG|SYSTEMDRIVE|PROGRAMDATA)$']\n\n [security.funcs]\n getenv = ['^HUGO_', '^CI$']\n\n [security.http]\n methods = ['(?i)GET|POST']\n proxyFromEnvironment = false\n urls = ['(?i)^https?://[a-z0-9]', '! (?i)^https?://\\d+\\.', '! (?i)localhost', '! (?i)^https?://[^/?#]*@']\n\n [security.node]\n [security.node.permissions]\n allowAddons = ['tailwindcss']\n allowChildProcess = ['tailwindcss']\n allowRead = ['.']\n allowWorker = ['tailwindcss']\n allowWrite = []\n disable = false", ) } @@ -156,6 +156,7 @@ func TestDecodeConfigDefault(t *testing.T) { c.Assert(pc.HTTP.Methods.Accept("get"), qt.IsTrue) c.Assert(pc.HTTP.Methods.Accept("DELETE"), qt.IsFalse) c.Assert(pc.HTTP.MediaTypes.Accept("application/msword"), qt.IsFalse)+ c.Assert(pc.HTTP.ProxyFromEnvironment, qt.IsFalse) c.Assert(pc.Exec.OsEnv.Accept("PATH"), qt.IsTrue) c.Assert(pc.Exec.OsEnv.Accept("GOROOT"), qt.IsTrue)docs/content/en/configuration/security.mdindex 9efbc0148..54ebb4521 100644@@ -34,9 +34,15 @@ This is the default security configuration: `http.mediaTypes` : (`[]string`) Applicable to the `resources.GetRemote` function, a slice of [regular expressions](g) matching the `Content-Type` in HTTP responses that Hugo trusts, bypassing file content analysis for media type detection. +`http.proxyFromEnvironment`+: {{< new-in 0.166.0 />}}+: (`bool`) Whether the `resources.GetRemote` function honors the `HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY` environment variables. Default is `false`. When a proxy is used, Hugo connects to the proxy rather than the destination, so the resolved address validation described under `http.urls` does not apply.+ `http.urls` : (`[]string`) A slice of [regular expressions](g) matching the URLs that the `resources.GetRemote` function is allowed to access. + The default allowlist denies URLs with an IP address or `localhost` as the host name. In addition, with the default allowlist, Hugo validates the resolved address when connecting and rejects connections to loopback, private, link-local, and other non-public addresses, e.g. a host name that resolves to a cloud metadata endpoint. This validation is disabled if you override `http.urls`, as the override may intentionally allow access to hosts on your local network, such as a development server.+ `node.permissions.disable` : {{< new-in 0.161.0 />}} : (`bool`) Whether to disable the Node.js [permission model][]. When `false`, Hugo runs Node.js tools with the `--permission` flag, restricting their file system and resource access to what is explicitly allowed below. Default is `false`.docs/data/docs.yamlindex b392c6a71..bf63fb6b0 100644@@ -2240,6 +2240,7 @@ config: mediaTypes: null methods: - (?i)GET|POST+ proxyFromEnvironment: false urls: - (?i)^https?://[a-z0-9] - '! (?i)^https?://\d+\.'resources/resource_factories/create/create.goindex a34f9fc7b..1570c2648 100644@@ -16,9 +16,11 @@ package create import (+ "context" "errors" "net" "net/http"+ "net/url" "os" "path" "path/filepath"@@ -26,6 +28,8 @@ import ( "syscall" "time" + "golang.org/x/net/http/httpproxy"+ "github.com/bep/helpers/contexthelpers" "github.com/bep/logg"@@ -65,6 +69,7 @@ type contextKey uint8 const ( contextKeyResourceID contextKey = iota+ contextKeyProxyURL ) // New creates a new Client with the given specification.@@ -154,17 +159,80 @@ func New(rs *resources.Spec) *Client { // hook that validates the resolved destination address against the security // policy, so a hostname resolving to an internal address cannot be used to // reach internal endpoints. See CheckAllowedHTTPAddress.+//+// Proxies are only used when security.http.proxyFromEnvironment is enabled.+// A proxied connection is dialed to the proxy, not the destination, so the+// address check is skipped for that dial; the user has opted into trusting+// the proxy with the destination. The proxy decision is carried per request+// in its context, so a direct dial is always validated, even to the proxy's+// own address. func newSecureBaseTransport(sec security.Config) http.RoundTripper { base := http.DefaultTransport.(*http.Transport).Clone()- d := &net.Dialer{+ secureDialer := &net.Dialer{ Timeout: 30 * time.Second, KeepAlive: 30 * time.Second, Control: func(network, address string, _ syscall.RawConn) error { return sec.CheckAllowedHTTPAddress(network, address) }, }- base.DialContext = d.DialContext- return base+ base.Proxy = nil+ base.DialContext = secureDialer.DialContext++ if !sec.HTTP.ProxyFromEnvironment {+ return base+ }++ proxyFromCtx := func(ctx context.Context) *url.URL {+ u, _ := ctx.Value(contextKeyProxyURL).(*url.URL)+ return u+ }+ base.Proxy = func(req *http.Request) (*url.URL, error) {+ return proxyFromCtx(req.Context()), nil+ }+ proxyDialer := &net.Dialer{Timeout: secureDialer.Timeout, KeepAlive: secureDialer.KeepAlive}+ base.DialContext = func(ctx context.Context, network, addr string) (net.Conn, error) {+ if u := proxyFromCtx(ctx); u != nil && canonicalAddr(u) == addr {+ return proxyDialer.DialContext(ctx, network, addr)+ }+ return secureDialer.DialContext(ctx, network, addr)+ }+ return &proxyTransport{base: base, proxyFunc: httpproxy.FromEnvironment().ProxyFunc()}+}++// proxyTransport resolves the proxy for each request up front and stores it+// in the request context, where both the Proxy and DialContext hooks of base+// can see it.+type proxyTransport struct {+ base *http.Transport+ proxyFunc func(*url.URL) (*url.URL, error)+}++func (t *proxyTransport) RoundTrip(req *http.Request) (*http.Response, error) {+ u, err := t.proxyFunc(req.URL)+ if err != nil {+ return nil, err+ }+ if u != nil {+ req = req.WithContext(context.WithValue(req.Context(), contextKeyProxyURL, u))+ }+ return t.base.RoundTrip(req)+}++// canonicalAddr mirrors net/http's canonicalAddr: the host:port the transport+// dials for u, with the port defaulted from the scheme.+func canonicalAddr(u *url.URL) string {+ port := u.Port()+ if port == "" {+ switch u.Scheme {+ case "http":+ port = "80"+ case "https":+ port = "443"+ case "socks5", "socks5h":+ port = "1080"+ }+ }+ return net.JoinHostPort(u.Hostname(), port) } // Copy copies r to the new targetPath.resources/resource_factories/create/remote_test.goindex f9830c842..d7bb98361 100644@@ -14,8 +14,10 @@ package create import (+ "io" "net/http" "net/http/httptest"+ "sync/atomic" "testing" qt "github.com/frankban/quicktest"@@ -173,3 +175,78 @@ urls = ['.*'] c.Assert(err, qt.IsNil) resp.Body.Close() }++// Proxies from the environment are ignored unless+// security.http.proxyFromEnvironment is enabled. When enabled, the dial-time+// address check does not apply to the proxy itself. See issue 15302.+func TestSecureBaseTransportProxyFromEnvironment(t *testing.T) {+ c := qt.New(t)++ var proxyHits, targetHits atomic.Int32+ proxy := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {+ proxyHits.Add(1)+ w.Write([]byte("via proxy"))+ }))+ t.Cleanup(proxy.Close)+ target := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {+ targetHits.Add(1)+ w.Write([]byte("direct"))+ }))+ t.Cleanup(target.Close)++ t.Setenv("HTTP_PROXY", proxy.URL)+ t.Setenv("NO_PROXY", "")+ t.Setenv("no_proxy", "")++ decode := func(toml string) security.Config {+ sec, err := security.DecodeConfig(config.FromTOMLConfigString(toml))+ c.Assert(err, qt.IsNil)+ return sec+ }+ fetch := func(sec security.Config, u string) (string, error) {+ req, err := http.NewRequest("GET", u, nil)+ c.Assert(err, qt.IsNil)+ resp, err := newSecureBaseTransport(sec).RoundTrip(req)+ if err != nil {+ return "", err+ }+ defer resp.Body.Close()+ b, err := io.ReadAll(resp.Body)+ c.Assert(err, qt.IsNil)+ return string(b), nil+ }++ // Default: proxy ignored, fetched directly.+ body, err := fetch(decode(`+[security.http]+urls = ['.*']+`), target.URL)+ c.Assert(err, qt.IsNil)+ c.Assert(body, qt.Equals, "direct")+ c.Assert(proxyHits.Load(), qt.Equals, int32(0))+ c.Assert(targetHits.Load(), qt.Equals, int32(1))++ // Enabled: the loopback proxy is dialed even under the default hardened+ // policy, and the destination is never dialed directly.+ body, err = fetch(decode(`+[security.http]+proxyFromEnvironment = true+`), "http://example.com/")+ c.Assert(err, qt.IsNil)+ c.Assert(body, qt.Equals, "via proxy")+ c.Assert(proxyHits.Load(), qt.Equals, int32(1))+ c.Assert(targetHits.Load(), qt.Equals, int32(1))++ // Enabled, but loopback destinations are never proxied: the address+ // check still applies to the direct dial, also when the destination is+ // the proxy's own address.+ for _, u := range []string{target.URL, proxy.URL} {+ _, err = fetch(decode(`+[security.http]+proxyFromEnvironment = true+`), u)+ c.Assert(security.IsAccessDenied(err), qt.IsTrue, qt.Commentf(u))+ }+ c.Assert(proxyHits.Load(), qt.Equals, int32(1))+ c.Assert(targetHits.Load(), qt.Equals, int32(1))+}
What the agent reported
The reviewer confirms the change is accurate, correctly scoped, and consistent. Work is complete.
Summary
What the report flagged: The dial-time address check (CheckAllowedHTTPAddress, added in d6e6f9e) blocks connections resolving to loopback/private/link-local IPs under the default security.http.urls allowlist, but is silently skipped when a user customizes security.http.urls. The reporter deemed the behavior pragmatically correct and asked for improved visibility/documentation, not a logic change.
What I found: The Go code (config/security/securityConfig.go) already carries thorough doc comments explaining both the address check and the default-only skip logic, and the behavior is already covered by existing tests (TestCheckAllowedHTTPAddress, TestSecureBaseTransportBlocksInternalAddress, and hugolib/securitypolicies_test.go). The actual gap was user-facing documentation: docs/content/en/configuration/security.md described http.urls only as a URL allowlist and never mentioned the resolved-address check or that customizing http.urls opts out of it.
What I changed (documentation only, docs/content/en/configuration/security.md):
- Expanded the
http.urlsconfig entry to note that, at its default value, Hugo also performs a remote-address-validation check at connection time, and that customizing the setting disables that check — with an intra-page link. - Added a new
## Remote address validationsection explaining the SSRF/DNS-rebinding rationale, exactly what gets blocked (loopback, private, link-local, non-public, incl. cloud metadata endpoint), the default-only enforcement, and how to intentionally allow an internal address.
Test evidence: No production logic or tests were changed, so the existing behavior tests remain valid and no new gate applies. The reviewer verified every documented claim against the code (securityConfig.go:227-256, the skip condition at line 232, isPublicAddr at line 279, and the dial-hook wiring in create.go:153-167), confirmed the anchor slug resolves, confirmed markdown formatting consistency, and confirmed no Go/test files were touched.
Note: The working tree also shows two pre-existing, unrelated deletions from before this task — AGENTS.md and CLAUDE.md at the repo root (present in the initial git status). These are not part of my change; I left them as-is. All my edits are uncommitted, confined to the one docs file.