Backtestshugorun-20261005-133023#15302
Add config option security.http.proxyFromEnvironment = false
The merged fix was judged better than our patch.
In this judgment, Patch A = our agent, Patch B = the merged fix.
Both patches fix the default case the same way. Each adds `security.http.proxyFromEnvironment` (default false) and sets `base.Proxy = nil` unless it is enabled. With the flag off, the dial-time `CheckAllowedHTTPAddress` therefore sees the real destination again, which is the core of the issue. Both update the TOML output test, docs.yaml and the security docs. The difference is what happens when the flag is enabled. - **Patch A** sets `Proxy = http.ProxyFromEnvironment` but keeps the dial-time control hook on every dial, including the dial to the proxy. Its own docs admit that with the default `http.urls` policy, a proxy on a loopback or private address is denied. Corporate and CI proxies are very often on private addresses, so the opt-in is largely unusable in exactly the setting the issue describes. - **Patch B** makes the opt-in work as intended. It resolves the proxy per request, carries it in the request context, and skips the address check only when the dial address matches that request's proxy (`canonicalAddr`). All other direct dials, including a direct request to the proxy's own address, are still validated. This scoping is careful and keeps the hardening intact for non-proxied traffic. Tests: Patch A's transport test only checks whether `tr.Proxy` is nil or non-nil. That does fail before the fix, but it is shallow. Patch B's test is behavioral, using real httptest proxy and target servers with hit counters. It shows the proxy is ignored by default, used when enabled even under the hardened policy, and that loopback destinations, including the proxy's own URL, are still denied on direct dials. Scope and craft: Patch B is more complex. It adds a wrapper `RoundTripper`, the `x/net/httpproxy` package, and a doc paragraph on `http.urls` that goes slightly beyond the ask. However, the code is well commented and the complexity serves the feature's correctness rather than obscuring the fix. On balance, Patch B delivers a working, well-tested option, while Patch A's enabled mode is self-defeating for common proxy setups.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
1h 11m
Wall-clock time in the container
Full test suite
No new failures
3,754 passed · 100 skipped · 3,858 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
62vs 167 merged
Changed lines, ours vs merged
The issue
View on GitHubThis is a security hardening issue.
In d6e6f9e500eebdeae8e28de830fff2e3bfc7d534 (unreleased) we added a dial-time URL validation.
If proxied (e.g. HTTPS_PROXY set on users machine or in CI) the above validation will only see the proxy address. There's no workaround for that, but I suggest by adding a new security.http.proxyFromEnvironment bool config flag (default disabled).
Thanks to @Reload3d for reporting this issue.
The two patches
config/security/securityConfig.goindex f7cc157b8..0185287c5 100644@@ -133,6 +133,12 @@ type HTTP struct { // Media types where the Content-Type in the response is used instead of resolving from the file content. MediaTypes Whitelist `json:"mediaTypes"`++ // Whether to honor the HTTP_PROXY, HTTPS_PROXY and NO_PROXY environment+ // variables for remote HTTP requests. Disabled by default: when proxied,+ // the dial-time address validation (see CheckAllowedHTTPAddress) only+ // sees the proxy address and not the actual destination.+ ProxyFromEnvironment bool `json:"proxyFromEnvironment"` } // Node holds Node.js security settings.config/security/securityConfig_test.goindex 4ac059a67..0fb09824c 100644@@ -129,13 +129,32 @@ osEnv="b" }) } +func TestDecodeConfigProxyFromEnvironment(t *testing.T) {+ c := qt.New(t)++ pc, err := DecodeConfig(config.New())+ c.Assert(err, qt.IsNil)+ c.Assert(pc.HTTP.ProxyFromEnvironment, qt.IsFalse)++ cfg, err := config.FromConfigString(`+[security.http]+proxyFromEnvironment = true+`, "toml")+ c.Assert(err, qt.IsNil)+ pc, err = DecodeConfig(cfg)+ c.Assert(err, qt.IsNil)+ c.Assert(pc.HTTP.ProxyFromEnvironment, qt.IsTrue)+ // Must not alter the default URL policy.+ c.Assert(pc.HTTP.URLs.String(), qt.Equals, DefaultConfig.HTTP.URLs.String())+}+ func TestToTOML(t *testing.T) { c := qt.New(t) got := DefaultConfig.ToTOML() c.Assert(got, qt.Equals,- "[security]\n allowContent = ['! ^text/html$', '! ^text/org$']\n enableInlineShortcodes = false\n\n [security.exec]\n allow = ['^(dart-)?sass$', '^go$', '^git$', '^node$', '^postcss$']\n osEnv = ['(?i)^((HTTPS?|NO)_PROXY|PATH(EXT)?|APPDATA|TE?MP|TERM|GO\\w+|(XDG_CONFIG_)?HOME|USERPROFILE|SSH_AUTH_SOCK|DISPLAY|LANG|SYSTEMDRIVE|PROGRAMDATA)$']\n\n [security.funcs]\n getenv = ['^HUGO_', '^CI$']\n\n [security.http]\n methods = ['(?i)GET|POST']\n urls = ['(?i)^https?://[a-z0-9]', '! (?i)^https?://\\d+\\.', '! (?i)localhost', '! (?i)^https?://[^/?#]*@']\n\n [security.node]\n [security.node.permissions]\n allowAddons = ['tailwindcss']\n allowChildProcess = ['tailwindcss']\n allowRead = ['.']\n allowWorker = ['tailwindcss']\n allowWrite = []\n disable = false",+ "[security]\n allowContent = ['! ^text/html$', '! ^text/org$']\n enableInlineShortcodes = false\n\n [security.exec]\n allow = ['^(dart-)?sass$', '^go$', '^git$', '^node$', '^postcss$']\n osEnv = ['(?i)^((HTTPS?|NO)_PROXY|PATH(EXT)?|APPDATA|TE?MP|TERM|GO\\w+|(XDG_CONFIG_)?HOME|USERPROFILE|SSH_AUTH_SOCK|DISPLAY|LANG|SYSTEMDRIVE|PROGRAMDATA)$']\n\n [security.funcs]\n getenv = ['^HUGO_', '^CI$']\n\n [security.http]\n methods = ['(?i)GET|POST']\n proxyFromEnvironment = false\n urls = ['(?i)^https?://[a-z0-9]', '! (?i)^https?://\\d+\\.', '! (?i)localhost', '! (?i)^https?://[^/?#]*@']\n\n [security.node]\n [security.node.permissions]\n allowAddons = ['tailwindcss']\n allowChildProcess = ['tailwindcss']\n allowRead = ['.']\n allowWorker = ['tailwindcss']\n allowWrite = []\n disable = false", ) } docs/content/en/configuration/security.mdindex 9efbc0148..5c14546fd 100644@@ -34,6 +34,10 @@ This is the default security configuration: `http.mediaTypes` : (`[]string`) Applicable to the `resources.GetRemote` function, a slice of [regular expressions](g) matching the `Content-Type` in HTTP responses that Hugo trusts, bypassing file content analysis for media type detection. +`http.proxyFromEnvironment`+: {{< new-in 0.166.0 />}}+: (`bool`) Whether the `resources.GetRemote` function honors the `HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY` environment variables. When enabled, Hugo can only validate the address of the proxy, not the address of the final destination. With the default `http.urls` setting, a proxy on a loopback or private network address is denied. Default is `false`.+ `http.urls` : (`[]string`) A slice of [regular expressions](g) matching the URLs that the `resources.GetRemote` function is allowed to access. docs/data/docs.yamlindex e51915c49..6d2578339 100644@@ -2253,6 +2253,7 @@ config: mediaTypes: null methods: - (?i)GET|POST+ proxyFromEnvironment: false urls: - (?i)^https?://[a-z0-9] - '! (?i)^https?://\d+\.'resources/resource_factories/create/create.goindex a34f9fc7b..cc57e1457 100644@@ -154,8 +154,17 @@ func New(rs *resources.Spec) *Client { // hook that validates the resolved destination address against the security // policy, so a hostname resolving to an internal address cannot be used to // reach internal endpoints. See CheckAllowedHTTPAddress.+//+// Proxies configured via the environment (HTTP_PROXY etc.) are ignored unless+// security.http.proxyFromEnvironment is set, as the dial-time hook would then+// only see the proxy address and not the actual destination. func newSecureBaseTransport(sec security.Config) http.RoundTripper { base := http.DefaultTransport.(*http.Transport).Clone()+ if sec.HTTP.ProxyFromEnvironment {+ base.Proxy = http.ProxyFromEnvironment+ } else {+ base.Proxy = nil+ } d := &net.Dialer{ Timeout: 30 * time.Second, KeepAlive: 30 * time.Second,resources/resource_factories/create/remote_test.goindex f9830c842..cdf4995f1 100644@@ -173,3 +173,24 @@ urls = ['.*'] c.Assert(err, qt.IsNil) resp.Body.Close() }++// Proxies from the environment (HTTP_PROXY etc.) must be ignored by default,+// as the dial-time address validation would otherwise only see the proxy+// address. They are honored when security.http.proxyFromEnvironment is set.+func TestSecureBaseTransportProxyFromEnvironment(t *testing.T) {+ t.Parallel()++ c := qt.New(t)++ tr := newSecureBaseTransport(security.DefaultConfig).(*http.Transport)+ c.Assert(tr.Proxy, qt.IsNil)++ sec, err := security.DecodeConfig(config.FromTOMLConfigString(`+[security.http]+proxyFromEnvironment = true+`))+ c.Assert(err, qt.IsNil)+ c.Assert(sec.HTTP.ProxyFromEnvironment, qt.IsTrue)+ tr = newSecureBaseTransport(sec).(*http.Transport)+ c.Assert(tr.Proxy, qt.IsNotNil)+}
config/security/securityConfig.goindex f7cc157b8..a2708288a 100644@@ -128,6 +128,12 @@ type HTTP struct { // URLs to allow in remote HTTP (resources.Get, getJSON, getCSV). URLs Whitelist `json:"urls"` + // Whether to honor the HTTP_PROXY, HTTPS_PROXY and NO_PROXY environment+ // variables for remote HTTP. Off by default, as a proxy hides the+ // resolved destination address from the security policy.+ // <docsmeta>{ "newIn": "0.166.0" }</docsmeta>+ ProxyFromEnvironment bool `json:"proxyFromEnvironment"`+ // HTTP methods to allow. Methods Whitelist `json:"methods"` config/security/securityConfig_test.goindex 4ac059a67..e6f8c68e3 100644@@ -135,7 +135,7 @@ func TestToTOML(t *testing.T) { got := DefaultConfig.ToTOML() c.Assert(got, qt.Equals,- "[security]\n allowContent = ['! ^text/html$', '! ^text/org$']\n enableInlineShortcodes = false\n\n [security.exec]\n allow = ['^(dart-)?sass$', '^go$', '^git$', '^node$', '^postcss$']\n osEnv = ['(?i)^((HTTPS?|NO)_PROXY|PATH(EXT)?|APPDATA|TE?MP|TERM|GO\\w+|(XDG_CONFIG_)?HOME|USERPROFILE|SSH_AUTH_SOCK|DISPLAY|LANG|SYSTEMDRIVE|PROGRAMDATA)$']\n\n [security.funcs]\n getenv = ['^HUGO_', '^CI$']\n\n [security.http]\n methods = ['(?i)GET|POST']\n urls = ['(?i)^https?://[a-z0-9]', '! (?i)^https?://\\d+\\.', '! (?i)localhost', '! (?i)^https?://[^/?#]*@']\n\n [security.node]\n [security.node.permissions]\n allowAddons = ['tailwindcss']\n allowChildProcess = ['tailwindcss']\n allowRead = ['.']\n allowWorker = ['tailwindcss']\n allowWrite = []\n disable = false",+ "[security]\n allowContent = ['! ^text/html$', '! ^text/org$']\n enableInlineShortcodes = false\n\n [security.exec]\n allow = ['^(dart-)?sass$', '^go$', '^git$', '^node$', '^postcss$']\n osEnv = ['(?i)^((HTTPS?|NO)_PROXY|PATH(EXT)?|APPDATA|TE?MP|TERM|GO\\w+|(XDG_CONFIG_)?HOME|USERPROFILE|SSH_AUTH_SOCK|DISPLAY|LANG|SYSTEMDRIVE|PROGRAMDATA)$']\n\n [security.funcs]\n getenv = ['^HUGO_', '^CI$']\n\n [security.http]\n methods = ['(?i)GET|POST']\n proxyFromEnvironment = false\n urls = ['(?i)^https?://[a-z0-9]', '! (?i)^https?://\\d+\\.', '! (?i)localhost', '! (?i)^https?://[^/?#]*@']\n\n [security.node]\n [security.node.permissions]\n allowAddons = ['tailwindcss']\n allowChildProcess = ['tailwindcss']\n allowRead = ['.']\n allowWorker = ['tailwindcss']\n allowWrite = []\n disable = false", ) } @@ -156,6 +156,7 @@ func TestDecodeConfigDefault(t *testing.T) { c.Assert(pc.HTTP.Methods.Accept("get"), qt.IsTrue) c.Assert(pc.HTTP.Methods.Accept("DELETE"), qt.IsFalse) c.Assert(pc.HTTP.MediaTypes.Accept("application/msword"), qt.IsFalse)+ c.Assert(pc.HTTP.ProxyFromEnvironment, qt.IsFalse) c.Assert(pc.Exec.OsEnv.Accept("PATH"), qt.IsTrue) c.Assert(pc.Exec.OsEnv.Accept("GOROOT"), qt.IsTrue)docs/content/en/configuration/security.mdindex 9efbc0148..54ebb4521 100644@@ -34,9 +34,15 @@ This is the default security configuration: `http.mediaTypes` : (`[]string`) Applicable to the `resources.GetRemote` function, a slice of [regular expressions](g) matching the `Content-Type` in HTTP responses that Hugo trusts, bypassing file content analysis for media type detection. +`http.proxyFromEnvironment`+: {{< new-in 0.166.0 />}}+: (`bool`) Whether the `resources.GetRemote` function honors the `HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY` environment variables. Default is `false`. When a proxy is used, Hugo connects to the proxy rather than the destination, so the resolved address validation described under `http.urls` does not apply.+ `http.urls` : (`[]string`) A slice of [regular expressions](g) matching the URLs that the `resources.GetRemote` function is allowed to access. + The default allowlist denies URLs with an IP address or `localhost` as the host name. In addition, with the default allowlist, Hugo validates the resolved address when connecting and rejects connections to loopback, private, link-local, and other non-public addresses, e.g. a host name that resolves to a cloud metadata endpoint. This validation is disabled if you override `http.urls`, as the override may intentionally allow access to hosts on your local network, such as a development server.+ `node.permissions.disable` : {{< new-in 0.161.0 />}} : (`bool`) Whether to disable the Node.js [permission model][]. When `false`, Hugo runs Node.js tools with the `--permission` flag, restricting their file system and resource access to what is explicitly allowed below. Default is `false`.docs/data/docs.yamlindex e51915c49..6d2578339 100644@@ -2253,6 +2253,7 @@ config: mediaTypes: null methods: - (?i)GET|POST+ proxyFromEnvironment: false urls: - (?i)^https?://[a-z0-9] - '! (?i)^https?://\d+\.'resources/resource_factories/create/create.goindex a34f9fc7b..1570c2648 100644@@ -16,9 +16,11 @@ package create import (+ "context" "errors" "net" "net/http"+ "net/url" "os" "path" "path/filepath"@@ -26,6 +28,8 @@ import ( "syscall" "time" + "golang.org/x/net/http/httpproxy"+ "github.com/bep/helpers/contexthelpers" "github.com/bep/logg"@@ -65,6 +69,7 @@ type contextKey uint8 const ( contextKeyResourceID contextKey = iota+ contextKeyProxyURL ) // New creates a new Client with the given specification.@@ -154,17 +159,80 @@ func New(rs *resources.Spec) *Client { // hook that validates the resolved destination address against the security // policy, so a hostname resolving to an internal address cannot be used to // reach internal endpoints. See CheckAllowedHTTPAddress.+//+// Proxies are only used when security.http.proxyFromEnvironment is enabled.+// A proxied connection is dialed to the proxy, not the destination, so the+// address check is skipped for that dial; the user has opted into trusting+// the proxy with the destination. The proxy decision is carried per request+// in its context, so a direct dial is always validated, even to the proxy's+// own address. func newSecureBaseTransport(sec security.Config) http.RoundTripper { base := http.DefaultTransport.(*http.Transport).Clone()- d := &net.Dialer{+ secureDialer := &net.Dialer{ Timeout: 30 * time.Second, KeepAlive: 30 * time.Second, Control: func(network, address string, _ syscall.RawConn) error { return sec.CheckAllowedHTTPAddress(network, address) }, }- base.DialContext = d.DialContext- return base+ base.Proxy = nil+ base.DialContext = secureDialer.DialContext++ if !sec.HTTP.ProxyFromEnvironment {+ return base+ }++ proxyFromCtx := func(ctx context.Context) *url.URL {+ u, _ := ctx.Value(contextKeyProxyURL).(*url.URL)+ return u+ }+ base.Proxy = func(req *http.Request) (*url.URL, error) {+ return proxyFromCtx(req.Context()), nil+ }+ proxyDialer := &net.Dialer{Timeout: secureDialer.Timeout, KeepAlive: secureDialer.KeepAlive}+ base.DialContext = func(ctx context.Context, network, addr string) (net.Conn, error) {+ if u := proxyFromCtx(ctx); u != nil && canonicalAddr(u) == addr {+ return proxyDialer.DialContext(ctx, network, addr)+ }+ return secureDialer.DialContext(ctx, network, addr)+ }+ return &proxyTransport{base: base, proxyFunc: httpproxy.FromEnvironment().ProxyFunc()}+}++// proxyTransport resolves the proxy for each request up front and stores it+// in the request context, where both the Proxy and DialContext hooks of base+// can see it.+type proxyTransport struct {+ base *http.Transport+ proxyFunc func(*url.URL) (*url.URL, error)+}++func (t *proxyTransport) RoundTrip(req *http.Request) (*http.Response, error) {+ u, err := t.proxyFunc(req.URL)+ if err != nil {+ return nil, err+ }+ if u != nil {+ req = req.WithContext(context.WithValue(req.Context(), contextKeyProxyURL, u))+ }+ return t.base.RoundTrip(req)+}++// canonicalAddr mirrors net/http's canonicalAddr: the host:port the transport+// dials for u, with the port defaulted from the scheme.+func canonicalAddr(u *url.URL) string {+ port := u.Port()+ if port == "" {+ switch u.Scheme {+ case "http":+ port = "80"+ case "https":+ port = "443"+ case "socks5", "socks5h":+ port = "1080"+ }+ }+ return net.JoinHostPort(u.Hostname(), port) } // Copy copies r to the new targetPath.resources/resource_factories/create/remote_test.goindex f9830c842..d7bb98361 100644@@ -14,8 +14,10 @@ package create import (+ "io" "net/http" "net/http/httptest"+ "sync/atomic" "testing" qt "github.com/frankban/quicktest"@@ -173,3 +175,78 @@ urls = ['.*'] c.Assert(err, qt.IsNil) resp.Body.Close() }++// Proxies from the environment are ignored unless+// security.http.proxyFromEnvironment is enabled. When enabled, the dial-time+// address check does not apply to the proxy itself. See issue 15302.+func TestSecureBaseTransportProxyFromEnvironment(t *testing.T) {+ c := qt.New(t)++ var proxyHits, targetHits atomic.Int32+ proxy := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {+ proxyHits.Add(1)+ w.Write([]byte("via proxy"))+ }))+ t.Cleanup(proxy.Close)+ target := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {+ targetHits.Add(1)+ w.Write([]byte("direct"))+ }))+ t.Cleanup(target.Close)++ t.Setenv("HTTP_PROXY", proxy.URL)+ t.Setenv("NO_PROXY", "")+ t.Setenv("no_proxy", "")++ decode := func(toml string) security.Config {+ sec, err := security.DecodeConfig(config.FromTOMLConfigString(toml))+ c.Assert(err, qt.IsNil)+ return sec+ }+ fetch := func(sec security.Config, u string) (string, error) {+ req, err := http.NewRequest("GET", u, nil)+ c.Assert(err, qt.IsNil)+ resp, err := newSecureBaseTransport(sec).RoundTrip(req)+ if err != nil {+ return "", err+ }+ defer resp.Body.Close()+ b, err := io.ReadAll(resp.Body)+ c.Assert(err, qt.IsNil)+ return string(b), nil+ }++ // Default: proxy ignored, fetched directly.+ body, err := fetch(decode(`+[security.http]+urls = ['.*']+`), target.URL)+ c.Assert(err, qt.IsNil)+ c.Assert(body, qt.Equals, "direct")+ c.Assert(proxyHits.Load(), qt.Equals, int32(0))+ c.Assert(targetHits.Load(), qt.Equals, int32(1))++ // Enabled: the loopback proxy is dialed even under the default hardened+ // policy, and the destination is never dialed directly.+ body, err = fetch(decode(`+[security.http]+proxyFromEnvironment = true+`), "http://example.com/")+ c.Assert(err, qt.IsNil)+ c.Assert(body, qt.Equals, "via proxy")+ c.Assert(proxyHits.Load(), qt.Equals, int32(1))+ c.Assert(targetHits.Load(), qt.Equals, int32(1))++ // Enabled, but loopback destinations are never proxied: the address+ // check still applies to the direct dial, also when the destination is+ // the proxy's own address.+ for _, u := range []string{target.URL, proxy.URL} {+ _, err = fetch(decode(`+[security.http]+proxyFromEnvironment = true+`), u)+ c.Assert(security.IsAccessDenied(err), qt.IsTrue, qt.Commentf(u))+ }+ c.Assert(proxyHits.Load(), qt.Equals, int32(1))+ c.Assert(targetHits.Load(), qt.Equals, int32(1))+}