Backtestshugorun-20261002-200313#15189
Issue with chained resource transformations
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Root cause: once the chain has run, `resourceAdapterInner.target` is replaced by the transformed resource. `TransformWithContext` then copies that transformed target but keeps the full transformation list, so the earlier steps (e.g. css.Build) run a second time on output that is already built. Patch A stores the original untransformed target in a new `sourceTarget` field and always starts new chains from it. This fixes the basic Data-then-fingerprint case, and replaying the full chain from source also keeps cache keys consistent. However, `cloneTo` sets `sourceTarget` to a clone of `r.target`. If the transformation has already been applied, that is the transformed resource, while the full chain is still attached. So `resources.Copy` on an already-triggered resource, followed by another transformation, would re-apply the chain to transformed content: the same class of bug. Patch A's only test is the css.Build/Data.Artifacts/fingerprint scenario. Patch B tracks an `applied` flag and an `appliedCount`. The `transform` loop skips transformations already applied to the target. `sourceKey` keeps the cache key equal to that of a chain built in one go, and the full chain is kept so `hasTransformationPermalinkHash` stays correct. It also avoids sharing the slice backing array with the previous chain, which is a latent aliasing hazard when appending. Finally, `cloneTo` propagates `applied`, explicitly covering the copy case. B's tests are broader. They trigger the transformation through Data, Content and RelPermalink, check that the chaining result is identical to the untriggered case, test copy before and after the trigger, and include the css.Build artifacts scenario with output file assertions. B is more complex, but its added logic is well commented and addresses edge cases A misses. On correctness and test coverage, B is the stronger fix.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
1h 41m
Wall-clock time in the container
Full test suite
No new failures
3,701 passed · 99 skipped · 3,809 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
173vs 53 merged
Changed lines, ours vs merged
The issue
View on GitHubSee the fix in #15188.
This has been an issue since we added this feature, but we haven't noticed it (or gotten reports about it) because the common setup is to set up the entire chain before starting to use it:
{{- with . | css.Build $opts }}
{{- if hugo.IsDevelopment }}
<link rel="stylesheet" href="{{ .RelPermalink }}">
{{- else }}
{{- with . | fingerprint }}
<link rel="stylesheet" href="{{ .RelPermalink }}" integrity="{{ .Data.Integrity }}">
{{- end }}
{{- end }}
{{- end }}
In the above:
.RelPermalinktriggers the transformation of the entire chain in a streaming way.
With the recently added .Data.Artifacts in css.Build I discovered this issue by accident when doing something ala:
{{- with . | css.Build $opts }}
{{ range .Data.Artifacts }}
{{ if eq .MediaType.MainType "font" }}
<link rel="preload" href="{{ .RelPermalink }}" as="font" type="{{ .MediaType.Type }}">
{{ end }}
{{ end }}
{{- if hugo.IsDevelopment }}
<link rel="stylesheet" href="{{ .RelPermalink }}">
{{- else }}
{{- with . | fingerprint }}
<link rel="stylesheet" href="{{ .RelPermalink }}" integrity="{{ .Data.Integrity }}">
{{- end }}
{{- end }}
{{- end }}
In the above:
.Datatriggers thecss.Buildtransformation.- Then
fingerprintstarts a new transformation with correct.Content, but wrongly including the full chain, includingcss.Build
The above could be rewritten to the below and it would work as expected:
{{- with . | css.Build $opts }}
{{- if hugo.IsDevelopment }}
<link rel="stylesheet" href="{{ .RelPermalink }}">
{{- else }}
{{- with . | fingerprint }}
{{ range .Data.Artifacts }}
{{ if eq .MediaType.MainType "font" }}
<link rel="preload" href="{{ .RelPermalink }}" as="font" type="{{ .MediaType.Type }}">
{{ end }}
{{ end }}
<link rel="stylesheet" href="{{ .RelPermalink }}" integrity="{{ .Data.Integrity }}">
{{- end }}
{{- end }}
{{- end }}
{{- end }}The two patches
resources/transform.goindex 18c768b36..1bfa092a9 100644@@ -22,6 +22,7 @@ import ( "path" "strings" "sync"+ "sync/atomic" "github.com/gohugoio/hugo/common/constants" "github.com/gohugoio/hugo/common/hashing"@@ -236,6 +237,9 @@ func (r resourceAdapter) cloneTo(targetPath string) resource.Resource { Staler: r.Staler, target: newtTarget.(transformableResource), }+ // The clone gets a new target, so it gets the cache key from that, but+ // we need to preserve whether the transformations are applied to it.+ newInner.applied.Store(r.resourceAdapterInner.applied.Load()) if r.resourceAdapterInner.publishOnce != nil { newInner.publishOnce = &publishOnce{} }@@ -370,16 +374,34 @@ func (r resourceAdapter) Transform(t ...ResourceTransformation) (ResourceTransfo } func (r resourceAdapter) TransformWithContext(ctx context.Context, t ...ResourceTransformation) (ResourceTransformer, error) {- r.resourceTransformations = &resourceTransformations{- transformations: append(r.transformations, t...),+ prev := r.resourceTransformations+ prevInner := r.resourceAdapterInner+ rt := &resourceTransformations{+ // Make sure we don't share the backing array with prev.+ transformations: append(prev.transformations[:len(prev.transformations):len(prev.transformations)], t...),+ appliedCount: prev.appliedCount, } + var sourceKey string+ // Check this first; target and sourceKey are set before applied is stored.+ if prevInner.applied.Load() {+ // The existing chain has already been applied to the target,+ // so we only need to apply the new transformations.+ // Note that we keep the full chain to get a stable cache key and+ // e.g. hasTransformationPermalinkHash right.+ rt.appliedCount = len(prev.transformations)+ sourceKey = prevInner.sourceKey+ }++ r.resourceTransformations = rt+ r.resourceAdapterInner = &resourceAdapterInner{ ctx: ctx, spec: r.spec, Staler: r.Staler, publishOnce: &publishOnce{},- target: r.target,+ target: prevInner.target,+ sourceKey: sourceKey, } return &r, nil@@ -451,7 +473,11 @@ func (r *resourceAdapter) TransformationKey() string { for _, tr := range r.transformations { key = key + "_" + tr.Key().Value() }- return r.spec.ResourceCache.cleanKey(r.target.Key()) + "_" + hashing.MD5FromStringHexEncoded(key)+ sourceKey := r.sourceKey+ if sourceKey == "" {+ sourceKey = r.target.Key()+ }+ return r.spec.ResourceCache.cleanKey(sourceKey) + "_" + hashing.MD5FromStringHexEncoded(key) } func (r *resourceAdapter) getOrTransform(publish, setContent bool) error {@@ -545,7 +571,9 @@ func (r *resourceAdapter) transform(key string, publish, setContent bool) (*reso var transformedContentr io.Reader - for i, tr := range r.transformations {+ // Any transformations at the start of the chain that are already applied+ // to the target must not be applied again.+ for i, tr := range r.transformations[r.appliedCount:] { if i != 0 { tctx.InMediaType = tctx.OutMediaType }@@ -727,7 +755,18 @@ func (r *resourceAdapter) doInit(publish, setContent bool) error { r.publishOnce = nil } + if r.sourceKey == "" {+ // Store the key of the resource this chain started from,+ // so any new transformations chained onto this one gets the same+ // cache key as if the full chain was applied in one go.+ r.sourceKey = r.target.Key()+ }+ r.transformationsErr = r.getOrTransform(publish, setContent)+ if r.transformationsErr == nil {+ // Note that getOrTransform may have replaced resourceAdapterInner.+ r.applied.Store(true)+ } }) if publish && r.publishOnce != nil {@@ -747,6 +786,13 @@ type resourceAdapterInner struct { spec *Spec + // The key of the resource the transformation chain started from.+ // Used to create the cache key.+ sourceKey string++ // Set when the transformations have been applied to target.+ applied atomic.Bool+ // Handles publishing (to /public) if needed. *publishOnce }@@ -764,6 +810,10 @@ type resourceTransformations struct { transformationsInit sync.Once transformationsErr error transformations []ResourceTransformation++ // The number of transformations at the start of transformations+ // that are already applied to the target.+ appliedCount int } // hasTransformationPermalinkHash reports whether any of the transformationsresources/transform_integration_test.goindex 48d601545..27580bfa1 100644@@ -14,6 +14,7 @@ package resources_test import (+ "strings" "testing" "github.com/gohugoio/hugo/hugolib"@@ -48,3 +49,74 @@ CSS: {{ $css.Content }} b.AssertFileContent("public/p1/index.html", "CSS: body{background:#fff}") }++// Issue #15189.+func TestTransformChainAfterTransformationApplied(t *testing.T) {+ t.Parallel()++ files := `+-- hugo.toml --+disableKinds = ["taxonomy", "term", "page", "section", "rss", "sitemap"]+-- assets/css/main.css --+body {+ background: #fff;+}+-- layouts/home.html --+{{ $fp := resources.Get "css/main.css" | fingerprint "md5" }}+{{/* Access a method that triggers the transformation before chaining. */}}+{{ with $fp }}+{{ $trigger := "" }}+{{ if eq "TRIGGER" "Data" }}{{ $trigger = .Data.Integrity }}{{ end }}+{{ if eq "TRIGGER" "Content" }}{{ $trigger = .Content }}{{ end }}+{{ if eq "TRIGGER" "RelPermalink" }}{{ $trigger = .RelPermalink }}{{ end }}+{{ end }}+{{ $min := $fp | minify }}+{{ $min2 := $min | resources.Copy "css/copy.css" }}+Min: {{ $min.RelPermalink }}|{{ $min.Content }}|{{ $min.Data.Integrity }}|+Copy: {{ $min2.RelPermalink }}|+`++ for _, trigger := range []string{"None", "Data", "Content", "RelPermalink"} {+ t.Run(trigger, func(t *testing.T) {+ t.Parallel()+ b := hugolib.Test(t, strings.ReplaceAll(files, "TRIGGER", trigger))+ b.AssertFileContent("public/index.html",+ "Min: /css/main.23f3e80cae4655a34b8ea2ef5424b4e6.min.css|body{background:#fff}|md5-I/PoDK5GVaNLjqLvVCS05g==|",+ "Copy: /css/copy.css|",+ )+ b.AssertFileExists("public/css/main.23f3e80cae4655a34b8ea2ef5424b4e6.min.css", true)+ })+ }+}++// Issue #15189.+func TestTransformChainOnCopyAfterTransformationApplied(t *testing.T) {+ t.Parallel()++ files := `+-- hugo.toml --+disableKinds = ["taxonomy", "term", "page", "section", "rss", "sitemap"]+-- assets/css/main.css --+body {+ background: #fff;+}+-- layouts/home.html --+{{ $css := resources.Get "css/main.css" | minify }}+{{ $copyEarly := $css | resources.Copy "css/early.css" }}+{{ $trigger := $css.Content }}+{{ $copy := $css | resources.Copy "css/copy.css" }}+{{ $fp1 := $css | fingerprint "md5" }}+{{ $fp2 := $copy | fingerprint "md5" }}+{{ $fp3 := $copyEarly | fingerprint "md5" }}+FP1: {{ $fp1.RelPermalink }}|{{ $fp1.Content }}|+FP2: {{ $fp2.RelPermalink }}|{{ $fp2.Content }}|+FP3: {{ $fp3.RelPermalink }}|{{ $fp3.Content }}|+`++ b := hugolib.Test(t, files)+ b.AssertFileContent("public/index.html",+ "FP1: /css/main.min.267d429567e3d9741bb48e2ed9e732fd.css|body{background:#fff}|",+ "FP2: /css/copy.267d429567e3d9741bb48e2ed9e732fd.css|body{background:#fff}|",+ "FP3: /css/early.min.267d429567e3d9741bb48e2ed9e732fd.css|body{background:#fff}|",+ )+}tpl/css/build_integration_test.goindex 6ddb23331..c1349e069 100644@@ -537,6 +537,47 @@ ARTIFACT: {{ .RelPermalink }}|{{ .Permalink }}|{{ .MediaType.Type }} b.AssertFileExists("public/css/main.css.map", true) } +// Issue #15189.+func TestCSSBuildDataArtifactsThenFingerprint(t *testing.T) {+ t.Parallel()++ files := `+-- hugo.toml --+disableKinds = ['page','rss','section','sitemap','taxonomy','term']+-- assets/css/main.css --+@import "/fonts/fonts.css";+body { color: #222; }+-- assets/fonts/fonts.css --+@font-face {+ font-family: 'Comic Neue';+ src: url(ComicNeue-Regular.woff2) format('woff2');+}+-- assets/fonts/ComicNeue-Regular.woff2 --+fakefontdata2+-- layouts/home.html --+{{ with resources.Get "css/main.css" }}+{{ with . | css.Build (dict "minify" true) }}+{{ range .Data.Artifacts }}{{ if eq .MediaType.MainType "font" }}+<link rel="preload" href="{{ .RelPermalink }}" as="font" type="{{ .MediaType.Type }}" crossorigin>+{{ end }}{{ end }}+{{ with . | fingerprint "md5" }}+<link rel="stylesheet" href="{{ .RelPermalink }}" integrity="{{ .Data.Integrity }}">+{{ range .Data.Artifacts }}FPARTIFACT: {{ .RelPermalink }}|{{ end }}+{{ end }}+{{ end }}+{{ end }}+`++ b := hugolib.Test(t, files, hugolib.TestOptOsFs())+ b.AssertFileContent("public/index.html",+ `as="font" type="font/woff2" crossorigin`,+ `<link rel="stylesheet" href="/css/main.222d6fa376dfed9b506387bebcd2847f.css" integrity="md5-`,+ "FPARTIFACT: /css/ComicNeue-Regular-",+ )+ b.AssertFileContent("public/css/main.222d6fa376dfed9b506387bebcd2847f.css", `src:url("/css/ComicNeue-Regular-VITKVZZR.woff2") format("woff2")}body{color:#222}`)+ b.AssertFileExists("public/css/ComicNeue-Regular-VITKVZZR.woff2", true)+}+ // Issue #14623 func TestCSSBuildLoadersPartial(t *testing.T) { t.Parallel()
resources/transform.goindex 18c768b36..cee11f8ba 100644@@ -82,6 +82,7 @@ func newResourceAdapter(spec *Spec, lazyPublish bool, target transformableResour return &resourceAdapter{ resourceTransformations: &resourceTransformations{}, metaProvider: target,+ sourceTarget: target, resourceAdapterInner: &resourceAdapterInner{ ctx: context.Background(), spec: spec,@@ -186,6 +187,12 @@ type resourceAdapter struct { commonResource *resourceTransformations *resourceAdapterInner++ // The original untransformed target. The inner target is replaced with the+ // transformed resource once the transformation chain has run, so any new+ // transformations appended to the chain must start from this.+ sourceTarget transformableResource+ metaProvider resource.ResourceMetaProvider } @@ -230,6 +237,7 @@ func (r *resourceAdapter) GetDependencyManager() identity.Manager { func (r resourceAdapter) cloneTo(targetPath string) resource.Resource { newtTarget := r.target.cloneTo(targetPath)+ r.sourceTarget = newtTarget.(transformableResource) newInner := &resourceAdapterInner{ ctx: r.ctx, spec: r.spec,@@ -379,7 +387,7 @@ func (r resourceAdapter) TransformWithContext(ctx context.Context, t ...Resource spec: r.spec, Staler: r.Staler, publishOnce: &publishOnce{},- target: r.target,+ target: r.sourceTarget, } return &r, niltpl/css/build_integration_test.goindex 6ddb23331..e72a1bb03 100644@@ -537,6 +537,49 @@ ARTIFACT: {{ .RelPermalink }}|{{ .Permalink }}|{{ .MediaType.Type }} b.AssertFileExists("public/css/main.css.map", true) } +func TestCSSBuildDataArtifactsAndThenFingerprint(t *testing.T) {+ t.Parallel()++ files := `+-- hugo.toml --+baseURL = "https://example.org/mysite/"+disableKinds = ['page','rss','section','sitemap','taxonomy','term']+-- assets/css/main.css --+@import "/fonts/fonts.css";+body { color: #222; }+-- assets/fonts/fonts.css --+@font-face {+ font-family: 'Comic Neue';+ src: url(ComicNeue-Regular.woff2) format('woff2'), url(ComicNeue-Regular.ttf) format('truetype');+}+-- assets/fonts/ComicNeue-Regular.ttf --+fakefontdata+-- assets/fonts/ComicNeue-Regular.woff2 --+fakefontdata2+-- layouts/home.html --+{{ with resources.Get "css/main.css" }}+ORIG CSS SHA256: {{ .Content | crypto.Hash "sha256" }}|+{{ with . | css.Build (dict "minify" true) }}+{{ range .Data.Artifacts }}+ARTIFACT: {{ .RelPermalink }}|{{ .Permalink }}|{{ .MediaType.Type }}+{{ end }}+BUILT CSS SHA256: {{ .Content | crypto.Hash "sha256" }}|+{{ with . | fingerprint }}+FINGERPRINTED: {{ .RelPermalink }}|+{{ end }}+{{ end }}+{{ end }}+`++ b := hugolib.Test(t, files, hugolib.TestOptOsFs())+ b.AssertFileContent("public/index.html",+ "ARTIFACT: /mysite/css/ComicNeue-Regular-UA4ODE7N.ttf|",+ "ORIG CSS SHA256: a2b878c05df0f346c3854caebd21f00bb0965b46628440b8ccb31ce675af6bfe|",+ "BUILT CSS SHA256: 5f8757d5579386b8755e0df759c7eecfd069c385294dfe7bc3695862d7f218b1|",+ "FINGERPRINTED: /mysite/css/main.5f8757d5579386b8755e0df759c7eecfd069c385294dfe7bc3695862d7f218b1.css|",+ )+}+ // Issue #14623 func TestCSSBuildLoadersPartial(t *testing.T) { t.Parallel()