Backtestshugorun-20260922-141813#15101

Issue #15101 Merged fix PR # Run 22 Sep 2026 · run-20260922-141813 gohugoio/hugo

hugo --gc deletes in-use image cache files when on-disk path case differs from the cache key (case-insensitive filesystems)

Worse

The merged fix was judged better than our patch.

In this judgment, Patch A = our agent, Patch B = the merged fix.

Both patches stop the deletion of in-use mixed-case cache files by lowercasing the walked name and comparing against the lowercased keys in `seen`, and both would fix the reported case-insensitive scenario. The decisive difference is correctness on case-SENSITIVE filesystems (Linux, most CI). Patch A applies `!seen.Has(name) && !seen.Has(strings.ToLower(name))` unconditionally, so if a genuinely stale mixed-case entry (`MyBundle/foo`) coexists with the currently-used lowercase entry (`mybundle/foo`) as two distinct files, Patch A will never prune the stale one — it silently defeats --gc for those files and lets cruft accumulate forever. Patch B defers the decision: it collects lower-case-collision candidates during the walk and only removes one if the true lowercase key `id` was ALSO independently visited by the walk (i.e., they are two distinct physical files → the mixed-case one is stale). On a case-insensitive FS only one physical file appears, so the candidate is correctly kept; on a case-sensitive FS both appear, so the stale duplicate is correctly pruned. That is the actual root-cause-correct behavior, discriminating the two filesystem semantics rather than blanket-suppressing pruning. Tests reinforce this: Patch A adds one MemMapFs test that only exercises the survives-when-only-mixed-case-exists path and never tests the case-sensitive distinction (a case-sensitive-distinct test would expose Patch A keeping the stale file). Patch B adds (1) a real-OS case-insensitive test guarded by a new `IsCaseInsensitiveFs` helper, (2) a MemMapFs case-sensitive test asserting the stale entry IS pruned (count 1) — directly validating the behavior Patch A gets wrong, and (3) a full end-to-end testscript reproduction with `hugo --gc` matching the issue's repro, gated on a `caseinsensitivefs` condition. Patch B is larger, but the extra surface is justified test infrastructure and a more precise fix; the added craft (issue references, clear deferred-decision comments) is solid. Patch A is smaller and conservative (it never deletes an in-use file), but it is a less complete fix that regresses prune completeness on the majority-case filesystem. Weighing correctness first, then the markedly stronger test evidence, B is the better fix despite its larger scope.

The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.

Solve

5h 5m22 turns

Wall-clock time in the container

Full test suite

No new failures

3,676 passed · 95 skipped · 3,784 total

Contamination audit

Unknown

See the audit notes for this run

Patch size

68vs 230 merged

Changed lines, ours vs merged

What happens

On a case-insensitive filesystem (macOS APFS, Windows), hugo --gc deletes image cache files under resources/_gen that the build actively uses, if the on-disk path case differs from the cache key. The build itself works fine (the case-insensitive filesystem resolves the read), but the pruner's usage check is a case-sensitive string comparison, so the just-used file is considered unused and removed.

The relevant code is in cache/filecache/filecache_pruner.go (unchanged on master):

if !shouldRemove && c.entryLocker.seen.Len() > 0 {
    // Remove it if it's not been touched/used in the last build.
    shouldRemove = !c.entryLocker.seen.Has(name)
}

seen contains the cache key (for images: the lowercased target path, via ImageCache.getOrCreate → c.fcache.ReadOrCreate(relTargetPath, …)), while name is the real on-disk name from the filesystem walk. If they differ only in case, seen.Has(name) is false and the file is deleted.

Why the case can differ in practice

Page paths have been lowercased since the v0.123 page-system rewrite, so cache keys for page-bundle images are lowercase. Caches created by older Hugo versions preserve the source directory case (e.g. resources/_gen/images/collections/PastFuture/…). Such caches are common in the wild: committed to git for CI cold-start speed, restored by CI cache plugins, or simply carried forward on a developer machine across Hugo upgrades.

The result is a permanent rebuild/evict cycle: --gc deletes the in-use entries → the next build silently re-processes all of them → the new files land inside the surviving mixed-case directory (the prune walk visits directories before deleting their files, so the directory is never seen empty and survives; on a case-insensitive filesystem the re-created file then lands in it) → the next --gc deletes them again. On the real site where we found this, every build re-scaled 267 images (~35 s per build, forever).

Minimal reproduction (macOS)

hugo new site repro && cd repro
mkdir -p content/MyBundle layouts/_default
printf 'baseURL = "https://example.org/"\ndisableKinds = ["taxonomy", "term", "rss", "sitemap"]\n' > hugo.toml
printf -- '---\ntitle: My Bundle\n---\n' > content/MyBundle/index.md
cp /path/to/any.jpeg content/MyBundle/cover.jpeg
printf '{{ with .Resources.GetMatch "cover.jpeg" }}{{ with .Resize "100x" }}<img src="{{ .RelPermalink }}">{{ end }}{{ end }}' > layouts/_default/single.html
printf 'home' > layouts/_default/home.html

hugo                      # creates resources/_gen/images/mybundle/cover_hu_<hash>.jpeg
# simulate a cache written by an older Hugo (or copied from elsewhere):
mv resources/_gen/images/mybundle resources/_gen/images/tmp
mv resources/_gen/images/tmp resources/_gen/images/MyBundle

hugo --gc                 # build succeeds (16 ms, cache hit), but reports "Cleaned 1"
find resources -type f    # -> empty: the file the build just used is gone

hugo                      # re-processes the image (474 ms) — and writes it back
                          # into the surviving MyBundle/ dir, so the cycle repeats

Observed output of the --gc build:

Processed images │  1
Cleaned          │  1
Total in 16 ms

A cache file that was read 16 ms earlier is deleted as "not been touched/used in the last build".

What I expected

--gc should never delete a cache entry that the same build read. On case-insensitive filesystems the usage check should match the way the filesystem resolved the read — e.g. compare case-insensitively (or normalize walk names against the seen set) when the cache filesystem is case-insensitive. Renaming the on-disk entry to the key's case would even self-heal old caches.

Version

hugo v0.164.0+extended+withdeploy darwin/arm64 BuildDate=2026-07-06T16:39:30Z VendorInfo=Homebrew

The pruner code is identical on current master.

The two patches

Our patch+67 −1
cache/filecache/filecache_pruner.goindex d6322cb70..e5a194343 100644@@ -17,6 +17,7 @@ import ( 	"fmt" 	"io" 	"os"+	"strings"  	"github.com/gohugoio/hugo/common/herrors" 	"github.com/gohugoio/hugo/hugofs"@@ -95,7 +96,13 @@ func (c *Cache) Prune(force bool) (int, error) {  		if !shouldRemove && c.entryLocker.seen.Len() > 0 { 			// Remove it if it's not been touched/used in the last build.-			shouldRemove = !c.entryLocker.seen.Has(name)+			// Cache keys have been lowercased since the v0.123 page-system rewrite,+			// but caches created by older Hugo versions (or copied across machines)+			// may keep the original directory case on disk. On a case-insensitive+			// filesystem the build still reads/uses such a file, so match the+			// lowercased name too; otherwise --gc would delete a file the build just used.+			seen := c.entryLocker.seen+			shouldRemove = !seen.Has(name) && !seen.Has(strings.ToLower(name)) 		}  		if shouldRemove {cache/filecache/filecache_pruner_test.goindex 117ff3430..9a4c2d777 100644@@ -15,6 +15,7 @@ package filecache_test  import ( 	"fmt"+	"io" 	"testing" 	"testing/synctest" 	"time"@@ -113,3 +114,61 @@ dir = ":resourceDir/_gen" 		} 	}) }++// TestPruneCaseInsensitiveInUse verifies that a cache file which was used+// during the build is not pruned just because its on-disk case differs from+// the lowercased cache key recorded in the "seen" set.+//+// Since Hugo v0.123 image/page-bundle cache keys are lowercased, but caches+// created by older Hugo versions keep the source directory case on disk. On a+// case-insensitive filesystem the build reads the file fine, so the file is in+// use and must survive a --gc prune.+func TestPruneCaseInsensitiveInUse(t *testing.T) {+	t.Parallel()+	c := qt.New(t)++	cfg := filecache.FileCacheConfig{+		MaxAge: 24 * time.Hour, // long, so the entry is NOT expired by age+		Dir:    "cache/c",+	}+	cache := filecache.NewCache(afero.NewMemMapFs(), cfg)++	// Create a cache entry using the LOWERCASE key. This both writes the file+	// at the lowercase path AND records the lowercase key in the internal+	// "seen" set.+	const lowerKey = "mybundle/cover_hu_x.jpg"+	_, err := cache.ReadOrCreate(lowerKey,+		func(info filecache.ItemInfo, r io.ReadSeeker) error { return nil }, // read (not called; file doesn't exist yet)+		func(info filecache.ItemInfo, w io.WriteCloser) error { // create+			_, err := w.Write([]byte("abc"))+			if err != nil {+				return err+			}+			return w.Close()+		})+	c.Assert(err, qt.IsNil)++	// Simulate a cache written by an OLDER Hugo (different on-disk case) by+	// renaming the on-disk file to a MIXED-case path, exactly like the report's+	// mv step. The mixed-case key is NOT in "seen".+	const mixedKey = "MyBundle/cover_hu_x.jpg"+	c.Assert(cache.Fs.MkdirAll("MyBundle", 0o777), qt.IsNil) // ensure parent dir exists for the rename+	c.Assert(cache.Fs.Rename(lowerKey, mixedKey), qt.IsNil)++	// Confirm the setup is as intended.+	lowerExists, err := afero.Exists(cache.Fs, lowerKey)+	c.Assert(err, qt.IsNil)+	c.Assert(lowerExists, qt.IsFalse)+	mixedExists, err := afero.Exists(cache.Fs, mixedKey)+	c.Assert(err, qt.IsNil)+	c.Assert(mixedExists, qt.IsTrue)++	// Prune and assert the in-use file is NOT deleted.+	count, err := cache.Prune(false)+	c.Assert(err, qt.IsNil)+	c.Assert(count, qt.Equals, 0)++	stillExists, err := afero.Exists(cache.Fs, mixedKey)+	c.Assert(err, qt.IsNil)+	c.Assert(stillExists, qt.IsTrue)+} 
Merged fix · PR #+217 −13
cache/filecache/filecache_pruner.goindex d6322cb70..cd95766be 100644@@ -17,6 +17,7 @@ import ( 	"fmt" 	"io" 	"os"+	"strings"  	"github.com/gohugoio/hugo/common/herrors" 	"github.com/gohugoio/hugo/hugofs"@@ -59,6 +60,29 @@ func (c *Cache) Prune(force bool) (int, error) {  	counter := 0 +	seen := c.entryLocker.seen+	seenByLower := make(map[string]string, seen.Len())+	for id := range seen.All() {+		seenByLower[strings.ToLower(id)] = id+	}++	// Names on disk matching a used cache key except for the case, and the used keys+	// actually walked. See the note about case-insensitive filesystems below.+	var candidates map[string]string+	visited := make(map[string]bool, seen.Len())++	remove := func(name string) error {+		err := c.Fs.Remove(name)+		if err == nil {+			counter+++			return nil+		}+		if !herrors.IsNotExist(err) {+			return err+		}+		return nil+	}+ 	err := afero.Walk(c.Fs, "", func(name string, info os.FileInfo, err error) error { 		if info == nil { 			return nil@@ -93,27 +117,44 @@ func (c *Cache) Prune(force bool) (int, error) {  		shouldRemove := force || c.isExpired(info.ModTime()) -		if !shouldRemove && c.entryLocker.seen.Len() > 0 {-			// Remove it if it's not been touched/used in the last build.-			shouldRemove = !c.entryLocker.seen.Has(name)+		if seen.Has(name) {+			visited[name] = true+		} else if !shouldRemove && seen.Len() > 0 {+			if id, found := seenByLower[strings.ToLower(name)]; found {+				// On case-insensitive filesystems this is the same file as id; e.g. an+				// entry created before Hugo started lowercasing the content paths in+				// v0.123 (content/MyBundle => _gen/images/MyBundle). Decided once the+				// walk is done: if id is walked too, they are distinct files and this+				// one is stale. See issue 15101.+				if candidates == nil {+					candidates = make(map[string]string)+				}+				candidates[name] = id+			} else {+				// Remove it if it's not been touched/used in the last build.+				shouldRemove = true+			} 		}  		if shouldRemove {-			err := c.Fs.Remove(name)-			if err == nil {-				counter++-			}--			if err != nil && !herrors.IsNotExist(err) {-				return err-			}-+			return remove(name) 		}  		return nil 	})+	if err != nil {+		return counter, err+	}++	for name, id := range candidates {+		if visited[id] {+			if err := remove(name); err != nil {+				return counter, err+			}+		}+	} -	return counter, err+	return counter, nil }  func (c *Cache) pruneRootDirs(force bool) (int, error) {cache/filecache/filecache_pruner_test.goindex 117ff3430..dc01e417a 100644@@ -20,11 +20,72 @@ import ( 	"time"  	"github.com/gohugoio/hugo/cache/filecache"+	"github.com/gohugoio/hugo/htesting" 	"github.com/spf13/afero"  	qt "github.com/frankban/quicktest" ) +// A cache entry created before Hugo started lowercasing content paths in v0.123+// (e.g. _gen/images/MyBundle) is on a case-insensitive filesystem the same file as+// the lowercased cache key used today, and must not be pruned.+// See issue 15101.+func TestPruneCacheEntryWithOtherCase(t *testing.T) {+	t.Parallel()+	c := qt.New(t)++	dir := t.TempDir()+	if isCaseInsensitive, err := htesting.IsCaseInsensitiveFs(dir); err != nil {+		t.Fatal(err)+	} else if !isCaseInsensitive {+		t.Skip("skip test on case-sensitive filesystem")+	}++	fs := afero.NewBasePathFs(afero.NewOsFs(), dir)+	newCache := func() *filecache.Cache {+		return filecache.NewCache(fs, filecache.FileCacheConfig{Dir: "cache", MaxAge: -1})+	}++	c.Assert(newCache().SetBytes("MyBundle/i1", []byte("abc")), qt.IsNil)++	cache := newCache()+	_, b, err := cache.GetOrCreateBytes("mybundle/i1", func() ([]byte, error) {+		return []byte("def"), nil+	})+	c.Assert(err, qt.IsNil)+	c.Assert(string(b), qt.Equals, "abc")++	count, err := cache.Prune(false)+	c.Assert(err, qt.IsNil)+	c.Assert(count, qt.Equals, 0)+	c.Assert(cache.GetString("MyBundle/i1"), qt.Equals, "abc")+}++// On a case-sensitive filesystem the entries above are distinct files,+// and the one not used in this build should be pruned.+func TestPruneCacheEntryWithOtherCaseCaseSensitiveFs(t *testing.T) {+	t.Parallel()+	c := qt.New(t)++	fs := afero.NewMemMapFs()+	cache := filecache.NewCache(fs, filecache.FileCacheConfig{Dir: "cache", MaxAge: -1})++	c.Assert(cache.SetBytes("MyBundle/i1", []byte("abc")), qt.IsNil)++	cache = filecache.NewCache(fs, filecache.FileCacheConfig{Dir: "cache", MaxAge: -1})+	_, b, err := cache.GetOrCreateBytes("mybundle/i1", func() ([]byte, error) {+		return []byte("def"), nil+	})+	c.Assert(err, qt.IsNil)+	c.Assert(string(b), qt.Equals, "def")++	count, err := cache.Prune(false)+	c.Assert(err, qt.IsNil)+	c.Assert(count, qt.Equals, 1)+	c.Assert(cache.GetString("MyBundle/i1"), qt.Equals, "")+	c.Assert(cache.GetString("mybundle/i1"), qt.Equals, "def")+}+ func TestPrune(t *testing.T) { 	t.Parallel() htesting/test_helpers.goindex 384695df4..46f6bd9ec 100644@@ -16,6 +16,7 @@ package htesting import ( 	"math/rand" 	"os"+	"path/filepath" 	"regexp" 	"runtime" 	"strconv"@@ -58,6 +59,27 @@ func CreateTempDir(fs afero.Fs, prefix string) (string, func(), error) { 	return tempDir, func() { fs.RemoveAll(tempDir) }, nil } +// IsCaseInsensitiveFs reports whether dir lives on a case-insensitive filesystem+// (e.g. the default on macOS and Windows).+func IsCaseInsensitiveFs(dir string) (bool, error) {+	f, err := os.CreateTemp(dir, "case-*.tmp")+	if err != nil {+		return false, err+	}+	f.Close()+	defer os.Remove(f.Name())++	_, err = os.Stat(filepath.Join(dir, strings.ToUpper(filepath.Base(f.Name()))))+	if err != nil {+		if os.IsNotExist(err) {+			return false, nil+		}+		return false, err+	}++	return true, nil+}+ // BailOut panics with a stack trace after the given duration. Useful for // hanging tests. func BailOut(after time.Duration) {main_test.goindex 12e708dc0..8cea78818 100644@@ -15,6 +15,7 @@ package main  import ( 	"bytes"+	"encoding/base64" 	"encoding/json" 	"fmt" 	"io"@@ -27,6 +28,7 @@ import ( 	"runtime" 	"strconv" 	"strings"+	"sync" 	"testing" 	"time" @@ -78,10 +80,22 @@ func TestMain(m *testing.M) { 	}) } +var isCaseInsensitiveFs = sync.OnceValues(func() (bool, error) {+	return htesting.IsCaseInsensitiveFs(os.TempDir())+})+ var commonTestScriptsParam = testscript.Params{ 	Setup: func(env *testscript.Env) error { 		return testSetupFunc()(env) 	},+	Condition: func(cond string) (bool, error) {+		switch cond {+		case "caseinsensitivefs":+			return isCaseInsensitiveFs()+		default:+			return false, fmt.Errorf("unknown condition %q", cond)+		}+	}, 	Cmds: map[string]func(ts *testscript.TestScript, neg bool, args []string){ 		// log prints to stderr. 		"log": func(ts *testscript.TestScript, neg bool, args []string) {@@ -252,6 +266,34 @@ var commonTestScriptsParam = testscript.Params{ 				ts.Fatalf("failed to create symlink: %v", err) 			} 		},+		// base64decode decodes a base64-encoded file into a binary file.+		"base64decode": func(ts *testscript.TestScript, neg bool, args []string) {+			if len(args) != 2 {+				ts.Fatalf("usage: base64decode src_base64_file dest_binary_file")+			}++			// Resolve paths relative to the test sandbox's current directory+			src := ts.MkAbs(args[0])+			dest := ts.MkAbs(args[1])++			// Read the base64 text+			base64Data, err := os.ReadFile(src)+			if err != nil {+				ts.Fatalf("failed to read base64 file: %v", err)+			}++			// Decode base64 back to raw bytes.+			binaryBytes, err := base64.StdEncoding.DecodeString(strings.TrimSpace(string(base64Data)))+			if err != nil {+				ts.Fatalf("failed to decode base64 data: %v", err)+			}++			// Write the binary file.+			err = os.WriteFile(dest, binaryBytes, 0o644)+			if err != nil {+				ts.Fatalf("failed to write binary file: %v", err)+			}+		},  		// httpget checks that a HTTP resource's body matches (if it compiles as a regexp) or contains all of the strings given as arguments. 		"httpget": func(ts *testscript.TestScript, neg bool, args []string) {testscripts/commands/hugo_gc_mixed_case_cache_dir.txtnew file mode 100644index 000000000..7c7af1373@@ -0,0 +1,38 @@+# In Hugo v0.123 we started lowercasing the content paths, but on case-insensitive+# filesystems existing caches keep their mixed-case dir names (e.g. MyBundle).+# The pruner would then consider those entries unused and remove them on every+# hugo --gc. See issue 15101.++[!caseinsensitivefs] skip++base64decode pixel.png.txt content/MyBundle/pix.png++hugo++stdout 'Processed images │  1 '+tree resources/_gen+stdout 'pix_hu_a354833fd576551d.png'++# Simulate a cache created by an older Hugo version.+mv resources/_gen/images/mybundle resources/_gen/images/MyBundle++hugo --gc+stdout 'Cleaned          │  0 '++tree resources/_gen+stdout 'pix_hu_a354833fd576551d.png'++-- hugo.toml --+disableKinds = ["term", "taxonomy", "home"]+-- layouts/all.html --+All.+-- layouts/page.html --+{{ $pix := .Resources.Get "pix.png" }}+{{ $resized := $pix.Resize "10x10" }}+Resized: {{ $resized.RelPermalink }}+-- content/MyBundle/index.md --+---+title: MyBundle+---+-- pixel.png.txt --+iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg== 

What the agent reported

The fix is complete, verified, and reviewed. Here is the summary.

What was wrong

On case-insensitive filesystems (macOS, Windows), hugo --gc deleted image cache files under resources/_gen that the build had just used. In cache/filecache/filecache_pruner.go, the pruner's "still in use?" test — c.entryLocker.seen.Has(name) — is a case-sensitive comparison. seen holds cache keys, which have been lowercased since the v0.123 page-system rewrite (e.g. mybundle/cover_hu_x.jpg), while name is the real on-disk name from the filesystem walk, which older caches keep in the source directory case (e.g. MyBundle/cover_hu_x.jpg). The case-insensitive filesystem resolves the build's read fine, but the pruner's exact-match lookup misses, so the in-use file is deleted — producing a permanent rebuild/evict cycle.

What I changed

  • cache/filecache/filecache_pruner.go — the usage check now also matches the lowercased on-disk name, so a file whose key was lowercased still counts as in-use: go seen := c.entryLocker.seen shouldRemove = !seen.Has(name) && !seen.Has(strings.ToLower(name)) Added the strings import and an explanatory comment. This is testable on the case-sensitive Linux filesystem used in CI, unlike an FS-case-insensitivity probe (which can't be triggered there). Any stale file this might conservatively keep on a genuinely case-sensitive FS is still removed by the existing MaxAge/isExpired fallback.
  • cache/filecache/filecache_pruner_test.go — added TestPruneCaseInsensitiveInUse, which mirrors the report's reproduction (create entry with the lowercase key, rename on disk to mixed case, prune, assert nothing pruned and the file survives).

Note: an intermediate step mistakenly rewrote the pre-existing TestPrune in this file; I restored TestPrune verbatim and added the new test alongside it. The reviewer confirmed TestPrune is fully intact.

Test evidence (./bt-test ./cache/filecache/)

  • New test failed before the fix: count was 1 (in-use file deleted) vs. expected 0.
  • After the fix, package result pass, zero failures:
  • TestPruneCaseInsensitiveInUse — pass (was failing pre-fix)
  • TestPrune (restored original) — pass
  • TestPruneShouldPreserveEmptyCacheRoots — pass
  • TestPruneImages — skip (unrelated, skipped by its own design)

Requirements disposition

  1. --gc must never delete a cache entry the same build read when on-disk case differs from the (lowercased) key — ✅ implemented in filecache_pruner.go; regression test asserts the in-use mixed-case file survives with count == 0.
  2. Regression test reproducing the reported scenario, failing before / passing after — ✅ TestPruneCaseInsensitiveInUse; observed failing pre-fix (count 1) and passing post-fix.
  3. Optional self-heal by renaming the on-disk entry to the key's case — ⏭️ skipped. The report offers this only as a nicety ("would even self-heal"). Evidence it's unnecessary: the fix already breaks the rebuild/evict cycle by not deleting the in-use file, so no re-processing occurs; renaming would add filesystem-mutation risk during a read-oriented prune walk for no correctness gain. Kept the change minimal.

Changes are left uncommitted in the working tree.