Backtestshugorun-20260922-141813#15267

Issue #15267 Merged fix PR # Run 22 Sep 2026 · run-20260922-141813 gohugoio/hugo

Bump fileCacheEntryVersion in transform.ToMath

Better

Our patch was judged better than the merged fix.

In this judgment, Patch A = our agent, Patch B = the merged fix.

Both patches make the identical, correct root-cause fix: bumping fileCacheEntryVersion from v1 to v2 so the ToMath cache key changes and stale KaTeX-0.16 output on disk is no longer served after the 0.18.4 upgrade. On correctness they are indistinguishable. The deciding criterion is tests. Patch A adds a targeted regression test (TestToMathCacheVersionInvalidation) that seeds a stale v1 entry on disk using the same hashing.HashString the code uses, sets caches.misc with maxAge=-1 so the seeded entry would be treated as valid rather than expired, then asserts the rendered output does NOT contain the poisoned stale marker and DOES contain real KaTeX markup (<span class="katex"><math). That directly encodes the fail-before/pass-after property: with the old v1 constant the code would read the seeded v1 path, whereas with v2 it reads a fresh path and regenerates. The test is well-scoped (only a new test function plus its imports, no drive-by edits) and clearly commented. There is some uncertainty about whether the seeded filecache/misc/tomath/v1 path exactly matches Hugo's on-disk layout when dir is overridden — if it doesn't, the negative assertion could pass trivially — but even in that worst case the test still exercises and validates real ToMath rendering, which is strictly more than Patch B offers. Patch B is the same fix with zero test evidence. Given equal correctness and scope, Patch A's meaningful test breaks the tie.

The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.

Solve

18m 34s22 turns

Wall-clock time in the container

Full test suite

No new failures

3,750 passed · 99 skipped · 3,855 total

Contamination audit

Unknown

See the audit notes for this run

Patch size

45vs 2 merged

Changed lines, ours vs merged

In ec52e638c we bumped KaTeX from 0.16.21 to 0.18.4. This produces different output.

I ran into this when testing the docs site, specifically the chem example.

The two patches

Our patch+44 −1
tpl/transform/transform.goindex 5c99b322c..f9d0dabb0 100644@@ -293,7 +293,7 @@ func (ns *Namespace) ToMath(ctx context.Context, args ...any) (template.HTML, er 		Warnings []string `json:"warnings,omitempty"` 	} -	const fileCacheEntryVersion = "v1" // Increment on incompatible changes.+	const fileCacheEntryVersion = "v2" // Increment on incompatible changes.  	s := hashing.HashString(args...) 	key := "tomath/" + fileCacheEntryVersion + "/" + s[:2] + "/" + s[2:]tpl/transform/transform_integration_test.goindex 9109e6c3e..a7e4f5442 100644@@ -14,11 +14,15 @@ package transform_test  import (+	"encoding/json" 	"fmt"+	"os"+	"path/filepath" 	"strings" 	"testing"  	qt "github.com/frankban/quicktest"+	"github.com/gohugoio/hugo/common/hashing" 	"github.com/gohugoio/hugo/hugolib" ) @@ -171,6 +175,45 @@ disableKinds = ['page','rss','section','sitemap','taxonomy','term'] 	`) } +// Verify that ToMath ignores stale on-disk cache entries written with an+// older fileCacheEntryVersion after the version constant is bumped.+func TestToMathCacheVersionInvalidation(t *testing.T) {+	cacheDir := t.TempDir()++	// Compute the hash exactly as ToMath does for a single-arg call.+	s := hashing.HashString("E=mc^2")++	// Seed a stale v1 cache entry on disk.+	staleFile := filepath.Join(cacheDir, "filecache", "misc", "tomath", "v1", s[:2], s[2:])+	if err := os.MkdirAll(filepath.Dir(staleFile), 0o777); err != nil {+		t.Fatal(err)+	}+	stale, err := json.Marshal(map[string]any{+		"version": "v1",+		"output":  "STALE_CACHED_MATH_DO_NOT_SERVE",+	})+	if err != nil {+		t.Fatal(err)+	}+	if err := os.WriteFile(staleFile, append(stale, '\n'), 0o666); err != nil {+		t.Fatal(err)+	}++	files := fmt.Sprintf(`+-- hugo.toml --+disableKinds = ['page','rss','section','sitemap','taxonomy','term']+[caches.misc]+dir = '%s'+maxAge = -1+-- layouts/home.html --+{{ transform.ToMath "E=mc^2" }}+`, filepath.ToSlash(cacheDir))++	b := hugolib.Test(t, files, hugolib.TestOptWithOSFs())++	b.AssertFileContent("public/index.html", "! STALE_CACHED_MATH_DO_NOT_SERVE", `<span class="katex"><math`)+}+ func TestToMathError(t *testing.T) { 	t.Run("Default", func(t *testing.T) { 		files := ` 
Merged fix · PR #+1 −1
tpl/transform/transform.goindex 5c99b322c..f9d0dabb0 100644@@ -293,7 +293,7 @@ func (ns *Namespace) ToMath(ctx context.Context, args ...any) (template.HTML, er 		Warnings []string `json:"warnings,omitempty"` 	} -	const fileCacheEntryVersion = "v1" // Increment on incompatible changes.+	const fileCacheEntryVersion = "v2" // Increment on incompatible changes.  	s := hashing.HashString(args...) 	key := "tomath/" + fileCacheEntryVersion + "/" + s[:2] + "/" + s[2:] 

What the agent reported

The fix is complete and verified. Final summary:

What was wrong

Commit ec52e638c upgraded KaTeX from 0.16.21 to 0.18.4, which changes the rendered math output. But transform.ToMath caches renderings on disk keyed by a version segment, fileCacheEntryVersion, which stayed at "v1". As a result, sites with an existing cache (like the docs site's chem example) kept serving stale pre-upgrade output from tomath/v1/... instead of re-rendering with the new KaTeX.

What I changed

  1. tpl/transform/transform.go:296 — bumped const fileCacheEntryVersion = "v1" → "v2". This changes the cache key path to tomath/v2/..., so old v1 entries are no longer looked up and everything re-renders with the new KaTeX.
  2. tpl/transform/transform_integration_test.go — added TestToMathCacheVersionInvalidation (plus imports). It pre-seeds a stale tomath/v1/... misc-cache entry on the real filesystem containing a marker STALE_CACHED_MATH_DO_NOT_SERVE, builds a site rendering E=mc^2, and asserts the stale marker is not served while a fresh <span class="katex"><math render is.

Test evidence

  • Before the production change: the new test FAILED — Got: STALE_CACHED_MATH_DO_NOT_SERVE (the seeded v1 entry was served).
  • After bumping to "v2": ./bt-test ./tpl/transform/ → package PASS. TestToMathCacheVersionInvalidation passes; all pre-existing ToMath tests (TestToMath, TestToMathError + subtests, TestToMathBigAndManyExpressions, TestToMathRenderHookPosition, TestToMathMacros, TestToMathStrictMode) still pass. No failures in the package.
  • Reviewer confirmed the diff is minimal, matches the report, adds no scratch files, weakens no tests, and the test genuinely distinguishes v1 vs v2 code.

Requirements checklist

  • Bump fileCacheEntryVersion in transform.ToMath — ✅ Implemented ("v1"→"v2"), with a regression test that fails before and passes after.

Notes: The AGENTS.md/CLAUDE.md deletions in git status are pre-existing in the starting snapshot and untouched by this work. Two unrelated pre-existing test failures (codegen.TestMethods, htmltemplate data-parsing tests) are outside tpl/transform and not caused by this change. All changes are left uncommitted in the working tree.