Backtestshugorun-20260917-015623#11302

Issue #11302 Merged fix PR # Run 17 Sep 2026 · run-20260917-015623 gohugoio/hugo

The "failed to extract" error refers to outer shortcode instead of inner shortcode

Worse

The merged fix was judged better than our patch.

In this judgment, Patch B = our agent, Patch A = the merged fix.

Patch B contains no production code change at all — it only adds a lexer test case to pageparser_shortcode_test.go. A patch that changes zero production logic cannot resolve the reported regression: if the current lexer already tokenizes the nested/closed structure correctly the new test is tautological (passes without any fix), and if the lexer is genuinely buggy the test would fail with no accompanying fix to make it pass. Either way Patch B resolves nothing on criterion 1 (correctness) and its test therefore fails the 'fail-before/pass-after via the fix' bar on criterion 2. Patch A, whatever its imperfections, actually modifies production code (hugolib/shortcode.go) and directly targets the behavior named in the issue title: attributing the 'failed to extract shortcode' error to the inner shortcode's position (line/column) rather than the outer one, by capturing currItem.Pos() before the recursive extractShortcode call and wrapping non-FileError results with NewFileErrorFromPos. Its added test asserts the concrete corrected position (line 6, col 12 vs the outer col 1) and message, which is a substantive, non-tautological check that would differ before vs after the change. Patch A is not a flawless root-cause fix — it improves error attribution/position for failing nested extractions rather than demonstrably touching the 'is-closed' detection the reporter suspected, and the 'template not found' phrasing in its test is a render-vs-extract concern I cannot fully verify without running it. But scope is tight (a single localized error-handling branch plus one focused test) and it is a real attempt at the stated defect. Between a real, scoped production fix with a meaningful test and a test-only diff that changes no behavior, Patch A is clearly better; this is not a tie.

The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.

Solve

10m 48s2 turns

Wall-clock time in the container

Full test suite

No new failures

3,480 passed · 85 skipped · 3,579 total

Contamination audit

Unknown

See the audit notes for this run

Patch size

9vs 43 merged

Changed lines, ours vs merged

<!-- Please answer these questions before submitting your issue. Thanks! -->

What version of Hugo are you using (hugo version)?

<pre>
$ hugo version
hugo v0.111.1-39a4a3cf676533859217805c36181c7a3dfa321c darwin/arm64 BuildDate=2023-03-02T10:08:26Z VendorInfo=gohugoio
</pre>

Does this issue reproduce with the latest release?

yes

When evaluating this file:

+++
Type = "note"
Date = "2020-03-30T01:19:59-05:00"

+++

{{< figure >}}
{{< img src="lighting-in-pantry.jpg" alt="a pantry with led lights shining bright light out of it" >}}
{{% figcaption %}}
Snazzy 💡
{{% /figcaption %}}
{{< /figure >}}

i get this error:

Start building sites … 
hugo v0.111.1-39a4a3cf676533859217805c36181c7a3dfa321c darwin/arm64 BuildDate=2023-03-02T10:08:26Z VendorInfo=gohugoio
Error: Error building site: "/Users/calebjasik/Git/jasik-xyz/content/2020-03-30-0119/index.md:7:1": failed to extract shortcode: unclosed shortcode "figure"
Total in 88 ms

Before v0.111.0, this compiled fine, but I'm not sure the "is closed" detector is effective if there is additional shortcodes in the .Inner of the shortcode?

The two patches

Our patch+9 −0
parser/pageparser/pageparser_shortcode_test.goindex 47c33a86d..ab39db569 100644@@ -162,6 +162,15 @@ var shortCodeLexerTests = []lexerTest{ 		tstLeftNoMD, tstSCClose, tstSC1, tstRightNoMD, 		nti(tText, "kl"), tstEOF, 	}, nil},+	{"nested, outer closed after inner (issue 11302)", `{{< figure >}}{{< img src="a.jpg" >}}{{% figcaption %}}cap{{% /figcaption %}}{{< /figure >}}`, []typeText{+		tstLeftNoMD, nti(tScName, "figure"), tstRightNoMD,+		tstLeftNoMD, nti(tScName, "img"), nti(tScParam, "src"), nti(tScParamVal, "a.jpg"), tstRightNoMD,+		tstLeftMD, nti(tScName, "figcaption"), tstRightMD,+		nti(tText, "cap"),+		tstLeftMD, tstSCClose, nti(tScName, "figcaption"), tstRightMD,+		tstLeftNoMD, tstSCClose, nti(tScName, "figure"), tstRightNoMD,+		tstEOF,+	}, nil},  	{"two quoted params", `{{< sc1 "param nr. 1" "param nr. 2" >}}`, []typeText{ 		tstLeftNoMD, tstSC1, nti(tScParam, "param nr. 1"), nti(tScParam, "param nr. 2"), tstRightNoMD, tstEOF, 
Merged fix · PR #+42 −1
hugolib/hugo_sites_build_errors_test.goindex 3209f664d..10e9d123e 100644@@ -421,6 +421,39 @@ line 4 	b.Assert(errors[3].ErrorContext().Lines, qt.DeepEquals, []string{"line 1", "line 2", "123{{ .ThisDoesNotExist }}", "line 4"}) } +// Issue #11302: "failed to extract" error should refer to inner shortcode, not outer.+func TestErrorNestedShortcodeExtractInnerNotFound(t *testing.T) {+	t.Parallel()++	files := `+-- hugo.toml --+-- content/_index.md --+---+title: "Home"+---++## Hello+{{< outer >}}{{< inner >}}{{< /outer >}}++-- layouts/home.html --+{{ .Content }}+-- layouts/_shortcodes/outer.html --+<div>{{ .Inner }}</div>+`++	b, err := TestE(t, files)++	b.Assert(err, qt.IsNotNil)+	fe := herrors.UnwrapFileError(err)+	b.Assert(fe, qt.Not(qt.IsNil))++	// The error should point to the inner shortcode's position (line 6, column 12),+	// not the outer shortcode's position (line 6, column 1).+	b.Assert(fe.Position().LineNumber, qt.Equals, 6)+	b.Assert(fe.Position().ColumnNumber, qt.Equals, 12)+	b.Assert(fe.Error(), qt.Contains, `failed to extract shortcode: template for shortcode "inner" not found`)+}+ func TestErrorRenderHookHeading(t *testing.T) { 	t.Parallel() hugolib/shortcode.goindex c680620cf..5275dc04b 100644@@ -612,6 +612,7 @@ Loop: 			if cnt > 0 { 				// nested shortcode; append it to inner content 				pt.Backup()+				nestedPos := currItem.Pos() 				nested, err := s.extractShortcode(nestedOrdinal, nextLevel, source, pt) 				nestedOrdinal++ 				if nested != nil && nested.name != "" {@@ -621,7 +622,14 @@ Loop: 				if err == nil { 					sc.inner = append(sc.inner, nested) 				} else {-					return sc, err+					// Wrap the error with the nested shortcode's position so that+					// the error points to the correct shortcode (the inner one)+					// rather than the outer shortcode.+					if fe, ok := err.(herrors.FileError); ok {+						return sc, fe+					}+					pos := posFromInput("", source, nestedPos)+					return sc, herrors.NewFileErrorFromPos(err, pos) 				}  			} else { 

What the agent reported

Summary

What was reported: Markdown with an outer shortcode whose .Inner contains nested shortcodes — e.g. {{< figure >}}{{< img … >}}{{% figcaption %}}…{{% /figcaption %}}{{< /figure >}} — failed in Hugo v0.111.x with failed to extract shortcode: unclosed shortcode "figure", even though the outer figure was properly closed. The "is closed" detector was matching against the wrong (outer) shortcode.

What I found: The root cause is already fixed in this checkout. The page lexer (parser/pageparser/pagelexer_shortcode.go) previously validated a closing tag against a single currShortcodeName field, which got overwritten each time a nested shortcode was opened (img, figcaption), so the later {{< /figure >}} no longer matched the "current" name and was flagged as unclosed. That single-value check was replaced by a set-based openShortcodes map (field at line 26; matching logic in lexIdentifierInShortcode, lines 260–270). The fix landed via the shortcode-nesting rework (commit 1e91e4652, an ancestor of HEAD).

I confirmed this by reproducing the exact reported scenario two ways with throwaway tests (both passed, then removed):

  • Lexer-level: the reported delimiter string lexes with no error item.
  • Full end-to-end site build (hugolib.Test) with figure/img/figcaption shortcodes and the verbatim reported markdown — the site built and rendered correctly.

What I changed: Since no production defect remains, no production code was modified (a needless change would violate minimality). I added one regression test to lock in the fixed behavior:

  • parser/pageparser/pageparser_shortcode_test.go — new TestShortcodeLexer table case "nested, outer closed after inner (issue 11302)", asserting the reported input lexes to a clean token stream with no tError (no spurious "unclosed shortcode").

No existing test was modified, weakened, or removed. (The AGENTS.md/CLAUDE.md deletions in git status pre-existed this session and are unrelated.)

Test evidence:

  • Gate ./bt-test -run 'TestShortcodeLexer|TestShortcodeParser|TestPage' ./parser/pageparser/: package PASS, 67 subtests passed, 0 failed.
  • New case TestShortcodeLexer/nested,_outer_closed_after_inner_(issue_11302): PASS.
  • Reviewer confirmed the diff is scoped to only the added test, the token sequence is correct, and nothing unrelated changed.

Note: The reviewer correctly observed the regression test is at the lexer level (the true root cause), not an added end-to-end render test. I separately verified the end-to-end render path passes during investigation; if you'd like, I can also add a permanent integration test (e.g. in tpl/tplimpl/shortcodes_integration_test.go or hugolib) for defense in depth.

All changes are left uncommitted in the working tree.