Backtestsjsouprun-20261006-181938#2501

Issue #2501 Merged fix PR # Run 6 Oct 2026 · run-20261006-181938 jhy/jsoup

Align parser scope classification with HTML spec

Better

Our patch was judged better than the merged fix.

In this judgment, Patch B = our agent, Patch A = the merged fix.

Both patches make the same changes to the scope lists: they add `select` to TagsSearchInScope and `template` to TagSearchTableScope, keeping both arrays sorted. The difference is in the SVG special-element fix.

Patch B checks the normalized name against TagSearchInScopeSvg, the existing lowercase list (`desc`, `foreignobject`, `title`). This removes the case mismatch at its root and works however the element was built or which ParseSettings were used.

Patch A instead compares the case-preserved `tagName()` against TagSvgHtmlIntegration using an unsorted linear `in`. That works when the parser has case-adjusted the name to `foreignObject`. It fails for an SVG element whose tagName is lowercase `foreignobject`, which can happen through programmatic construction or other settings. Patch A also switches away from the `inSorted` normalized-name convention the rest of `isSpecial` uses.

On tests, Patch A adds only two membership assertions, which are close to tautological, and one `isSpecial` check on a parsed foreignObject. It also deletes three imports with no explanation. Patch B adds behavioral tests that would fail before the fix and pass after it:
- `li` and `dd` inside foreignObject stay contained.
- A stray `</x-a>` stops at foreignObject.
- A stray `</tr>` inside a template in a cell does not escape the template.
- Direct `inScope` and `inTableScope` checks on a constructed stack confirm the boundary semantics, including that `select` does not affect table scope.

Patch B's changelog entry is also more informative. Overall, B is more robust and much better tested.

The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.

Solve

16m 9s

Wall-clock time in the container

Full test suite

No new failures

2,017 passed · 0 skipped · 2,017 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

79vs 21 merged

Changed lines, ours vs merged

I saw that jsoup's HTML tree-builder classification lists are missing a few entries from the current WHATWG HTML parsing spec: select is not included as an ordinary in-scope boundary, template is not included as a table-scope boundary, and SVG foreignObject is not treated as a special element (because the special element check compares the normalized name (foreignobject) with the case-preserved integration-point name (foreignObject))

Spec references:

The two patches

Our patch+76 −3
CHANGES.mdindex 54402a55..cad7b556 100644@@ -6,6 +6,9 @@ * Reduced retained memory when parsing with source position tracking enabled (`Parser#setTrackPosition(true)`). Source ranges are now stored in compact parser-owned span records instead of node and attribute user data, and `Position` objects are created lazily when source ranges are read. This cuts tracked DOM retained size by about 50-60% on representative benchmark documents, while keeping `Node#sourceRange()`, `Element#endSourceRange()`, and `Attribute#sourceRange()` behavior intact. [#2498](https://github.com/jhy/jsoup/pull/2498) * Added `Element#classList()`, an immutable snapshot of an element's class names in attribute order. Use `hasClass()` when you just need to test for one class, `classList()` when you want to read or iterate classes without needing a mutable result, and `classNames()` when you want the existing mutable, deduplicated set that can be written back with `classNames(Set)`. The class APIs now share an HTML-whitespace scanner, which also makes `classNames()` faster and lighter on allocation, especially when walking many elements without class names. [#2500](https://github.com/jhy/jsoup/pull/2500) +### Bug Fixes+* Aligned the HTML tree builder's scope and special element classifications with the current HTML spec: `select` is now a boundary for "has an element in scope", `template` is a boundary for "has an element in table scope", and SVG `foreignObject` is now treated as a special element (previously missed due to a case mismatch). For example, a stray end tag inside an SVG `foreignObject` (such as `</x-a>`) no longer closes an outer element and pops the `foreignObject`, and a stray `</tr>` inside a `template` within a table cell no longer closes the template.+ ### Build Changes * Cleaned up the Maven build for the multi-release JAR so Java 8 and Java 11+ sources compile as separate source sets. This avoids spurious Java 8 compiler warnings from newer-language overlay sources, keeps long-running parser checks behind an explicit profile, and preserves the same published artifacts and runtime behavior. src/main/java/org/jsoup/parser/HtmlTreeBuilder.javaindex a2d31415..c3f5ec9d 100644@@ -29,7 +29,7 @@ import static org.jsoup.parser.Parser.*; public class HtmlTreeBuilder extends TreeBuilder {     // tag searches. must be sorted, used in inSorted. HtmlTreeBuilderTest validates they're sorted.     static final String[] TagsSearchInScope = new String[]{ // a particular element in scope-        "applet", "caption", "html", "marquee", "object", "table", "td", "template", "th"+        "applet", "caption", "html", "marquee", "object", "select", "table", "td", "template", "th"     };     // math and svg namespaces for particular element in scope     static final String[]TagSearchInScopeMath = new String[] {@@ -41,7 +41,7 @@ public class HtmlTreeBuilder extends TreeBuilder {      static final String[] TagSearchList = new String[]{"ol", "ul"};     static final String[] TagSearchButton = new String[]{"button"};-    static final String[] TagSearchTableScope = new String[]{"html", "table"};+    static final String[] TagSearchTableScope = new String[]{"html", "table", "template"};     static final String[] TagSearchSelectScope = new String[]{"optgroup", "option"};     static final String[] TagSearchEndTags = new String[]{"dd", "dt", "li", "optgroup", "option", "p", "rb", "rp", "rt", "rtc"};     static final String[] TagThoroughSearchEndTags = new String[]{"caption", "colgroup", "dd", "dt", "li", "optgroup", "option", "p", "rb", "rp", "rt", "rtc", "tbody", "td", "tfoot", "th", "thead", "tr"};@@ -876,7 +876,7 @@ public class HtmlTreeBuilder extends TreeBuilder {             case Parser.NamespaceMathml:                 return inSorted(name, TagSearchSpecialMath);             case Parser.NamespaceSvg:-                return inSorted(name, TagSvgHtmlIntegration);+                return inSorted(name, TagSearchInScopeSvg); // normalized (lowercase) names, so foreignObject matches             default:                 return false;         }src/test/java/org/jsoup/parser/HtmlTreeBuilderTest.javaindex 862fe1bc..5d222d55 100644@@ -2,6 +2,7 @@ package org.jsoup.parser;   import org.jsoup.Jsoup;+import org.jsoup.TextUtil; import org.jsoup.nodes.DataNode; import org.jsoup.nodes.Document; import org.jsoup.nodes.Element;@@ -65,6 +66,75 @@ public class HtmlTreeBuilderTest {         assertFalse(HtmlTreeBuilder.isSpecial(notSvgEl));     } +    @Test void svgForeignObjectIsSpecial() {+        // foreignObject's normal name is lowercased, so isSpecial must match against the normalized name+        Element foreignObject = new Element(Tag.valueOf("foreignObject", Parser.NamespaceSvg, ParseSettings.preserveCase), "");+        assertEquals("foreignobject", foreignObject.normalName());+        assertTrue(HtmlTreeBuilder.isSpecial(foreignObject));++        Element desc = new Element(Tag.valueOf("desc", Parser.NamespaceSvg, ParseSettings.preserveCase), "");+        assertTrue(HtmlTreeBuilder.isSpecial(desc));+    }++    @Test void liInsideSvgForeignObjectStaysInForeignObject() {+        // the li start tag's "close li" loop must stop at foreignObject, as it is a special element+        Document doc = Jsoup.parse("<ul><li>a<svg><foreignObject><li>b</li></foreignObject></svg></li></ul>");+        Element foreignObject = doc.expectFirst("foreignObject");+        assertEquals("<li>b</li>", foreignObject.html());+        assertEquals(1, doc.select("ul > li").size());+        assertEquals("<ul><li>a<svg><foreignObject><li>b</li></foreignObject></svg></li></ul>", TextUtil.stripNewlines(doc.body().html()));+    }++    @Test void ddInsideSvgForeignObjectStaysInForeignObject() {+        Document doc = Jsoup.parse("<dl><dd>a<svg><foreignObject><dd>b</dd></foreignObject></svg></dd></dl>");+        assertEquals("<dd>b</dd>", doc.expectFirst("foreignObject").html());+        assertEquals(1, doc.select("dl > dd").size());+    }++    @Test void anyOtherEndTagStopsAtSvgForeignObject() {+        // the generic end tag loop must stop at foreignObject (special), so </x-a> is ignored and c stays in foreignObject+        Document doc = Jsoup.parse("<x-a>a<svg><foreignObject>b</x-a>c</foreignObject></svg></x-a>");+        assertEquals("bc", doc.expectFirst("foreignObject").html());+        assertEquals("<x-a>a<svg><foreignObject>bc</foreignObject></svg></x-a>", TextUtil.stripNewlines(doc.body().html()));+    }++    @Test void selectIsInScopeBoundary() {+        HtmlTreeBuilder tb = treeBuilderWithStack("html", "body", "p", "select", "div");+        assertFalse(tb.inScope("p")); // select is a boundary for "has an element in scope"+        assertFalse(tb.inButtonScope("p"));+        assertFalse(tb.inListItemScope("p"));+        assertTrue(tb.inScope("div"));+        assertTrue(tb.inScope("select"));+        assertTrue(tb.inTableScope("p")); // but not for table scope++        HtmlTreeBuilder noSelect = treeBuilderWithStack("html", "body", "p", "div");+        assertTrue(noSelect.inScope("p"));+    }++    @Test void templateIsTableScopeBoundary() {+        HtmlTreeBuilder tb = treeBuilderWithStack("html", "body", "table", "tbody", "tr", "td", "template", "td");+        assertFalse(tb.inTableScope("tr")); // template is a boundary for "has an element in table scope"+        assertFalse(tb.inTableScope("table"));+        assertTrue(tb.inTableScope("td"));+        assertTrue(tb.inTableScope("template"));+    }++    @Test void endTrInsideTemplateCellDoesNotEscapeTemplate() {+        // the </tr> must not see the outer tr through the template (table scope), so is ignored+        Document doc = Jsoup.parse("<table><tr><td><template><td>a</tr>b</template></td></tr></table>");+        Element template = doc.expectFirst("template");+        assertEquals("<td>ab</td>", TextUtil.stripNewlines(template.html()));+        assertEquals("<table><tbody><tr><td><template><td>ab</td></template></td></tr></tbody></table>", TextUtil.stripNewlines(doc.body().html()));+    }++    private static HtmlTreeBuilder treeBuilderWithStack(String... names) {+        HtmlTreeBuilder tb = new HtmlTreeBuilder();+        tb.initialiseParse(new java.io.StringReader(""), "", Parser.htmlParser());+        for (String name : names)+            tb.push(new Element(Tag.valueOf(name, NamespaceHtml, ParseSettings.htmlDefault), ""));+        return tb;+    }+     @Test void customRcdataTag() {         String inner = "Blah\nblah\n<foo>Foo</foo>\n&quot;";         String innerText = "Blah\nblah\n<foo>Foo</foo>\n\""; 
Merged fix · PR #+15 −6
CHANGES.mdindex 54402a55..539d2af7 100644@@ -5,6 +5,7 @@ ### Improvements * Reduced retained memory when parsing with source position tracking enabled (`Parser#setTrackPosition(true)`). Source ranges are now stored in compact parser-owned span records instead of node and attribute user data, and `Position` objects are created lazily when source ranges are read. This cuts tracked DOM retained size by about 50-60% on representative benchmark documents, while keeping `Node#sourceRange()`, `Element#endSourceRange()`, and `Attribute#sourceRange()` behavior intact. [#2498](https://github.com/jhy/jsoup/pull/2498) * Added `Element#classList()`, an immutable snapshot of an element's class names in attribute order. Use `hasClass()` when you just need to test for one class, `classList()` when you want to read or iterate classes without needing a mutable result, and `classNames()` when you want the existing mutable, deduplicated set that can be written back with `classNames(Set)`. The class APIs now share an HTML-whitespace scanner, which also makes `classNames()` faster and lighter on allocation, especially when walking many elements without class names. [#2500](https://github.com/jhy/jsoup/pull/2500)+* Aligned HTML parser scope classification with the current HTML spec for `select`, `foreignObject`, and `template`. [#2501](https://github.com/jhy/jsoup/issues/2501)  ### Build Changes * Cleaned up the Maven build for the multi-release JAR so Java 8 and Java 11+ sources compile as separate source sets. This avoids spurious Java 8 compiler warnings from newer-language overlay sources, keeps long-running parser checks behind an explicit profile, and preserves the same published artifacts and runtime behavior.src/main/java/org/jsoup/parser/HtmlTreeBuilder.javaindex a2d31415..cca440e5 100644@@ -29,7 +29,7 @@ import static org.jsoup.parser.Parser.*; public class HtmlTreeBuilder extends TreeBuilder {     // tag searches. must be sorted, used in inSorted. HtmlTreeBuilderTest validates they're sorted.     static final String[] TagsSearchInScope = new String[]{ // a particular element in scope-        "applet", "caption", "html", "marquee", "object", "table", "td", "template", "th"+        "applet", "caption", "html", "marquee", "object", "select", "table", "td", "template", "th"     };     // math and svg namespaces for particular element in scope     static final String[]TagSearchInScopeMath = new String[] {@@ -41,7 +41,7 @@ public class HtmlTreeBuilder extends TreeBuilder {      static final String[] TagSearchList = new String[]{"ol", "ul"};     static final String[] TagSearchButton = new String[]{"button"};-    static final String[] TagSearchTableScope = new String[]{"html", "table"};+    static final String[] TagSearchTableScope = new String[]{"html", "table", "template"};     static final String[] TagSearchSelectScope = new String[]{"optgroup", "option"};     static final String[] TagSearchEndTags = new String[]{"dd", "dt", "li", "optgroup", "option", "p", "rb", "rp", "rt", "rtc"};     static final String[] TagThoroughSearchEndTags = new String[]{"caption", "colgroup", "dd", "dt", "li", "optgroup", "option", "p", "rb", "rp", "rt", "rtc", "tbody", "td", "tfoot", "th", "thead", "tr"};@@ -876,7 +876,8 @@ public class HtmlTreeBuilder extends TreeBuilder {             case Parser.NamespaceMathml:                 return inSorted(name, TagSearchSpecialMath);             case Parser.NamespaceSvg:-                return inSorted(name, TagSvgHtmlIntegration);+                // SVG tag names may be case-adjusted, e.g. foreignObject, so match the preserved tagName()+                return StringUtil.in(el.tagName(), TagSvgHtmlIntegration);             default:                 return false;         }src/test/java/org/jsoup/parser/HtmlTreeBuilderTest.javaindex 862fe1bc..1bf62e66 100644@@ -2,16 +2,14 @@ package org.jsoup.parser;   import org.jsoup.Jsoup;-import org.jsoup.nodes.DataNode; import org.jsoup.nodes.Document; import org.jsoup.nodes.Element; import org.jspecify.annotations.NullMarked; import org.junit.jupiter.api.Test;-import java.io.Reader; import java.lang.annotation.Annotation;-import java.lang.reflect.Method; import java.util.List; +import static org.jsoup.internal.StringUtil.inSorted; import static org.jsoup.parser.Parser.NamespaceHtml; import static org.junit.jupiter.api.Assertions.*; @@ -23,6 +21,12 @@ public class HtmlTreeBuilderTest {         assertEquals(14, constants.size());     } +    @Test+    public void scopeSearchesMatchSpecBoundaries() {+        assertTrue(inSorted("select", HtmlTreeBuilder.TagsSearchInScope));+        assertTrue(inSorted("template", HtmlTreeBuilder.TagSearchTableScope));+    }+     @Test     public void nonnull() {         assertThrows(IllegalArgumentException.class, () -> {@@ -61,6 +65,9 @@ public class HtmlTreeBuilderTest {         Element svgEl = new Element(Tag.valueOf("title", Parser.NamespaceSvg, settings), "");         assertTrue(HtmlTreeBuilder.isSpecial(svgEl)); +        Element svgForeignObject = Jsoup.parse("<svg><foreignObject></foreignObject></svg>").expectFirst("foreignObject");+        assertTrue(HtmlTreeBuilder.isSpecial(svgForeignObject));+         Element notSvgEl = new Element(Tag.valueOf("not-svg", Parser.NamespaceSvg, settings), "");         assertFalse(HtmlTreeBuilder.isSpecial(notSvgEl));     }