Backtestsjsouprun-20261006-181938#2501
Align parser scope classification with HTML spec
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Both patches make the same changes to the scope lists: they add `select` to TagsSearchInScope and `template` to TagSearchTableScope, keeping both arrays sorted. The difference is in the SVG special-element fix. Patch B checks the normalized name against TagSearchInScopeSvg, the existing lowercase list (`desc`, `foreignobject`, `title`). This removes the case mismatch at its root and works however the element was built or which ParseSettings were used. Patch A instead compares the case-preserved `tagName()` against TagSvgHtmlIntegration using an unsorted linear `in`. That works when the parser has case-adjusted the name to `foreignObject`. It fails for an SVG element whose tagName is lowercase `foreignobject`, which can happen through programmatic construction or other settings. Patch A also switches away from the `inSorted` normalized-name convention the rest of `isSpecial` uses. On tests, Patch A adds only two membership assertions, which are close to tautological, and one `isSpecial` check on a parsed foreignObject. It also deletes three imports with no explanation. Patch B adds behavioral tests that would fail before the fix and pass after it: - `li` and `dd` inside foreignObject stay contained. - A stray `</x-a>` stops at foreignObject. - A stray `</tr>` inside a template in a cell does not escape the template. - Direct `inScope` and `inTableScope` checks on a constructed stack confirm the boundary semantics, including that `select` does not affect table scope. Patch B's changelog entry is also more informative. Overall, B is more robust and much better tested.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
16m 9s
Wall-clock time in the container
Full test suite
No new failures
2,017 passed · 0 skipped · 2,017 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
79vs 21 merged
Changed lines, ours vs merged
The issue
View on GitHubI saw that jsoup's HTML tree-builder classification lists are missing a few entries from the current WHATWG HTML parsing spec: select is not included as an ordinary in-scope boundary, template is not included as a table-scope boundary, and SVG foreignObject is not treated as a special element (because the special element check compares the normalized name (foreignobject) with the case-preserved integration-point name (foreignObject))
Spec references:
- Has an element in scope: https://html.spec.whatwg.org/multipage/parsing.html#has-an-element-in-scope
- Has an element in table scope: https://html.spec.whatwg.org/multipage/parsing.html#has-an-element-in-table-scope
- Special elements: https://html.spec.whatwg.org/multipage/parsing.html#special
The two patches
CHANGES.mdindex 54402a55..cad7b556 100644@@ -6,6 +6,9 @@ * Reduced retained memory when parsing with source position tracking enabled (`Parser#setTrackPosition(true)`). Source ranges are now stored in compact parser-owned span records instead of node and attribute user data, and `Position` objects are created lazily when source ranges are read. This cuts tracked DOM retained size by about 50-60% on representative benchmark documents, while keeping `Node#sourceRange()`, `Element#endSourceRange()`, and `Attribute#sourceRange()` behavior intact. [#2498](https://github.com/jhy/jsoup/pull/2498) * Added `Element#classList()`, an immutable snapshot of an element's class names in attribute order. Use `hasClass()` when you just need to test for one class, `classList()` when you want to read or iterate classes without needing a mutable result, and `classNames()` when you want the existing mutable, deduplicated set that can be written back with `classNames(Set)`. The class APIs now share an HTML-whitespace scanner, which also makes `classNames()` faster and lighter on allocation, especially when walking many elements without class names. [#2500](https://github.com/jhy/jsoup/pull/2500) +### Bug Fixes+* Aligned the HTML tree builder's scope and special element classifications with the current HTML spec: `select` is now a boundary for "has an element in scope", `template` is a boundary for "has an element in table scope", and SVG `foreignObject` is now treated as a special element (previously missed due to a case mismatch). For example, a stray end tag inside an SVG `foreignObject` (such as `</x-a>`) no longer closes an outer element and pops the `foreignObject`, and a stray `</tr>` inside a `template` within a table cell no longer closes the template.+ ### Build Changes * Cleaned up the Maven build for the multi-release JAR so Java 8 and Java 11+ sources compile as separate source sets. This avoids spurious Java 8 compiler warnings from newer-language overlay sources, keeps long-running parser checks behind an explicit profile, and preserves the same published artifacts and runtime behavior. src/main/java/org/jsoup/parser/HtmlTreeBuilder.javaindex a2d31415..c3f5ec9d 100644@@ -29,7 +29,7 @@ import static org.jsoup.parser.Parser.*; public class HtmlTreeBuilder extends TreeBuilder { // tag searches. must be sorted, used in inSorted. HtmlTreeBuilderTest validates they're sorted. static final String[] TagsSearchInScope = new String[]{ // a particular element in scope- "applet", "caption", "html", "marquee", "object", "table", "td", "template", "th"+ "applet", "caption", "html", "marquee", "object", "select", "table", "td", "template", "th" }; // math and svg namespaces for particular element in scope static final String[]TagSearchInScopeMath = new String[] {@@ -41,7 +41,7 @@ public class HtmlTreeBuilder extends TreeBuilder { static final String[] TagSearchList = new String[]{"ol", "ul"}; static final String[] TagSearchButton = new String[]{"button"};- static final String[] TagSearchTableScope = new String[]{"html", "table"};+ static final String[] TagSearchTableScope = new String[]{"html", "table", "template"}; static final String[] TagSearchSelectScope = new String[]{"optgroup", "option"}; static final String[] TagSearchEndTags = new String[]{"dd", "dt", "li", "optgroup", "option", "p", "rb", "rp", "rt", "rtc"}; static final String[] TagThoroughSearchEndTags = new String[]{"caption", "colgroup", "dd", "dt", "li", "optgroup", "option", "p", "rb", "rp", "rt", "rtc", "tbody", "td", "tfoot", "th", "thead", "tr"};@@ -876,7 +876,7 @@ public class HtmlTreeBuilder extends TreeBuilder { case Parser.NamespaceMathml: return inSorted(name, TagSearchSpecialMath); case Parser.NamespaceSvg:- return inSorted(name, TagSvgHtmlIntegration);+ return inSorted(name, TagSearchInScopeSvg); // normalized (lowercase) names, so foreignObject matches default: return false; }src/test/java/org/jsoup/parser/HtmlTreeBuilderTest.javaindex 862fe1bc..5d222d55 100644@@ -2,6 +2,7 @@ package org.jsoup.parser; import org.jsoup.Jsoup;+import org.jsoup.TextUtil; import org.jsoup.nodes.DataNode; import org.jsoup.nodes.Document; import org.jsoup.nodes.Element;@@ -65,6 +66,75 @@ public class HtmlTreeBuilderTest { assertFalse(HtmlTreeBuilder.isSpecial(notSvgEl)); } + @Test void svgForeignObjectIsSpecial() {+ // foreignObject's normal name is lowercased, so isSpecial must match against the normalized name+ Element foreignObject = new Element(Tag.valueOf("foreignObject", Parser.NamespaceSvg, ParseSettings.preserveCase), "");+ assertEquals("foreignobject", foreignObject.normalName());+ assertTrue(HtmlTreeBuilder.isSpecial(foreignObject));++ Element desc = new Element(Tag.valueOf("desc", Parser.NamespaceSvg, ParseSettings.preserveCase), "");+ assertTrue(HtmlTreeBuilder.isSpecial(desc));+ }++ @Test void liInsideSvgForeignObjectStaysInForeignObject() {+ // the li start tag's "close li" loop must stop at foreignObject, as it is a special element+ Document doc = Jsoup.parse("<ul><li>a<svg><foreignObject><li>b</li></foreignObject></svg></li></ul>");+ Element foreignObject = doc.expectFirst("foreignObject");+ assertEquals("<li>b</li>", foreignObject.html());+ assertEquals(1, doc.select("ul > li").size());+ assertEquals("<ul><li>a<svg><foreignObject><li>b</li></foreignObject></svg></li></ul>", TextUtil.stripNewlines(doc.body().html()));+ }++ @Test void ddInsideSvgForeignObjectStaysInForeignObject() {+ Document doc = Jsoup.parse("<dl><dd>a<svg><foreignObject><dd>b</dd></foreignObject></svg></dd></dl>");+ assertEquals("<dd>b</dd>", doc.expectFirst("foreignObject").html());+ assertEquals(1, doc.select("dl > dd").size());+ }++ @Test void anyOtherEndTagStopsAtSvgForeignObject() {+ // the generic end tag loop must stop at foreignObject (special), so </x-a> is ignored and c stays in foreignObject+ Document doc = Jsoup.parse("<x-a>a<svg><foreignObject>b</x-a>c</foreignObject></svg></x-a>");+ assertEquals("bc", doc.expectFirst("foreignObject").html());+ assertEquals("<x-a>a<svg><foreignObject>bc</foreignObject></svg></x-a>", TextUtil.stripNewlines(doc.body().html()));+ }++ @Test void selectIsInScopeBoundary() {+ HtmlTreeBuilder tb = treeBuilderWithStack("html", "body", "p", "select", "div");+ assertFalse(tb.inScope("p")); // select is a boundary for "has an element in scope"+ assertFalse(tb.inButtonScope("p"));+ assertFalse(tb.inListItemScope("p"));+ assertTrue(tb.inScope("div"));+ assertTrue(tb.inScope("select"));+ assertTrue(tb.inTableScope("p")); // but not for table scope++ HtmlTreeBuilder noSelect = treeBuilderWithStack("html", "body", "p", "div");+ assertTrue(noSelect.inScope("p"));+ }++ @Test void templateIsTableScopeBoundary() {+ HtmlTreeBuilder tb = treeBuilderWithStack("html", "body", "table", "tbody", "tr", "td", "template", "td");+ assertFalse(tb.inTableScope("tr")); // template is a boundary for "has an element in table scope"+ assertFalse(tb.inTableScope("table"));+ assertTrue(tb.inTableScope("td"));+ assertTrue(tb.inTableScope("template"));+ }++ @Test void endTrInsideTemplateCellDoesNotEscapeTemplate() {+ // the </tr> must not see the outer tr through the template (table scope), so is ignored+ Document doc = Jsoup.parse("<table><tr><td><template><td>a</tr>b</template></td></tr></table>");+ Element template = doc.expectFirst("template");+ assertEquals("<td>ab</td>", TextUtil.stripNewlines(template.html()));+ assertEquals("<table><tbody><tr><td><template><td>ab</td></template></td></tr></tbody></table>", TextUtil.stripNewlines(doc.body().html()));+ }++ private static HtmlTreeBuilder treeBuilderWithStack(String... names) {+ HtmlTreeBuilder tb = new HtmlTreeBuilder();+ tb.initialiseParse(new java.io.StringReader(""), "", Parser.htmlParser());+ for (String name : names)+ tb.push(new Element(Tag.valueOf(name, NamespaceHtml, ParseSettings.htmlDefault), ""));+ return tb;+ }+ @Test void customRcdataTag() { String inner = "Blah\nblah\n<foo>Foo</foo>\n""; String innerText = "Blah\nblah\n<foo>Foo</foo>\n\"";
CHANGES.mdindex 54402a55..539d2af7 100644@@ -5,6 +5,7 @@ ### Improvements * Reduced retained memory when parsing with source position tracking enabled (`Parser#setTrackPosition(true)`). Source ranges are now stored in compact parser-owned span records instead of node and attribute user data, and `Position` objects are created lazily when source ranges are read. This cuts tracked DOM retained size by about 50-60% on representative benchmark documents, while keeping `Node#sourceRange()`, `Element#endSourceRange()`, and `Attribute#sourceRange()` behavior intact. [#2498](https://github.com/jhy/jsoup/pull/2498) * Added `Element#classList()`, an immutable snapshot of an element's class names in attribute order. Use `hasClass()` when you just need to test for one class, `classList()` when you want to read or iterate classes without needing a mutable result, and `classNames()` when you want the existing mutable, deduplicated set that can be written back with `classNames(Set)`. The class APIs now share an HTML-whitespace scanner, which also makes `classNames()` faster and lighter on allocation, especially when walking many elements without class names. [#2500](https://github.com/jhy/jsoup/pull/2500)+* Aligned HTML parser scope classification with the current HTML spec for `select`, `foreignObject`, and `template`. [#2501](https://github.com/jhy/jsoup/issues/2501) ### Build Changes * Cleaned up the Maven build for the multi-release JAR so Java 8 and Java 11+ sources compile as separate source sets. This avoids spurious Java 8 compiler warnings from newer-language overlay sources, keeps long-running parser checks behind an explicit profile, and preserves the same published artifacts and runtime behavior.src/main/java/org/jsoup/parser/HtmlTreeBuilder.javaindex a2d31415..cca440e5 100644@@ -29,7 +29,7 @@ import static org.jsoup.parser.Parser.*; public class HtmlTreeBuilder extends TreeBuilder { // tag searches. must be sorted, used in inSorted. HtmlTreeBuilderTest validates they're sorted. static final String[] TagsSearchInScope = new String[]{ // a particular element in scope- "applet", "caption", "html", "marquee", "object", "table", "td", "template", "th"+ "applet", "caption", "html", "marquee", "object", "select", "table", "td", "template", "th" }; // math and svg namespaces for particular element in scope static final String[]TagSearchInScopeMath = new String[] {@@ -41,7 +41,7 @@ public class HtmlTreeBuilder extends TreeBuilder { static final String[] TagSearchList = new String[]{"ol", "ul"}; static final String[] TagSearchButton = new String[]{"button"};- static final String[] TagSearchTableScope = new String[]{"html", "table"};+ static final String[] TagSearchTableScope = new String[]{"html", "table", "template"}; static final String[] TagSearchSelectScope = new String[]{"optgroup", "option"}; static final String[] TagSearchEndTags = new String[]{"dd", "dt", "li", "optgroup", "option", "p", "rb", "rp", "rt", "rtc"}; static final String[] TagThoroughSearchEndTags = new String[]{"caption", "colgroup", "dd", "dt", "li", "optgroup", "option", "p", "rb", "rp", "rt", "rtc", "tbody", "td", "tfoot", "th", "thead", "tr"};@@ -876,7 +876,8 @@ public class HtmlTreeBuilder extends TreeBuilder { case Parser.NamespaceMathml: return inSorted(name, TagSearchSpecialMath); case Parser.NamespaceSvg:- return inSorted(name, TagSvgHtmlIntegration);+ // SVG tag names may be case-adjusted, e.g. foreignObject, so match the preserved tagName()+ return StringUtil.in(el.tagName(), TagSvgHtmlIntegration); default: return false; }src/test/java/org/jsoup/parser/HtmlTreeBuilderTest.javaindex 862fe1bc..1bf62e66 100644@@ -2,16 +2,14 @@ package org.jsoup.parser; import org.jsoup.Jsoup;-import org.jsoup.nodes.DataNode; import org.jsoup.nodes.Document; import org.jsoup.nodes.Element; import org.jspecify.annotations.NullMarked; import org.junit.jupiter.api.Test;-import java.io.Reader; import java.lang.annotation.Annotation;-import java.lang.reflect.Method; import java.util.List; +import static org.jsoup.internal.StringUtil.inSorted; import static org.jsoup.parser.Parser.NamespaceHtml; import static org.junit.jupiter.api.Assertions.*; @@ -23,6 +21,12 @@ public class HtmlTreeBuilderTest { assertEquals(14, constants.size()); } + @Test+ public void scopeSearchesMatchSpecBoundaries() {+ assertTrue(inSorted("select", HtmlTreeBuilder.TagsSearchInScope));+ assertTrue(inSorted("template", HtmlTreeBuilder.TagSearchTableScope));+ }+ @Test public void nonnull() { assertThrows(IllegalArgumentException.class, () -> {@@ -61,6 +65,9 @@ public class HtmlTreeBuilderTest { Element svgEl = new Element(Tag.valueOf("title", Parser.NamespaceSvg, settings), ""); assertTrue(HtmlTreeBuilder.isSpecial(svgEl)); + Element svgForeignObject = Jsoup.parse("<svg><foreignObject></foreignObject></svg>").expectFirst("foreignObject");+ assertTrue(HtmlTreeBuilder.isSpecial(svgForeignObject));+ Element notSvgEl = new Element(Tag.valueOf("not-svg", Parser.NamespaceSvg, settings), ""); assertFalse(HtmlTreeBuilder.isSpecial(notSvgEl)); }