Backtestsjsouprun-20261006-181938#2562
Pop foreign elements before processing HTML breakout tags
Our patch was judged better than the merged fix.
In this judgment, Patch A = our agent, Patch B = the merged fix.
Both patches add the core fix the issue asks for. When a breakout start tag (or `</p>`/`</br>`) appears in foreign content, they pop elements until reaching an HTML-namespace element, a MathML text integration point, or an HTML integration point, then reprocess the token as HTML. Both handle the issue's example correctly and update the existing mixed-namespace test. Patch A is more correct at an edge that the new popping behavior makes important. The existing `isHtmlIntegration` matched `foreignObject` case-sensitively on `tagName`. jsoup preserves foreign tag case, so input like `<svg><foreignobject><p>` would not be seen as an integration point. Once breakout pops elements, Patch B would pop that `foreignobject`/`DESC` and move the `<p>` out of the SVG. Browsers don't do this, because the tokenizer lowercases tags and the tree builder adjusts them to `foreignObject`. Patch A fixes this by matching on `normalName` against a lowercased list, and adds a test for it. Patch A also: - explicitly avoids popping the bottom element of the stack, which guards fragment and root edge cases; - records a parse error, as the spec requires. On tests, Patch A is much more thorough. It covers: - nested SVG and MathML; - `font` with and without color/face/size attributes; - end-tag breakouts (`</p>`, `</br>`); - stopping at `foreignObject`, `mi`, and `annotation-xml` integration points; - lowercase integration points; - fragment parsing with an SVG context. Patch B tests the basic case, the `</p>` end tag, and a negative case for other end tags. Patch A is stronger on correctness, tests, and craft.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
12m 51s
Wall-clock time in the container
Full test suite
No new failures
2,159 passed · 0 skipped · 2,159 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
124vs 45 merged
Changed lines, ours vs merged
The issue
View on GitHubThe HTML parser reprocesses foreign-content breakout tags as HTML, but does not first pop the foreign elements from the stack of open elements as required by the HTML specification.
For example:
Document doc = Jsoup.parse("<svg><s><p>One</p></s></svg>");
System.out.println(doc.body().html());
jsoup currently leaves the HTML <s> inside <svg>:
<svg>
<s>
<p>One</p>
</s>
</svg>
Browsers close the foreign content before processing <s>:
<svg></svg>
<s>
<p>One</p>
</s>
Need to update the foreign-content breakout handling to pop elements until reaching a MathML text integration point, an HTML integration point, or an element in the HTML namespace before reprocessing the token.
Spec: https://html.spec.whatwg.org/multipage/parsing.html#parsing-main-inforeign
The two patches
CHANGES.mdindex dfe7b1f9..b0e9d7d2 100644@@ -20,6 +20,7 @@ * Fixed HTTP response content-type matching to handle media types case-insensitively and recognize structured `+xml` suffixes, including vendor-specific media types. [#2550](https://github.com/jhy/jsoup/pull/2550) * Corrected multipart form encoding to percent-escape CR and LF in field names and filenames, matching the HTML form submission specification. Multipart file content-types containing CR or LF are now rejected with a `ValidationException`. [#2555](https://github.com/jhy/jsoup/pull/2555) * Aligned trailing comment placement with the HTML specification: comments after `</body>` remain children of the `html` element, while comments after `</html>` remain children of the document. [#2557](https://github.com/jhy/jsoup/pull/2557)+* Aligned foreign content breakout handling with the HTML specification: when an HTML start tag such as `<p>`, `<s>`, or `<b>` (or `</p>`, `</br>`) appears inside SVG or MathML content, the open foreign elements are now closed before the tag is processed as HTML, up to the nearest HTML element or integration point. E.g. `<svg><s><p>One</p></s></svg>` now parses to `<svg></svg><s><p>One</p></s>`, rather than leaving the `<s>` inside the `<svg>`. * When using the optional `re2j` regular expression engine, heap exhaustion caused by complex selector patterns during matching is now normalized to a `ValidationException` with a `Pattern complexity error` message. src/main/java/org/jsoup/parser/HtmlTreeBuilder.javaindex eaeeccb1..ee505a0a 100644@@ -29,7 +29,7 @@ import static org.jsoup.parser.Parser.*; */ public class HtmlTreeBuilder extends TreeBuilder { static final String[] TagMathMlTextIntegration = new String[]{"mi", "mn", "mo", "ms", "mtext"};- static final String[] TagSvgHtmlIntegration = new String[]{"desc", "foreignObject", "title"};+ static final String[] TagSvgHtmlIntegration = new String[]{"desc", "foreignobject", "title"}; static final String[] TagFormListed = { "button", "fieldset", "input", "keygen", "object", "output", "select", "textarea" };@@ -304,8 +304,8 @@ public class HtmlTreeBuilder extends TreeBuilder { if (encoding.equals("text/html") || encoding.equals("application/xhtml+xml")) return true; }- // note using .tagName for case-sensitive hit here of foreignObject- return Parser.NamespaceSvg.equals(el.tag().namespace()) && StringUtil.in(el.tagName(), TagSvgHtmlIntegration);+ // note using .normalName, as foreign tag names preserve the input case (foreignObject / foreignobject)+ return Parser.NamespaceSvg.equals(el.tag().namespace()) && StringUtil.in(el.normalName(), TagSvgHtmlIntegration); } boolean process(Token token, HtmlTreeBuilderState state) {src/main/java/org/jsoup/parser/HtmlTreeBuilderState.javaindex 84b8504c..c0aedb32 100644@@ -1768,12 +1768,12 @@ enum HtmlTreeBuilderState { case StartTag: Token.StartTag start = t.asStartTag(); if (StringUtil.in(start.normalName, InForeignToHtml))- return processAsHtml(t, tb);+ return breakoutAsHtml(t, tb); if (start.normalName.equals("font") && ( start.hasAttributeIgnoreCase("color") || start.hasAttributeIgnoreCase("face") || start.hasAttributeIgnoreCase("size")))- return processAsHtml(t, tb);+ return breakoutAsHtml(t, tb); // Any other start: // (whatwg says to fix up tag name and attribute case per a table - we will preserve original case instead)@@ -1796,7 +1796,7 @@ enum HtmlTreeBuilderState { case EndTag: Token.EndTag end = t.asEndTag(); if (end.normalName.equals("br") || end.normalName.equals("p"))- return processAsHtml(t, tb);+ return breakoutAsHtml(t, tb); if (end.normalName.equals("script") && tb.currentElementIs("script", Parser.NamespaceSvg)) { // script level and execution elided. tb.pop();@@ -1836,6 +1836,25 @@ enum HtmlTreeBuilderState { boolean processAsHtml(Token t, HtmlTreeBuilder tb) { return tb.state().process(t, tb); }++ /**+ Breaks out of foreign content: pops elements until the current node is a MathML text integration point, an HTML+ integration point, or an element in the HTML namespace; then reprocesses the token as HTML.+ */+ boolean breakoutAsHtml(Token t, HtmlTreeBuilder tb) {+ tb.error(this);+ // never pop the bottom element: the html root, or the context element (the adjusted current node) in fragment parsing+ ArrayList<Element> stack = tb.getStack();+ while (stack.size() > 1) {+ Element el = tb.currentElement();+ if (el.tag().namespace().equals(Parser.NamespaceHtml)+ || HtmlTreeBuilder.isMathmlTextIntegration(el)+ || HtmlTreeBuilder.isHtmlIntegration(el))+ break;+ tb.pop();+ }+ return processAsHtml(t, tb);+ } }; private static void mergeAttributes(Token.StartTag source, Element dest) {src/test/java/org/jsoup/parser/HtmlParserTest.javaindex 86718fe2..154bd2fc 100644@@ -2241,6 +2241,91 @@ public class HtmlParserTest { assertEquals("<math>\n <mi>\n <p>One</p>\n <svg>\n <text>Blah</text>\n </svg>\n </mi><ms></ms>\n</math>", serialized); } + @Test void foreignBreakoutStartTagPopsSvg() {+ // https://html.spec.whatwg.org/multipage/parsing.html#parsing-main-inforeign+ Document doc = Jsoup.parse("<svg><s><p>One</p></s></svg>");+ assertEquals("<svg></svg>\n<s>\n <p>One</p>\n</s>", doc.body().html());+ assertHtmlNamespace(doc.expectFirst("s"));+ assertSame(doc.body(), doc.expectFirst("s").parent());+ }++ @Test void foreignBreakoutStartTagPopsNestedForeign() {+ Document doc = Jsoup.parse("<div><svg><g><b>Two</b></g></svg></div><math><mrow><div>Three</div></mrow></math>");+ Element b = doc.expectFirst("b");+ assertHtmlNamespace(b);+ assertEquals("div", b.parent().normalName());+ assertEquals(0, doc.expectFirst("g").childNodeSize());++ Element div = doc.expectFirst("math + div");+ assertHtmlNamespace(div);+ assertSame(doc.body(), div.parent());+ assertEquals("Three", div.text());+ assertEquals(0, doc.expectFirst("mrow").childNodeSize());+ }++ @Test void foreignBreakoutFontWithAttributesPops() {+ Document doc = Jsoup.parse("<svg><g><font color=red>Four</font></g></svg><svg><font>Five</font></svg>");+ Element font = doc.expectFirst("font[color]");+ assertHtmlNamespace(font);+ assertSame(doc.body(), font.parent());++ // a font without color/face/size is not a breakout and stays in svg+ Element svgFont = doc.expectFirst("font:not([color])");+ assertSvgNamespace(svgFont);+ assertEquals("svg", svgFont.parent().normalName());+ }++ @Test void foreignBreakoutEndTagsPop() {+ Document doc = Jsoup.parse("<svg><g></p></g></svg><math><mrow></br></mrow></math>");+ Element p = doc.expectFirst("p");+ assertHtmlNamespace(p);+ assertSame(doc.body(), p.parent());+ assertEquals(0, doc.expectFirst("g").childNodeSize());++ Element br = doc.expectFirst("br");+ assertHtmlNamespace(br);+ assertSame(doc.body(), br.parent());+ assertEquals(0, doc.expectFirst("mrow").childNodeSize());+ }++ @Test void foreignBreakoutStopsAtIntegrationPoints() {+ Document doc = Jsoup.parse("<svg><foreignObject><svg><g><b>One</b></g></svg></foreignObject></svg>"+ + "<math><mi><svg><p>Two</p></svg></mi></math>"+ + "<math><annotation-xml encoding=text/html><svg><i>Three</i></svg></annotation-xml></math>");+ Element b = doc.expectFirst("b");+ assertHtmlNamespace(b);+ assertEquals("foreignObject", b.parent().tagName());++ Element p = doc.expectFirst("p");+ assertHtmlNamespace(p);+ assertEquals("mi", p.parent().normalName());++ Element i = doc.expectFirst("i");+ assertHtmlNamespace(i);+ assertEquals("annotation-xml", i.parent().normalName());+ }++ @Test void foreignBreakoutStopsAtLowercaseIntegrationPoints() {+ // the tag name may not be in the canonical SVG case, but is still an integration point+ Document doc = Jsoup.parse("<svg><foreignobject><p>One</p></foreignobject><DESC><b>Two</b></DESC></svg>");+ assertEquals("<svg><foreignobject><p>One</p></foreignobject><DESC><b>Two</b></DESC></svg>",+ TextUtil.stripNewlines(doc.body().html()));+ assertHtmlNamespace(doc.expectFirst("p"));+ assertHtmlNamespace(doc.expectFirst("b"));+ }++ @Test void foreignBreakoutInSvgFragmentContext() {+ Element context = Jsoup.parse("<svg></svg>").expectFirst("svg");+ List<Node> nodes = Parser.parseFragment("<g><b>One</b></g>", context, "");+ assertEquals(2, nodes.size());+ Element g = (Element) nodes.get(0);+ assertEquals("g", g.normalName());+ assertEquals(0, g.childNodeSize());+ Element b = (Element) nodes.get(1);+ assertEquals("b", b.normalName());+ assertHtmlNamespace(b);+ }+ private static void assertHtmlNamespace(Element el) { assertEquals(Parser.NamespaceHtml, el.tag().namespace()); }@@ -2263,8 +2348,11 @@ public class HtmlParserTest { Element svgStyle = doc.expectFirst("svg style"); assertMathNamespace(svgStyle); // in inherited math namespace as not an HTML integration point- Element styleImg = svgStyle.expectFirst("img");- assertHtmlNamespace(styleImg); // this one is an img tag - in foreign to html elements+ // this one is an img tag - in foreign to html elements, so breaks out: the foreign style, svg, math are popped first+ assertEquals(0, svgStyle.childNodeSize());+ Element styleImg = doc.expectFirst("math + img");+ assertHtmlNamespace(styleImg);+ assertSame(doc.body(), styleImg.parent()); assertMathNamespace(doc.expectFirst("svg")); assertMathNamespace(doc.expectFirst("math"));
CHANGES.mdindex dfe7b1f9..bd2cf35c 100644@@ -21,6 +21,7 @@ * Corrected multipart form encoding to percent-escape CR and LF in field names and filenames, matching the HTML form submission specification. Multipart file content-types containing CR or LF are now rejected with a `ValidationException`. [#2555](https://github.com/jhy/jsoup/pull/2555) * Aligned trailing comment placement with the HTML specification: comments after `</body>` remain children of the `html` element, while comments after `</html>` remain children of the document. [#2557](https://github.com/jhy/jsoup/pull/2557) * When using the optional `re2j` regular expression engine, heap exhaustion caused by complex selector patterns during matching is now normalized to a `ValidationException` with a `Pattern complexity error` message.+* Fixed parsing of malformed SVG and MathML content so that breakout HTML tags are placed according to the HTML specification. [#2562](https://github.com/jhy/jsoup/issues/2562) ## 1.23.1 (2026-Jul-30)src/main/java/org/jsoup/parser/HtmlTreeBuilderState.javaindex 84b8504c..b6bbcfe7 100644@@ -1768,12 +1768,12 @@ enum HtmlTreeBuilderState { case StartTag: Token.StartTag start = t.asStartTag(); if (StringUtil.in(start.normalName, InForeignToHtml))- return processAsHtml(t, tb);+ return breakoutToHtml(t, tb); if (start.normalName.equals("font") && ( start.hasAttributeIgnoreCase("color") || start.hasAttributeIgnoreCase("face") || start.hasAttributeIgnoreCase("size")))- return processAsHtml(t, tb);+ return breakoutToHtml(t, tb); // Any other start: // (whatwg says to fix up tag name and attribute case per a table - we will preserve original case instead)@@ -1796,7 +1796,7 @@ enum HtmlTreeBuilderState { case EndTag: Token.EndTag end = t.asEndTag(); if (end.normalName.equals("br") || end.normalName.equals("p"))- return processAsHtml(t, tb);+ return breakoutToHtml(t, tb); if (end.normalName.equals("script") && tb.currentElementIs("script", Parser.NamespaceSvg)) { // script level and execution elided. tb.pop();@@ -1833,9 +1833,24 @@ enum HtmlTreeBuilderState { return true; } + /** Reprocesses a token as HTML, retaining the open-element stack. */ boolean processAsHtml(Token t, HtmlTreeBuilder tb) { return tb.state().process(t, tb); }++ /** Pops foreign elements, then reprocesses the breakout token as HTML. */+ boolean breakoutToHtml(Token t, HtmlTreeBuilder tb) {+ // https://html.spec.whatwg.org/multipage/parsing.html#parsing-main-inforeign+ while (!tb.getStack().isEmpty()) {+ Element current = tb.currentElement();+ if (Parser.NamespaceHtml.equals(current.tag().namespace())+ || HtmlTreeBuilder.isMathmlTextIntegration(current)+ || HtmlTreeBuilder.isHtmlIntegration(current))+ break;+ tb.pop();+ }+ return processAsHtml(t, tb);+ } }; private static void mergeAttributes(Token.StartTag source, Element dest) {src/test/java/org/jsoup/parser/HtmlParserTest.javaindex 86718fe2..f3cc2178 100644@@ -2225,6 +2225,24 @@ public class HtmlParserTest { assertEquals("One", div.expectFirst("p").text()); } + @Test void popsForeignElementsForHtmlBreakoutTags() {+ Document doc = Jsoup.parse("<svg><s><p>One</p></s></svg>");++ assertEquals("<svg></svg>\n<s>\n <p>One</p>\n</s>", doc.body().html());+ assertHtmlNamespace(doc.expectFirst("s"));++ Document endTag = Jsoup.parse("<svg><g></p><circle></circle>");+ assertEquals("<svg>\n <g></g>\n</svg>\n<p></p>\n<circle></circle>", endTag.body().html());+ assertHtmlNamespace(endTag.expectFirst("circle"));+ }++ @Test void doesNotPopForeignElementsForOtherEndTags() {+ Document doc = Jsoup.parse("<svg><g></div><circle></circle></g></svg>");++ Element circle = doc.expectFirst("svg > g > circle");+ assertSvgNamespace(circle);+ }+ @Test void mathParseText() { String html = "<div><math><mi><p>One</p><svg><text>Blah</text></svg></mi><ms></ms></div>"; Document doc = Jsoup.parse(html);@@ -2263,8 +2281,9 @@ public class HtmlParserTest { Element svgStyle = doc.expectFirst("svg style"); assertMathNamespace(svgStyle); // in inherited math namespace as not an HTML integration point- Element styleImg = svgStyle.expectFirst("img");- assertHtmlNamespace(styleImg); // this one is an img tag - in foreign to html elements+ assertEquals("", svgStyle.html());+ Element breakoutImg = doc.expectFirst("body > img");+ assertHtmlNamespace(breakoutImg); // img breaks out of foreign content assertMathNamespace(doc.expectFirst("svg")); assertMathNamespace(doc.expectFirst("math"));