Backtestsjsouprun-20261006-181938#2605

Issue #2605 Merged fix PR # Run 6 Oct 2026 · run-20261006-181938 jhy/jsoup

Parser reuse interrupts StreamParser

Better

Our patch was judged better than the merged fix.

In this judgment, Patch A = our agent, Patch B = the merged fix.

Both patches fix the root cause the same way. `StreamParser` now drives `parser.getTreeBuilder().newInstance()` instead of sharing the supplied parser's tree builder. The supplied `Parser` is still passed into parse initialisation, so configuration and error tracking stay on it. Both therefore resolve the two scenarios in the issue: reusing the parser mid-stream, and `Element.append(String)` on a streamed element.

Tests are comparable in quality:
- **Patch A** adds tests for parser reuse mid-stream, including `parseInput`, `parseFragmentInput` and `unescape`. It also tests append mid-stream, checking the remaining streamed elements via `stream().forEach` and the final document, and tests that errors land on the supplied parser.
- **Patch B** adds equivalent reuse and append tests, plus a settings/errors-sharing test. Both sets would fail before the fix and pass after.
- Both update existing tests that reached into `parser.getTreeBuilder()`. A preserves the adoption-recovery tests' use of `StreamParser` by giving the test subclass a proper `newInstance()` override. B rewrites that helper to drive the tree builder directly, bypassing `StreamParser`.

The difference is scope and risk. Patch B also makes changes the fix does not need:
- It flips `stopped` to `true` by default, which changes iterator semantics before `parse()`.
- It removes the `document = treeBuilder.doc` refresh from `document()`. That makes correctness depend on `parse()`/`parseFragment()` always assigning the field, which the diff does not show.
- It widens the `treeBuilder` field to package-private.

These extra behavioural edits add risk without being required by the issue. Patch A makes the minimal core change, adds a narrow package-private accessor, and documents the reasoning inline. Patch B's CHANGES entry is a minor plus but does not outweigh the added risk.

The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.

Solve

13m 25s

Wall-clock time in the container

Full test suite

No new failures

2,290 passed · 0 skipped · 2,290 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

100vs 99 merged

Changed lines, ours vs merged

Reusing a Parser supplied to a StreamParser overwrites the active streaming parse state.

And calling Element.append(String) on a streamed element has the same effect, because it uses that parser to parse the added HTML.

The fix will just copy the treebuilder, while preserving configuration and error diagnostics on the supplied parser.

The two patches

Our patch+84 −16
src/main/java/org/jsoup/parser/StreamParser.javaindex 607c1ad1..afe15640 100644@@ -68,7 +68,10 @@ public class StreamParser implements Closeable {      */     public StreamParser(Parser parser) {         this.parser = parser;-        treeBuilder = parser.getTreeBuilder();+        // use an independent tree builder, so that reusing the parser (or appending HTML to a streamed element, which+        // parses via the document's parser) doesn't clobber the streaming parse state. The supplied parser is still+        // used for configuration and error tracking.+        treeBuilder = parser.getTreeBuilder().newInstance();         treeBuilder.nodeListener(it);     } @@ -185,6 +188,11 @@ public class StreamParser implements Closeable {         return document;     } +    /** The tree builder driving this stream; independent of the supplied Parser's own tree builder. */+    TreeBuilder treeBuilder() {+        return treeBuilder;+    }+     /**      Runs the parser until the input is fully read, and returns the completed Document.      @return the completed Documentsrc/test/java/org/jsoup/parser/HtmlTreeBuilderTest.javaindex c61eddca..5f9f8e4d 100644@@ -62,15 +62,7 @@ public class HtmlTreeBuilderTest {     }      @Test void missingFormattingElementDuringTraversalReportsError() throws IOException {-        HtmlTreeBuilder tb = new HtmlTreeBuilder() {-            @Override boolean removeFromStack(Element el) {-                boolean removed = super.removeFromStack(el);-                if (el.normalName().equals("span"))-                    stack.clear(); // simulate losing the remaining stack during the inner loop-                return removed;-            }-        };-        assertAdoptionRecovery(tb, builder -> {+        assertAdoptionRecovery(new StackLosingTreeBuilder(), builder -> {             Element root = builder.stack.get(0);             Element formatting = builder.doc.child(0).appendElement("b");             Element span = formatting.appendElement("span");@@ -127,11 +119,25 @@ public class HtmlTreeBuilderTest {         }, "Unexpected EndTag token [</b>] when in state [InBody]");     } +    /** Simulates losing the remaining stack during the adoption agency inner loop. */+    private static class StackLosingTreeBuilder extends HtmlTreeBuilder {+        @Override boolean removeFromStack(Element el) {+            boolean removed = super.removeFromStack(el);+            if (el.normalName().equals("span"))+                stack.clear();+            return removed;+        }++        @Override StackLosingTreeBuilder newInstance() {+            return new StackLosingTreeBuilder(); // the StreamParser drives its own copy+        }+    }+     // exercise an adoption error, then verify that the parser can consume the remaining input     private static void assertAdoptionRecovery(HtmlTreeBuilder tb, Consumer<HtmlTreeBuilder> exercise, String message) throws IOException {         Parser parser = new Parser(tb).setTrackErrors(20);         try (StreamParser stream = new StreamParser(parser).parseFragment("<p>After</p>", new Element("div"), "")) {-            exercise.accept(tb);+            exercise.accept((HtmlTreeBuilder) stream.treeBuilder()); // the builder driving the stream             assertTrue(parser.getErrors().stream().anyMatch(error -> error.getErrorMessage().equals(message)),                 () -> "Expected error: " + message + "; got: " + parser.getErrors());             assertEquals("After", stream.complete().text());@@ -185,7 +191,7 @@ public class HtmlTreeBuilderTest {         // a table removed while still open uses the element above it on the stack         Parser parser = Parser.htmlParser();         try (StreamParser stream = new StreamParser(parser).parseFragment("", new Element("div"), "")) {-            HtmlTreeBuilder tb = (HtmlTreeBuilder) parser.getTreeBuilder();+            HtmlTreeBuilder tb = (HtmlTreeBuilder) stream.treeBuilder();             Element container = stream.document().child(0);             Element table = container.appendElement("table");             tb.push(table);@@ -239,10 +245,11 @@ public class HtmlTreeBuilderTest {      @Test void tracksParseLifecycle() throws IOException {         Parser parser = Parser.htmlParser();-        TreeBuilder treeBuilder = parser.getTreeBuilder();+        StreamParser streamer = new StreamParser(parser);+        TreeBuilder treeBuilder = streamer.treeBuilder();         assertFalse(treeBuilder.isComplete()); -        try (StreamParser streamParser = new StreamParser(parser).parse("<title>One</title><p id=hit>Full</p>", "")) {+        try (StreamParser streamParser = streamer.parse("<title>One</title><p id=hit>Full</p>", "")) {             streamParser.expectFirst("title");             Element open = streamParser.document().expectFirst("#hit");             assertTrue(treeBuilder.isOpen(open));src/test/java/org/jsoup/parser/StreamParserTest.javaindex 2f24613d..fb33f52e 100644@@ -531,7 +531,7 @@ class StreamParserTest {     }       private static CharacterReader getReader(StreamParser streamer) {-        return streamer.document().parser().getTreeBuilder().reader;+        return streamer.treeBuilder().reader;     }      @Test void doesNotReadPastParse() throws IOException {@@ -694,7 +694,7 @@ class StreamParserTest {             .parseFragment("<div>One</div><div>Two</div>", context, "")) {             Element container = parser.expectFirst("#context");             assertEquals("One Two", container.text());-            assertFalse(parser.document().parser().getTreeBuilder().isOpen(container));+            assertFalse(parser.treeBuilder().isOpen(container));             assertSame(parser.document().child(0), container);             assertEquals(container.childNodes(), parser.completeFragment());         }@@ -819,4 +819,57 @@ class StreamParserTest {         }     } +    @Test void reusingSuppliedParserDoesNotInterruptStream() throws IOException {+        Parser base = Parser.htmlParser();+        try (StreamParser streamer = new StreamParser(base).parse("<div id=1>One</div><div id=2>Two</div><div id=3>Three</div>", "")) {+            Element first = streamer.selectFirst("div");+            assertNotNull(first);+            assertEquals("1", first.id());++            // reuse the supplied parser for an unrelated parse, mid-stream+            Document other = base.parseInput("<p>Other</p>", "");+            assertEquals("Other", other.selectFirst("p").text());+            List<Node> frag = base.parseFragmentInput("<span>Frag</span>", new Element("div"), "");+            assertEquals("<span>Frag</span>", frag.get(0).outerHtml());+            assertEquals("a&b", base.unescape("a&amp;b", false));++            Element second = streamer.selectNext("div");+            assertNotNull(second);+            assertEquals("2", second.id());+            Element third = streamer.selectNext("div");+            assertNotNull(third);+            assertEquals("Three", third.text());+            assertNull(streamer.selectNext("p"));++            Document doc = streamer.complete();+            assertEquals("<div id=\"1\">One</div><div id=\"2\">Two</div><div id=\"3\">Three</div>", doc.body().html().replace("\n", ""));+            assertSame(base, doc.parser());+        }+    }++    @Test void appendingHtmlToStreamedElementDoesNotInterruptStream() throws IOException {+        try (StreamParser streamer = new StreamParser(Parser.htmlParser()).parse("<div id=1>One</div><div id=2>Two</div><div id=3>Three</div>", "")) {+            Element first = streamer.selectFirst("div");+            assertNotNull(first);+            first.append("<p>Added</p>"); // parses via the document's parser++            StringBuilder seen = new StringBuilder();+            streamer.stream().forEach(el -> trackSeen(el, seen));+            assertEquals("div#2[Two]+;div#3[Three];body;html;#root;", seen.toString());++            Document doc = streamer.document();+            assertEquals(3, doc.select("div").size());+            assertEquals("Added", doc.expectFirst("div#1 > p").text());+            assertEquals("Three", doc.expectFirst("div#3").text());+        }+    }++    @Test void streamErrorsTrackedOnSuppliedParser() throws IOException {+        Parser base = Parser.htmlParser().setTrackErrors(10);+        try (StreamParser streamer = new StreamParser(base).parse("<div>One</div></span><p>Two", "")) {+            streamer.complete();+            assertFalse(base.getErrors().isEmpty());+            assertTrue(base.getErrors().get(0).getErrorMessage().contains("span"));+        }+    } } 
Merged fix · PR #+85 −14
CHANGES.mdindex 0b7c25e0..fe616b80 100644@@ -11,6 +11,7 @@ * Named character references without a semicolon before `-` or `_` now decode correctly in attribute values, matching HTML and browser behavior (e.g., `&copy-` becomes `©-`). [#2588](https://github.com/jhy/jsoup/issues/2588) * Support downloads > 2GB via `Response.bodyStream()` when `maxBodySize(0)` is configured. [#2593](https://github.com/jhy/jsoup/issues/2593) * Updated table and table-fragment parsing to place misnested content correctly per the HTML5 spec. For example, `<table><b><p>X</b>` places the paragraph before the table. [#2601](https://github.com/jhy/jsoup/issues/2601)+* `StreamParser` now preserves its progress if you use the same `Parser` to parse another document, or call `Element.append(String)` on a streamed element. Previously, these operations replaced the active parse state and prevented streaming from continuing correctly. [#2605](https://github.com/jhy/jsoup/issues/2605) * Updated the adoption-agency algorithm to match the current HTML5 spec and to preserve formatting order when recovering misnested elements. [#2604](https://github.com/jhy/jsoup/pull/2604)  ## 1.23.2 (2026-Aug-26)src/main/java/org/jsoup/parser/StreamParser.javaindex 607c1ad1..3e52bebd 100644@@ -57,18 +57,19 @@ import java.util.stream.StreamSupport;  @since 1.18.1 */ public class StreamParser implements Closeable {     final private Parser parser;-    final private TreeBuilder treeBuilder;+    final TreeBuilder treeBuilder;     final private ElementIterator it = new ElementIterator();     @Nullable private Document document;-    private boolean stopped = false;+    private boolean stopped = true;      /**      Construct a new StreamParser, using the supplied base Parser.+     The supplied parser provides configuration and collects parse errors; streaming uses an independent tree builder.      @param parser the configured base parser      */     public StreamParser(Parser parser) {         this.parser = parser;-        treeBuilder = parser.getTreeBuilder();+        treeBuilder = parser.getTreeBuilder().newInstance();         treeBuilder.nodeListener(it);     } @@ -180,7 +181,6 @@ public class StreamParser implements Closeable {      @return the (partial) Document      */     public Document document() {-        document = treeBuilder.doc;         Validate.notNull(document, "Must run parse() before calling.");         return document;     }src/test/java/org/jsoup/parser/HtmlTreeBuilderTest.javaindex c61eddca..6e0f19ed 100644@@ -8,6 +8,7 @@ import org.jspecify.annotations.NullMarked; import org.junit.jupiter.api.Test;  import java.io.IOException;+import java.io.StringReader; import java.lang.annotation.Annotation; import java.util.ArrayList; import java.util.List;@@ -130,11 +131,16 @@ public class HtmlTreeBuilderTest {     // exercise an adoption error, then verify that the parser can consume the remaining input     private static void assertAdoptionRecovery(HtmlTreeBuilder tb, Consumer<HtmlTreeBuilder> exercise, String message) throws IOException {         Parser parser = new Parser(tb).setTrackErrors(20);-        try (StreamParser stream = new StreamParser(parser).parseFragment("<p>After</p>", new Element("div"), "")) {+        tb.initialiseParse(new StringReader("<p>After</p>"), "", parser);+        tb.initialiseParseFragment(new Element("div"));+        try {             exercise.accept(tb);             assertTrue(parser.getErrors().stream().anyMatch(error -> error.getErrorMessage().equals(message)),                 () -> "Expected error: " + message + "; got: " + parser.getErrors());-            assertEquals("After", stream.complete().text());+            tb.runParser();+            assertEquals("After", tb.doc.text());+        } finally {+            tb.closeParse();         }     } @@ -185,7 +191,7 @@ public class HtmlTreeBuilderTest {         // a table removed while still open uses the element above it on the stack         Parser parser = Parser.htmlParser();         try (StreamParser stream = new StreamParser(parser).parseFragment("", new Element("div"), "")) {-            HtmlTreeBuilder tb = (HtmlTreeBuilder) parser.getTreeBuilder();+            HtmlTreeBuilder tb = (HtmlTreeBuilder) stream.treeBuilder;             Element container = stream.document().child(0);             Element table = container.appendElement("table");             tb.push(table);@@ -239,10 +245,9 @@ public class HtmlTreeBuilderTest {      @Test void tracksParseLifecycle() throws IOException {         Parser parser = Parser.htmlParser();-        TreeBuilder treeBuilder = parser.getTreeBuilder();-        assertFalse(treeBuilder.isComplete());-         try (StreamParser streamParser = new StreamParser(parser).parse("<title>One</title><p id=hit>Full</p>", "")) {+            TreeBuilder treeBuilder = streamParser.treeBuilder;+            assertFalse(treeBuilder.isComplete());             streamParser.expectFirst("title");             Element open = streamParser.document().expectFirst("#hit");             assertTrue(treeBuilder.isOpen(open));src/test/java/org/jsoup/parser/StreamParserTest.javaindex 2f24613d..a6f288a1 100644@@ -96,6 +96,72 @@ class StreamParserTest {         assertEquals("", seen3.toString());     } +    @Test void streamsAfterParserReuse() throws IOException {+        Parser parser = Parser.htmlParser();+        try (StreamParser streamer = new StreamParser(parser).parse("<p>One</p><p>Two</p>", "")) {+            Document doc = streamer.document();+            assertSame(parser, doc.parser());+            assertEquals("One", streamer.expectFirst("p").text());++            Document ordinary = parser.parseInput("<div>Other</div>", "");+            assertEquals("Other", ordinary.body().text());++            assertSame(doc, streamer.document());+            assertEquals("Two", streamer.selectNext("p").text());+            assertEquals("<p>One</p>\n<p>Two</p>", streamer.complete().body().html());+        }+    }++    @Test void streamsAfterHtmlAppend() throws IOException {+        Parser parser = Parser.htmlParser();+        try (StreamParser streamer = new StreamParser(parser).parse("<p>One</p><p>Two</p>", "")) {+            Document doc = streamer.document();+            assertSame(parser, doc.parser());+            Element first = streamer.expectFirst("p");+            assertEquals("One", first.text());++            first.append("<b>Added</b>");+            assertEquals("One<b>Added</b>", first.html());++            assertSame(doc, streamer.document());+            assertEquals("Two", streamer.selectNext("p").text());+            assertEquals("<p>One<b>Added</b></p>\n<p>Two</p>", streamer.complete().body().html());+        }+    }++    @Test void sharesParserSettingsAndErrors() throws IOException {+        Parser parser = Parser.htmlParser();+        try (StreamParser streamer = new StreamParser(parser)) {+            parser.settings(ParseSettings.preserveCase).setTrackPosition(true).setTrackErrors(10);+            Document doc = streamer.parse("<P ID=one>One</P></span>", "").complete();++            assertSame(parser, doc.parser());+            Element p = doc.expectFirst("P");+            assertEquals("P", p.tagName());+            assertTrue(p.attributes().hasKey("ID"));+            assertTrue(p.sourceRange().isTracked());+            assertFalse(parser.getErrors().isEmpty());+        }+    }++    @Test void iteratorEmptyBeforeParse() {+        try (StreamParser streamer = new StreamParser(Parser.htmlParser())) {+            Iterator<Element> iterator = streamer.iterator();+            assertFalse(iterator.hasNext());+            assertThrows(NoSuchElementException.class, iterator::next);+        }+    }++    @Test void parserReuseDoesNotStartStream() {+        Parser parser = Parser.htmlParser();+        try (StreamParser streamer = new StreamParser(parser)) {+            parser.parseInput("<p>Other</p>", "");++            assertThrows(IllegalArgumentException.class, streamer::document);+            assertFalse(streamer.iterator().hasNext());+        }+    }+     @Test void canStopAndCompleteAndReuse() throws IOException {         StreamParser parser = new StreamParser(Parser.htmlParser());         String html1 = "<p id=one>One<p id=two>Two";@@ -526,12 +592,11 @@ class StreamParserTest {     }      static boolean isClosed(StreamParser streamer) {-        // a bit of a back door in!         return getReader(streamer) == null;     } -     private static CharacterReader getReader(StreamParser streamer) {-        return streamer.document().parser().getTreeBuilder().reader;+    private static CharacterReader getReader(StreamParser streamer) {+        return streamer.treeBuilder.reader;     }      @Test void doesNotReadPastParse() throws IOException {@@ -694,7 +759,7 @@ class StreamParserTest {             .parseFragment("<div>One</div><div>Two</div>", context, "")) {             Element container = parser.expectFirst("#context");             assertEquals("One Two", container.text());-            assertFalse(parser.document().parser().getTreeBuilder().isOpen(container));+            assertFalse(parser.treeBuilder.isOpen(container));             assertSame(parser.document().child(0), container);             assertEquals(container.childNodes(), parser.completeFragment());         }