Backtestsjsouprun-20261006-181938#2486
XML parser does not support DOCTYPE with <!ENTITY> declarations
Our patch was judged better than the merged fix.
In this judgment, Patch A = our agent, Patch B = the merged fix.
The issue is about `Jsoup.clean()`. `clean()` parses its input with the HTML body-fragment parser, not the XML parser. Patch A changes the doctype tokeniser states for every syntax and adds a `DoctypeInternalSubset` state. That state skips quoted strings, comments, PIs and markup declarations, then hands the trailing `]...>` to BogusDoctype. As a result, the `]>` and the entity declarations no longer leak as text in the HTML or the XML parser. Patch A's CleanerTest reproduces the issue's exact `Jsoup.clean` scenario, and HtmlParserTest checks `parseBodyFragment`. It also covers many doctype variants (`svg[`, `SYSTEM[`, `PUBLIC`, quoted `]>`), protects against a stray `[` swallowing the document, and handles EOF. Patch B gates every new transition on `t.syntax == xml`. The HTML tokeniser path used by `Jsoup.clean()` is therefore unchanged, and the reported `]>` artifact would most likely still appear. B's own test shows that HTML output simply drops the subset after the doctype; it never exercises `clean()`. B adds a reasonable XML round-trip feature (keeping the subset in `DocumentType`), but that does not address the reported scenario. B also lacks A's guard: an unterminated subset consumes the whole rest of the document, a behaviour B's test asserts as intended. A's change to HTML tokenisation departs slightly from the spec's bogus-doctype handling, but it is well contained and tested. Patch A fixes the actual bug; Patch B likely does not.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
21m 12s
Wall-clock time in the container
Full test suite
No new failures
1,964 passed · 3 skipped · 1,967 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
181vs 198 merged
Changed lines, ours vs merged
The issue
View on GitHubWhen Jsoup.clean() processes XML/SVG input that contains a DOCTYPE declaration with an internal DTD subset (i.e., entity declarations enclosed in [...]), jsoup correctly removes the DOCTYPE tag itself but leaves the closing ]> of the subset in the output string.
This was discovered while sanitizing SVG files that follow the Adobe Illustrator export format.
jsoup version: 1.22.1
Java version: 25
Steps to Reproduce
Call Jsoup.clean() with the following input SVG that contains a DOCTYPE with multiple <!ENTITY> declarations inside an internal DTD subset:
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.safety.Safelist;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
class SvgSanitizerTest {
Safelist safelist = Safelist.none()
.addTags("svg", "circle")
.addAttributes("svg", "viewBox")
.addAttributes("circle", "cx", "cy", "r", "fill");
@Test
void shouldRemoveDocTypeTag() {
String dirtySvg = """
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd" [
<!ENTITY ns_extend "http://ns.adobe.com/Extensibility/1.0/">
<!ENTITY ns_ai "http://ns.adobe.com/AdobeIllustrator/10.0/">
<!ENTITY ns_graphs "http://ns.adobe.com/Graphs/1.0/">
<!ENTITY ns_vars "http://ns.adobe.com/Variables/1.0/">
<!ENTITY ns_imrep "http://ns.adobe.com/ImageReplacement/1.0/">
<!ENTITY ns_sfw "http://ns.adobe.com/SaveForWeb/1.0/">
<!ENTITY ns_custom "http://ns.adobe.com/GenericCustomNamespace/1.0/">
<!ENTITY ns_adobe_xpath "http://ns.adobe.com/XPath/1.0/">
]>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100">
<circle cx="50" cy="50" r="40" fill="red"/>
</svg>
""";
String expectedSvg = """
<svg viewBox="0 0 100 100">
<circle cx="50" cy="50" r="40" fill="red" />
</svg>
""";
Document.OutputSettings outputSettings = new Document.OutputSettings()
.syntax(Document.OutputSettings.Syntax.xml)
.prettyPrint(false);
var cleanedSvg = Jsoup.clean(dirtySvg, "", safelist, outputSettings);
assertThat(cleanedSvg).isEqualToNormalizingWhitespace(expectedSvg);
}
}
Expected Behaviour
The entire DOCTYPE block (including its internal subset [...]>) is removed. Output:
<svg viewBox="0 0 100 100">
<circle cx="50" cy="50" r="40" fill="red" />
</svg>
Actual Behaviour
The DOCTYPE declaration is removed but the closing ]> of the internal DTD subset is left behind in the output:
]>
<svg viewBox="0 0 100 100">
<circle cx="50" cy="50" r="40" fill="red" />
</svg>
Impact
Any application using Jsoup.clean() to sanitize SVG (or XML) produced by tools like Adobe Illustrator will silently receive malformed output containing the ]> artifact, which can break downstream XML parsers and SVG renderers.
Could you please have a look into this please ?
Kindly let me know if you need any more details about the issue.
Many thanks in advance!!
BR
Ram
The two patches
CHANGES.mdindex 5192070b..c23b70d0 100644@@ -14,6 +14,7 @@ * `Cleaner` no longer duplicates enforced attributes when the input `Document` preserves attribute case. A case-variant source attribute is now replaced by the enforced attribute in the cleaned output. [#2476](https://github.com/jhy/jsoup/issues/2476) * If a per-request SOCKS proxy is configured, jsoup now avoids using the JDK `HttpClient`, because the JDK would silently ignore that proxy and attempt to connect directly. Those requests now fall back to the legacy `HttpURLConnection` transport instead, which does support SOCKS. [#2468](https://github.com/jhy/jsoup/issues/2468) * `Connection.Response.streamParser()` and `DataUtil.streamParser(Path, ...)` could fail on small inputs without a declared charset, if the initial 5 KB charset sniff fully consumed the input and closed it before the stream parse began. [#2483](https://github.com/jhy/jsoup/issues/2483)+* A DOCTYPE with an XML internal DTD subset (e.g. `<!DOCTYPE svg [ <!ENTITY ...> ]>`, as exported by Adobe Illustrator) is now skipped as a whole. Previously the doctype ended at the first `>` within the subset, and the remaining declarations and the closing `]>` leaked into the document as text (including in `Jsoup.clean()` output). ## 1.22.1 (2026-Jan-01) src/main/java/org/jsoup/parser/TokeniserState.javaindex 1c50b078..faa61cc5 100644@@ -1215,6 +1215,11 @@ enum TokeniserState { case ' ': t.transition(AfterDoctypeName); break;+ case '[': // internal subset directly after the name, e.g. <!DOCTYPE svg[ ... ]>+ t.error(this);+ t.doctypePending.forceQuirks = true;+ t.transition(DoctypeInternalSubset);+ break; case nullChar: t.error(this); t.doctypePending.name.append(replacementChar);@@ -1253,7 +1258,7 @@ enum TokeniserState { } else { t.error(this); t.doctypePending.forceQuirks = true;- t.advanceTransition(BogusDoctype);+ t.advanceTransition(r.matches('[') ? DoctypeInternalSubset : BogusDoctype); } }@@ -1294,7 +1299,7 @@ enum TokeniserState { default: t.error(this); t.doctypePending.forceQuirks = true;- t.transition(BogusDoctype);+ t.transition(c == '[' ? DoctypeInternalSubset : BogusDoctype); } } },@@ -1331,7 +1336,7 @@ enum TokeniserState { default: t.error(this); t.doctypePending.forceQuirks = true;- t.transition(BogusDoctype);+ t.transition(c == '[' ? DoctypeInternalSubset : BogusDoctype); } } },@@ -1425,7 +1430,7 @@ enum TokeniserState { default: t.error(this); t.doctypePending.forceQuirks = true;- t.transition(BogusDoctype);+ t.transition(c == '[' ? DoctypeInternalSubset : BogusDoctype); } } },@@ -1462,7 +1467,7 @@ enum TokeniserState { default: t.error(this); t.doctypePending.forceQuirks = true;- t.transition(BogusDoctype);+ t.transition(c == '[' ? DoctypeInternalSubset : BogusDoctype); } } },@@ -1502,7 +1507,7 @@ enum TokeniserState { default: t.error(this); t.doctypePending.forceQuirks = true;- t.emitDoctypePending();+ t.transition(c == '[' ? DoctypeInternalSubset : BogusDoctype); } } },@@ -1539,7 +1544,7 @@ enum TokeniserState { default: t.error(this); t.doctypePending.forceQuirks = true;- t.transition(BogusDoctype);+ t.transition(c == '[' ? DoctypeInternalSubset : BogusDoctype); } } },@@ -1621,7 +1626,7 @@ enum TokeniserState { break; default: t.error(this);- t.transition(BogusDoctype);+ t.transition(c == '[' ? DoctypeInternalSubset : BogusDoctype); // NOT force quirks } }@@ -1638,12 +1643,84 @@ enum TokeniserState { t.emitDoctypePending(); t.transition(Data); break;+ case '[':+ t.transition(DoctypeInternalSubset);+ break; default: // ignore char break; } } },+ /**+ Skips over an XML internal DTD subset, e.g. {@code <!DOCTYPE svg [ <!ENTITY a "b"> ]>}, which is entered after+ the {@code [}. Markup declarations, comments, PIs, and quoted strings within the subset may contain {@code >} or+ {@code ]}, so are skipped as units. The subset contents are not retained. On the closing {@code ]}, the remainder+ of the doctype (up to {@code >}) is handled by BogusDoctype. A {@code >} outside a declaration (i.e. an unclosed+ {@code [}) or a tag ends the doctype, so a stray {@code [} can't consume the rest of the document.+ */+ DoctypeInternalSubset {+ @Override void read(Tokeniser t, CharacterReader r) {+ char c = r.consume();+ switch (c) {+ case ']':+ t.transition(BogusDoctype);+ break;+ case '>':+ t.error(this);+ t.emitDoctypePending();+ t.transition(Data);+ break;+ case eof:+ t.eofError(this);+ t.doctypePending.forceQuirks = true;+ t.emitDoctypePending();+ t.transition(Data);+ break;+ case '"':+ case '\'':+ skipQuoted(r, c);+ break;+ case '<':+ if (r.matchConsume("!--"))+ skipPast(r, "-->");+ else if (r.matchConsume("?"))+ skipPast(r, "?>");+ else if (r.matches('!'))+ skipDeclaration(r);+ else { // not a declaration (e.g. a tag following a stray [): end the doctype, and reconsume as data+ t.error(this);+ r.unconsume();+ t.emitDoctypePending();+ t.transition(Data);+ }+ break;+ default:+ // ignore char (whitespace, parameter entity references)+ break;+ }+ }++ /** Skips a quoted string, after the opening quote. */+ private void skipQuoted(CharacterReader r, char quote) {+ while (!r.isEmpty() && r.consume() != quote) {}+ }++ /** Skips through the end sequence, or to EOF. */+ private void skipPast(CharacterReader r, String end) {+ while (!r.isEmpty() && !r.matchConsume(end))+ r.consume();+ }++ /** Skips a markup declaration like {@code <!ENTITY name "value">}, after the {@code <}. */+ private void skipDeclaration(CharacterReader r) {+ while (!r.isEmpty()) {+ char c = r.consume();+ if (c == '>') return;+ if (c == '"' || c == '\'') skipQuoted(r, c);+ }+ }+ }, CdataSection { @Override void read(Tokeniser t, CharacterReader r) { String data = r.consumeTo("]]>");src/test/java/org/jsoup/parser/HtmlParserTest.javaindex a7336676..885860a4 100644@@ -1047,6 +1047,24 @@ public class HtmlParserTest { assertEquals("<img>\n<svg>\n <image />\n</svg>", doc.body().html()); } + @Test public void handlesDoctypeWithInternalSubset() {+ String html = "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\" \"svg11.dtd\" [\n<!ENTITY a \"b\">\n<!ENTITY c \"d\">\n]>\n<p>One</p>";+ Document doc = Jsoup.parse(html);+ assertNotNull(doc.documentType());+ assertEquals("svg", doc.documentType().name());+ assertEquals("<p>One</p>", doc.body().html());++ Document frag = Jsoup.parseBodyFragment(html);+ assertEquals("<p>One</p>", frag.body().html());+ }++ @Test public void strayDoctypeSubsetBracketDoesNotConsumeTags() {+ // a [ without a closing ] in the doctype ends at the next tag, rather than skipping it as a declaration+ Document doc = Jsoup.parse("<!DOCTYPE html [<html><body><p>Hello</p>");+ assertNotNull(doc.documentType());+ assertEquals("<p>Hello</p>", doc.body().html());+ }+ @Test public void handlesInvalidDoctypes() { // would previously throw invalid name exception on empty doctype Document doc = Jsoup.parse("<!DOCTYPE>");src/test/java/org/jsoup/parser/XmlTreeBuilderTest.javaindex ff709918..b576ec5a 100644@@ -684,4 +684,53 @@ public class XmlTreeBuilderTest { } return d; }++ @Test void doctypeWithInternalSubset() {+ // the internal subset is skipped, and does not leak "]>" (or anything else) as text+ String xml = "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\" [\n" ++ " <!ENTITY ns_a \"http://ns.adobe.com/a/1.0/\">\n" ++ " <!ENTITY ns_b 'has ] and > inside'>\n" ++ " <!-- a comment with ]> in it -->\n" ++ " <?pi with ]> ?>\n" ++ " <!ELEMENT svg ANY>\n" ++ " %pe;\n" ++ "]><svg><circle/></svg>";+ Document doc = Jsoup.parse(xml, "", Parser.xmlParser());+ DocumentType doctype = doc.documentType();+ assertNotNull(doctype);+ assertEquals("svg", doctype.name());+ assertEquals("-//W3C//DTD SVG 1.1//EN", doctype.publicId());+ assertEquals("http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd", doctype.systemId());+ assertEquals("<svg><circle /></svg>", doc.selectFirst("svg").outerHtml());+ assertEquals(2, doc.childNodeSize()); // doctype, svg; no stray text+ }++ @Test void doctypeWithInternalSubsetVariants() {+ String[] xmls = {+ "<!DOCTYPE svg [<!ENTITY a \"b\">]><svg/>",+ "<!DOCTYPE svg[<!ENTITY a \"b\">]><svg/>",+ "<!DOCTYPE svg SYSTEM \"s.dtd\" [<!ENTITY a \"b\">]><svg/>",+ "<!DOCTYPE svg SYSTEM \"s.dtd\"[<!ENTITY a \"b\">] ><svg/>",+ "<!DOCTYPE svg PUBLIC \"p\" [<!ENTITY a \"b\">]><svg/>",+ "<!DOCTYPE svg SYSTEM[<!ENTITY a \"b\">]><svg/>",+ };+ for (String xml : xmls) {+ Document doc = Jsoup.parse(xml, "", Parser.xmlParser());+ assertNotNull(doc.documentType(), xml);+ assertEquals("svg", doc.documentType().name(), xml);+ assertEquals(2, doc.childNodeSize(), xml);+ assertEquals("<svg />", doc.child(0).outerHtml(), xml);+ }+ }++ @Test void unclosedInternalSubsetEndsAtGt() {+ // a stray [ without a closing ] should not consume the rest of the document+ Document doc = Jsoup.parse("<!DOCTYPE svg [ foo><svg/>", "", Parser.xmlParser());+ assertNotNull(doc.documentType());+ assertEquals("<svg />", doc.child(0).outerHtml());++ Document eof = Jsoup.parse("<!DOCTYPE svg [<!ENTITY a \"b", "", Parser.xmlParser());+ assertNotNull(eof.documentType());+ assertEquals(1, eof.childNodeSize());+ } }src/test/java/org/jsoup/safety/CleanerTest.javaindex cb3bbabf..2c4b0703 100644@@ -637,4 +637,24 @@ public class CleanerTest { String clean2 = cleaner.clean(doc2).body().html(); assertEquals("content is <data>", clean2); }++ @Test void cleanRemovesDoctypeWithInternalSubset() {+ // the ]> closing an internal DTD subset was leaking into the output+ String svg = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n" ++ "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\" [\n" ++ "\t<!ENTITY ns_extend \"http://ns.adobe.com/Extensibility/1.0/\">\n" ++ "\t<!ENTITY ns_ai \"http://ns.adobe.com/AdobeIllustrator/10.0/\">\n" ++ "]>\n" ++ "<svg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 100 100\"><circle cx=\"50\" cy=\"50\" r=\"40\" fill=\"red\"/></svg>";+ Safelist safelist = Safelist.none()+ .addTags("svg", "circle")+ .addAttributes("svg", "viewBox")+ .addAttributes("circle", "cx", "cy", "r", "fill");+ Document.OutputSettings out = new Document.OutputSettings()+ .syntax(Document.OutputSettings.Syntax.xml)+ .prettyPrint(false);++ String clean = Jsoup.clean(svg, "", safelist, out);+ assertEquals("<svg viewBox=\"0 0 100 100\"><circle cx=\"50\" cy=\"50\" r=\"40\" fill=\"red\" /></svg>", clean.trim());+ } }
CHANGES.mdindex 5192070b..55fc9b42 100644@@ -14,6 +14,7 @@ * `Cleaner` no longer duplicates enforced attributes when the input `Document` preserves attribute case. A case-variant source attribute is now replaced by the enforced attribute in the cleaned output. [#2476](https://github.com/jhy/jsoup/issues/2476) * If a per-request SOCKS proxy is configured, jsoup now avoids using the JDK `HttpClient`, because the JDK would silently ignore that proxy and attempt to connect directly. Those requests now fall back to the legacy `HttpURLConnection` transport instead, which does support SOCKS. [#2468](https://github.com/jhy/jsoup/issues/2468) * `Connection.Response.streamParser()` and `DataUtil.streamParser(Path, ...)` could fail on small inputs without a declared charset, if the initial 5 KB charset sniff fully consumed the input and closed it before the stream parse began. [#2483](https://github.com/jhy/jsoup/issues/2483)+* In XML mode, doctypes with an internal subset (the `[...]` section inside `<!DOCTYPE ...>` used for entity declarations) now round-trip correctly. The subset is preserved as raw text only; entities are not expanded and external DTDs are not loaded. [#2486](https://github.com/jhy/jsoup/issues/2486) ## 1.22.1 (2026-Jan-01) src/main/java/org/jsoup/nodes/DocumentType.javaindex 279a91a6..19148049 100644@@ -18,6 +18,7 @@ public class DocumentType extends LeafNode { private static final String PubSysKey = "pubSysKey"; // PUBLIC or SYSTEM private static final String PublicId = "publicId"; private static final String SystemId = "systemId";+ private static final String InternalSubsetKey = Attributes.internalKey("doctypeInternalSubset"); /** * Create a new doctype element.@@ -41,6 +42,14 @@ public class DocumentType extends LeafNode { attr(PubSysKey, value); } + /**+ Sets the raw XML internal subset for serialization.+ @param value the internal subset contents+ */+ public void setInternalSubset(String value) {+ attributes().put(InternalSubsetKey, value);+ }+ private void updatePubSyskey() { if (has(PublicId)) { attributes().add(PubSysKey, PUBLIC_KEY);@@ -93,6 +102,8 @@ public class DocumentType extends LeafNode { accum.append(" \"").append(attr(PublicId)).append('"'); if (has(SystemId)) accum.append(" \"").append(attr(SystemId)).append('"');+ if (attributes().hasKey(InternalSubsetKey)) // only if via the xml parser; html parser will drop+ accum.append(" [").append(attr(InternalSubsetKey)).append(']'); accum.append('>'); } src/main/java/org/jsoup/parser/Token.javaindex 6d6882ea..d810fb7b 100644@@ -53,6 +53,8 @@ abstract class Token { @Nullable String pubSysKey = null; final TokenData publicIdentifier = new TokenData(); final TokenData systemIdentifier = new TokenData();+ final TokenData internalSubset = new TokenData();+ boolean sawInternalSubset = false; boolean forceQuirks = false; Doctype() {@@ -66,6 +68,8 @@ abstract class Token { pubSysKey = null; publicIdentifier.reset(); systemIdentifier.reset();+ internalSubset.reset();+ sawInternalSubset = false; forceQuirks = false; return this; }@@ -86,6 +90,14 @@ abstract class Token { return systemIdentifier.value(); } + String getInternalSubset() {+ return internalSubset.value();+ }++ boolean hasInternalSubset() {+ return sawInternalSubset;+ }+ public boolean isForceQuirks() { return forceQuirks; }src/main/java/org/jsoup/parser/TokeniserState.javaindex 1c50b078..e61b8c27 100644@@ -1,5 +1,6 @@ package org.jsoup.parser; +import org.jsoup.internal.StringUtil; import org.jsoup.nodes.DocumentType; import static org.jsoup.nodes.Document.OutputSettings.Syntax.xml;@@ -1244,6 +1245,10 @@ enum TokeniserState { else if (r.matches('>')) { t.emitDoctypePending(); t.advanceTransition(Data);+ } else if (t.syntax == xml && r.matches('[')) {+ // special case when in XML parse mode so we can support entity round trop+ t.doctypePending.sawInternalSubset = true;+ t.advanceTransition(DoctypeInternalSubset); } else if (r.matchConsumeIgnoreCase(DocumentType.PUBLIC_KEY)) { t.doctypePending.pubSysKey = DocumentType.PUBLIC_KEY; t.transition(AfterDoctypePublicKeyword);@@ -1443,6 +1448,16 @@ enum TokeniserState { t.emitDoctypePending(); t.transition(Data); break;+ case '[':+ if (t.syntax == xml) {+ t.doctypePending.sawInternalSubset = true;+ t.transition(DoctypeInternalSubset);+ break;+ }+ t.error(this);+ t.doctypePending.forceQuirks = true;+ t.transition(BogusDoctype);+ break; case '"': t.error(this); // system id empty@@ -1613,6 +1628,16 @@ enum TokeniserState { t.emitDoctypePending(); t.transition(Data); break;+ case '[':+ if (t.syntax == xml) {+ t.doctypePending.sawInternalSubset = true;+ t.transition(DoctypeInternalSubset);+ break;+ }+ t.error(this);+ t.transition(BogusDoctype);+ // NOT force quirks+ break; case eof: t.eofError(this); t.doctypePending.forceQuirks = true;@@ -1644,6 +1669,11 @@ enum TokeniserState { } } },+ DoctypeInternalSubset {+ @Override void read(Tokeniser t, CharacterReader r) {+ readDoctypeInternalSubset(t, r, this);+ }+ }, CdataSection { @Override void read(Tokeniser t, CharacterReader r) { String data = r.consumeTo("]]>");@@ -1779,4 +1809,84 @@ enum TokeniserState { t.transition(fallback); } }++ /**+ Reads an XML doctype internal subset as opaque text so it can be re-emitted without parsing declarations.+ Only used when in XML mode; HTML spec will drop these as Bogus.+ */+ private static void readDoctypeInternalSubset(Tokeniser t, CharacterReader r, TokeniserState current) {+ final byte None = 0, SingleQuote = 1, DoubleQuote = 2, Comment = 3, ProcessingInstruction = 4;+ byte context = None;+ TokenData subset = t.doctypePending.internalSubset;++ while (true) {+ char c = r.consume();+ switch (c) {+ case '\'':+ subset.append(c);+ if (context == None) context = SingleQuote;+ else if (context == SingleQuote) context = None;+ break;+ case '"':+ subset.append(c);+ if (context == None) context = DoubleQuote;+ else if (context == DoubleQuote) context = None;+ break;+ case '<':+ subset.append(c);+ if (context == None) {+ if (r.matchConsume("!--")) {+ subset.append("!--");+ context = Comment;+ } else if (r.matchConsume("?")) {+ subset.append('?');+ context = ProcessingInstruction;+ }+ }+ break;+ case '-':+ subset.append(c);+ if (context == Comment && r.matchConsume("->")) {+ subset.append("->");+ context = None;+ }+ break;+ case '?':+ subset.append(c);+ if (context == ProcessingInstruction && r.matches('>')) {+ r.advance();+ subset.append('>');+ context = None;+ }+ break;+ case ']':+ if (context == None) {+ String ws = r.consumeMatching(StringUtil::isWhitespace);+ if (r.matches('>')) {+ r.advance();+ t.emitDoctypePending();+ t.transition(Data);+ return;+ }+ subset.append(c);+ subset.append(ws);+ break;+ }+ subset.append(c);+ break;+ case nullChar:+ t.error(current);+ subset.append(replacementChar);+ break;+ case eof:+ t.eofError(current);+ t.emitDoctypePending();+ t.transition(Data);+ return;+ default:+ subset.append(c);+ break;+ }+ }+ } }src/main/java/org/jsoup/parser/XmlTreeBuilder.javaindex 796f1695..7941e810 100644@@ -238,6 +238,8 @@ public class XmlTreeBuilder extends TreeBuilder { void insertDoctypeFor(Token.Doctype token) { DocumentType doctypeNode = new DocumentType(settings.normalizeTag(token.getName()), token.getPublicIdentifier(), token.getSystemIdentifier()); doctypeNode.setPubSysKey(token.getPubSysKey());+ if (token.hasInternalSubset())+ doctypeNode.setInternalSubset(token.getInternalSubset()); insertLeafNode(doctypeNode); } src/test/java/org/jsoup/nodes/DocumentTypeTest.javaindex f5e561bc..b4a5b8d4 100644@@ -61,6 +61,15 @@ public class DocumentTypeTest { String legacyDoc = "<!DOCTYPE html SYSTEM \"about:legacy-compat\">"; assertEquals(legacyDoc, htmlOutput(legacyDoc)); assertEquals(legacyDoc, xmlOutput(legacyDoc));++ // round trips internal subset in xml; dropped in html+ String internalSubset = "<!DOCTYPE svg SYSTEM \"example.dtd\" [<!ENTITY ns_extend \"http://ns.adobe.com/Extensibility/1.0/\">]>";+ assertEquals("<!DOCTYPE svg SYSTEM \"example.dtd\">", htmlOutput(internalSubset));+ assertEquals(internalSubset, xmlOutput(internalSubset));++ String emptySubset = "<!DOCTYPE root []>";+ assertEquals("<!doctype root>", htmlOutput(emptySubset));+ assertEquals(emptySubset, xmlOutput(emptySubset)); } private String htmlOutput(String in) {src/test/java/org/jsoup/parser/XmlTreeBuilderTest.javaindex ff709918..4d82c613 100644@@ -387,6 +387,59 @@ public class XmlTreeBuilderTest { assertEquals(expect, doc.html()); } + @Test void roundtripsDoctypeInternalSubset() {+ // https://github.com/jhy/jsoup/issues/2486+ String xml = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n" ++ "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\" [\n" ++ "<!ENTITY ns_extend \"http://ns.adobe.com/Extensibility/1.0/\">\n" ++ "<!ENTITY ns_ai \"http://ns.adobe.com/AdobeIllustrator/10.0/\">\n" ++ "]>\n" ++ "<svg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 100 100\">\n" ++ "<circle cx=\"50\" cy=\"50\" r=\"40\" fill=\"red\"/>\n" ++ "</svg>";+ Document doc = Jsoup.parse(xml, Parser.xmlParser());++ DocumentType doctype = doc.documentType();+ assertNotNull(doctype);+ assertEquals("<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\" [\n" ++ "<!ENTITY ns_extend \"http://ns.adobe.com/Extensibility/1.0/\">\n" ++ "<!ENTITY ns_ai \"http://ns.adobe.com/AdobeIllustrator/10.0/\">\n" ++ "]>", doctype.outerHtml());+ assertFalse(doc.outerHtml().contains("]>"));+ }++ @Test void doesNotCloseDoctypeSubsetOnQuotedGt() {+ String xml = "<!DOCTYPE root [<!ENTITY example \"keep ]> quoted\">]><root/>";+ Document doc = Jsoup.parse(xml, Parser.xmlParser());++ assertEquals(2, doc.childNodeSize());+ DocumentType doctype = (DocumentType) doc.childNode(0);+ assertEquals("<!DOCTYPE root [<!ENTITY example \"keep ]> quoted\">]>", doctype.outerHtml());+ assertEquals("<!DOCTYPE root [<!ENTITY example \"keep ]> quoted\">]><root />", doc.outerHtml());+ }++ @Test void preservesEmptyDoctypeInternalSubset() {+ String xml = "<!DOCTYPE root []><root/>";+ Document doc = Jsoup.parse(xml, Parser.xmlParser());++ assertEquals(2, doc.childNodeSize());+ DocumentType doctype = (DocumentType) doc.childNode(0);+ assertEquals("<!DOCTYPE root []>", doctype.outerHtml());+ assertEquals("<!DOCTYPE root []><root />", doc.outerHtml());+ }++ @Test void unterminatedDoctypeSubsetConsumesToEof() {+ String xml = "<!DOCTYPE root [<!ENTITY x \"unterminated]><root/>";+ Parser parser = Parser.xmlParser().setTrackErrors(10);+ Document doc = Jsoup.parse(xml, "", parser);++ assertEquals(1, doc.childNodeSize());+ DocumentType doctype = (DocumentType) doc.childNode(0);+ assertEquals("<!DOCTYPE root [<!ENTITY x \"unterminated]><root/>]>", doctype.outerHtml());+ assertEquals(1, parser.getErrors().size());+ assertEquals("Unexpectedly reached end of file (EOF) in input state [DoctypeInternalSubset]", parser.getErrors().get(0).getErrorMessage());+ }+ @Test void canSetCustomRcdataTag() { String inner = "Blah\nblah\n<foo></foo>""; String innerText = "Blah\nblah\n<foo></foo>\"";