Backtestsjsouprun-20261006-181938#2588
Decode named entities before - and _ in attributes
The merged fix was judged better than our patch.
In this judgment, Patch B = our agent, Patch A = the merged fix.
The production change is identical in both patches. Each removes '-' and '_' from the in-attribute rewind condition in Tokeniser, so semicolonless named references followed by those characters now decode. References followed by '=' or an ASCII alphanumeric are still left alone, which matches the HTML spec's attribute rule. Correctness is therefore equal: the issue's cases ('×tamp=', '¶m=') still stay literal in attributes, and '©-' / '×-' now decode. Both patches add tests that would fail before the change and pass after it.
Patch B covers several distinct entities (×, ¶, ©, &, <, >, ¬) across single-quoted, double-quoted and unquoted values. It also covers the issue's query-string case and the static Parser.unescapeEntities path.
Patch A's tests are more systematic:
- Each scenario runs over all quoting styles.
- They separate the three behaviours (punctuation decodes; '=' does not; alphanumerics in upper case, lower case and digits do not) and contrast semicolon-terminated forms in each case.
- They pin the attribute-versus-text difference for the issue's exact URL, both through parsing and through Parser.unescapeEntities in both modes.
- Patch A also adds a CHANGES.md entry, following the project's convention.
Scope and craft are otherwise comparable, with clear comments in both. Patch A edges ahead on more thorough, well-structured tests and the changelog entry.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
7m 13s
Wall-clock time in the container
Full test suite
No new failures
2,227 passed · 0 skipped · 2,227 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
20vs 51 merged
Changed lines, ours vs merged
The issue
View on GitHubSince Jsoup version 1.18.1, when parsing input parameters, it performs fuzzy matching on the string following '&' against HTML entities and automatically converts them to HTML entities. For example, when a parameter contains '×tamp=', it fuzzy-matches to '×' and automatically converts it to '×' (multiplication sign). When a parameter contains '¶m', it fuzzy-matches to '¶' and converts it to the paragraph symbol (¶). This causes input validation to fail. Before version 1.18.1, Jsoup would automatically convert '&' to '&', which was fine. However, after the upgrade, it performs fuzzy matching that incorrectly matches HTML entities that should not be matched. I believe this is a bug.
Jsoup 自从1.18.1之后会把入参的&后面的字符串进行模糊匹配html并自动转换为html的实体,例如传参涉及×tamp=会模糊匹配到×自动转成*,传参涉及¶m会模糊匹配到¶自动转成段落符号;导致入参失败,1.18.1之前是会将&自动转成&这是没问题的,但是升级之后会导致模糊匹配到不应该匹配的html实体格式,我理解这是个bug
The two patches
src/main/java/org/jsoup/parser/Tokeniser.javaindex 4765429f..08d58fc3 100644@@ -198,8 +198,8 @@ final class Tokeniser { reader.matchConsume(prefix); nameRef = prefix; }- if (inAttribute && (reader.matchesAsciiAlpha() || reader.matchesDigit() || reader.matchesAny('=', '-', '_'))) {- // don't want that to match+ if (inAttribute && (reader.matchesAsciiAlpha() || reader.matchesDigit() || reader.matches('='))) {+ // in an attribute, a reference without ; followed by = or alphanumeric is not decoded (per spec). Others like - or _ are decoded reader.rewindToMark(); return null; }src/test/java/org/jsoup/parser/AttributeParseTest.javaindex d328b9fb..d8b4c04f 100644@@ -73,6 +73,22 @@ public class AttributeParseTest { assertEquals("&wr_id=123&mid-size=true&ok=&wr", els.first().attr("href")); } + @Test public void decodesNamedEntitiesBeforeDashAndUnderscoreInAttributes() {+ // per spec, a legacy named ref without ; in an attribute is only left as-is if followed by = or alphanumeric+ String html = "<a id=1 href='?a×-b¶_c©-&_x'>One</a> <a id=2 title=\"<->_\">Two</a> <a id=3 class=¬-x>Three</a>";+ Elements els = Jsoup.parse(html).select("a");+ assertEquals("?a\u00d7-b\u00b6_c\u00a9-&_x", els.get(0).attr("href"));+ assertEquals("<->_", els.get(1).attr("title"));+ assertEquals("\u00ac-x", els.get(2).attr("class"));++ // and still not decoded when followed by = or alphanumerics, or when not a full entity name+ String html2 = "<a href='?x=1×tamp=2¶m=3×=4¶5'>Four</a>";+ assertEquals("?x=1×tamp=2¶m=3×=4¶5", Jsoup.parse(html2).selectFirst("a").attr("href"));++ // static unescape in attribute mode follows the same rules+ assertEquals("\u00d7-1×=2", Parser.unescapeEntities("×-1×=2", true));+ }+ @Test public void parsesBooleanAttributes() { String html = "<a normal=\"123\" boolean empty=\"\"></a>"; Element el = Jsoup.parse(html).select("a").first();
CHANGES.mdindex 5f5e8222..120d76db 100644@@ -4,6 +4,7 @@ ### Bug Fixes * Standardized parser normalization of tag and attribute names so that HTML comparisons use ASCII-only case folding, and accepted control characters are preserved, aligning name handling to the HTML and XML specs. [#2594](https://github.com/jhy/jsoup/issues/2594)+* Named character references without a semicolon before `-` or `_` now decode correctly in attribute values, matching HTML and browser behavior (e.g., `©-` becomes `©-`). [#2588](https://github.com/jhy/jsoup/issues/2588) ## 1.23.2 (2026-Aug-26) src/main/java/org/jsoup/parser/Tokeniser.javaindex 4765429f..39e7748d 100644@@ -198,8 +198,8 @@ final class Tokeniser { reader.matchConsume(prefix); nameRef = prefix; }- if (inAttribute && (reader.matchesAsciiAlpha() || reader.matchesDigit() || reader.matchesAny('=', '-', '_'))) {- // don't want that to match+ if (inAttribute && (reader.matchesAsciiAlpha() || reader.matchesDigit() || reader.matches('='))) {+ // in attributes, don't consume semicolonless references followed by ASCII alphanumeric or equals reader.rewindToMark(); return null; }src/test/java/org/jsoup/parser/HtmlParserTest.javaindex dbc69fc1..4db0c738 100644@@ -1311,6 +1311,39 @@ public class HtmlParserTest { assertEquals("& \" ® &icy &hopf и 𝕙", doc.body().html()); } + @Test public void decodesAttributeEntitiesBeforePunctuation() {+ for (String quote : new String[]{"\"", "'", ""}) {+ String html = "<a title=" + quote + "©-©_" + quote + ">One</a>";+ assertEquals("©-©_", Jsoup.parse(html).expectFirst("a").attr("title"), html);+ }+ }++ @Test public void preservesAttributeEntitiesBeforeEquals() {+ for (String quote : new String[]{"\"", "'"}) {+ String html = "<a title=" + quote + "©=©=" + quote + ">One</a>";+ assertEquals("©=©=", Jsoup.parse(html).expectFirst("a").attr("title"), html);+ }+ }++ @Test public void preservesAttributeEntitiesBeforeAlphanumeric() {+ for (String quote : new String[]{"\"", "'", ""}) {+ String value = "©a©Z©1©A©1";+ String html = "<a title=" + quote + value + quote + ">One</a>";+ assertEquals("©a©Z©1©A©1", Jsoup.parse(html).expectFirst("a").attr("title"), html);+ }+ }++ @Test public void queryParametersDecodeEntitiesInTextOnly() {+ // https://github.com/jhy/jsoup/issues/2588+ String value = "?one=1×tamp=2¶m=3";+ for (String quote : new String[]{"\"", "'"}) {+ String html = "<a href=" + quote + value + quote + ">" + value + "</a>";+ Element el = Jsoup.parse(html).expectFirst("a");+ assertEquals(value, el.attr("href"), html);+ assertEquals("?one=1×tamp=2¶m=3", el.text(), html);+ }+ }+ @Test public void findsBasePrefixEntity() { // https://github.com/jhy/jsoup/issues/2207 String html = "a c­c I'm ¬it; I tell you. I'm ∉ I tell you.";src/test/java/org/jsoup/parser/ParserTest.javaindex bfebf6be..3e51544a 100644@@ -22,6 +22,19 @@ public class ParserTest { assertEquals("One & Two", s); } + @Test public void unescapeEntitiesInAttributes() {+ String input = "©- ©_ ©= ©a ©Z ©1 ©= ©A ©1";+ assertEquals("©- ©_ ©= ©a ©Z ©1 ©= ©A ©1", Parser.unescapeEntities(input, true));+ assertEquals("©- ©_ ©= ©a ©Z ©1 ©= ©A ©1", Parser.unescapeEntities(input, false));+ }++ @Test public void unescapeQueryParametersInTextOnly() {+ // https://github.com/jhy/jsoup/issues/2588+ String input = "?one=1×tamp=2¶m=3";+ assertEquals(input, Parser.unescapeEntities(input, true));+ assertEquals("?one=1×tamp=2¶m=3", Parser.unescapeEntities(input, false));+ }+ @Test public void unescapeEntitiesHandlesLargeInput() { StringBuilder longBody = new StringBuilder(500000);