Backtestspydanticrun-20261027-165153#13729

Issue #13729 Merged fix PR #13731 Run 27 Oct 2026 · run-20261027-165153 pydantic/pydantic

Smart union selects the wrong model in JSON mode with a before validator

Better

Our patch was judged better than the merged fix.

In this judgment, Patch A = our agent, Patch B = the merged fix.

The Rust change is byte-for-byte equivalent in both patches (apart from one blank line): both remove the outer `state.add_fields_set(fields_set.len())` in `ModelFieldsValidator::validate` and instead count successfully validated known fields inside `validate_json_object`, adding that count once at the end. That matches the issue's diagnosis: the Python-input helper already added its own count, so the outer add double-counted on the before-validator path, while the plain JSON path got only the outer count. Independent tests confirm both patches fix every scenario, including reversed union order and nested-field usage. The difference is tests. Patch A adds a regression test at the pydantic-core level (tests/validators/test_union.py) using a raw `no_info_before_validator_function` wrapping a `model_schema`, parametrized over python/json and over permuted choice order, which exercises the fix where it actually lives and would catch a regression in pydantic-core independently of the pydantic wrapper. It also adds an end-to-end reproduction of the issue example in tests/test_validators.py. Patch B adds only pydantic-level tests, albeit with extra/reverse-order parametrization, and leaves an unfilled `<link>` docstring placeholder (stripped reference, ignored for scoring but still a stub). Both patches include an extra-fields hardening test that is neutral. Scope and craft are otherwise equal, so Patch A's two-layer test coverage, matched to the location of the fix, gives it a modest but real edge.

The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.

Solve

7m 28s

Wall-clock time in the container

Full test suite

No new failures

488 passed · 92 skipped · 580 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

103vs 55 merged

Changed lines, ours vs merged

Initial Checks

  • I confirm that I'm using Pydantic V2

Description

Adding a no-op mode='before' model validator makes smart-union validation select a model with fewer matching fields for JSON input. The equivalent Python input selects the model with more matching fields.

The example below prints:

Larger
Smaller

I expect Larger in both cases: it has three supplied valid fields, while Smaller has two. This follows the documented number-of-valid-fields-set ranking.

Reproduced on main at 965c23dd93bd5ca7b86224ba39ccbe79399f117b. In model_fields.rs, the Python-input helper adds its known-field count, then the outer validator adds fields_set.len() again. The before validator sends the smaller model through that Python-input path, while the larger model uses the plain JSON path and receives only the outer count.

I can send a focused fix with regression tests. Could you assign this to me?

Example Code

from typing import Any

from pydantic import BaseModel, TypeAdapter, model_validator


class Smaller(BaseModel):
    a: int
    b: int

    @model_validator(mode='before')
    @classmethod
    def keep_input(cls, value: Any) -> Any:
        return value


class Larger(BaseModel):
    a: int
    b: int
    c: int


adapter = TypeAdapter(Smaller | Larger)
print(type(adapter.validate_python({'a': 1, 'b': 2, 'c': 3})).__name__)
print(type(adapter.validate_json('{"a": 1, "b": 2, "c": 3}')).__name__)

Python, Pydantic & OS Version

             pydantic version: 2.14.0b1
        pydantic-core version: 2.48.0
          pydantic-core build: profile=debug pgo=false
               python version: 3.13.14 (main, Jul 18 2026, 16:56:45) [Clang 22.1.3 ]
                     platform: macOS-26.6.2-arm64-arm-64bit-Mach-O
             related packages: mypy-2.3.0 pyright-1.1.411 typing_extensions-4.15.0
                       commit: 965c23dd9

Prepared with AI assistance.

The two patches

Our patch+102 −1
pydantic-core/src/validators/model_fields.rsindex 5e8fe034..c0dcd5e7 100644@@ -179,7 +179,6 @@ impl Validator for ModelFieldsValidator {             };             self.validate_by_get_item(py, input, dict, state)?         };-        state.add_fields_set(fields_set.len());          // if we have extra=allow, but we didn't create a dict because we were validating         // from attributes, set it now so __pydantic_extra__ is always a dict if extra=allow@@ -560,6 +559,7 @@ impl ModelFieldsValidator {             (0..self.fields.len()).map(|_| None).collect();         let mut errors: Vec<ValLineError> = Vec::new();         let fields_set = PySet::empty(py)?;+        let mut fields_set_count: usize = 0;          let state = &mut state.scoped_set_data(Some(model_dict.clone()));         let state = &mut state.scoped_clear_field_error();@@ -636,6 +636,7 @@ impl ModelFieldsValidator {                 match field.validator.validate(py, field_json_value, state) {                     Ok(value) => {                         fields_set.add(&field.name)?;+                        fields_set_count += 1;                         value                     }                     Err(ValError::Omit) => continue,@@ -697,6 +698,8 @@ impl ModelFieldsValidator {             return Err(ValError::LineErrors(errors));         } +        state.add_fields_set(fields_set_count);+         Ok((model_dict, model_extra_dict_op, fields_set))     } }pydantic-core/tests/validators/test_union.pyindex 268e5a40..4a49ec3b 100644@@ -1424,3 +1424,79 @@ def test_smart_union_wrap_validator_should_not_change_nested_model_field_counts(     assert isinstance(m, RootModel)     assert isinstance(m.ab, ModelA)     assert m.ab.sub.x == 'y'+++@pytest.mark.parametrize('mode', ['python', 'json'])+def test_smart_union_before_validator_does_not_change_model_field_counts(mode) -> None:+    """A before validator on one choice must not change how model fields are counted for that choice."""++    class Smaller:+        a: int+        b: int++    class Larger:+        a: int+        b: int+        c: int++    smaller_schema = core_schema.no_info_before_validator_function(+        lambda v: v,+        core_schema.model_schema(+            Smaller,+            core_schema.model_fields_schema(+                fields={+                    'a': core_schema.model_field(core_schema.int_schema()),+                    'b': core_schema.model_field(core_schema.int_schema()),+                }+            ),+        ),+    )+    larger_schema = core_schema.model_schema(+        Larger,+        core_schema.model_fields_schema(+            fields={+                'a': core_schema.model_field(core_schema.int_schema()),+                'b': core_schema.model_field(core_schema.int_schema()),+                'c': core_schema.model_field(core_schema.int_schema()),+            }+        ),+    )++    for choices in permute_choices([smaller_schema, larger_schema]):+        validator = SchemaValidator(core_schema.union_schema(choices))+        if mode == 'python':+            result = validator.validate_python({'a': 1, 'b': 2, 'c': 3})+        else:+            result = validator.validate_json('{"a": 1, "b": 2, "c": 3}')+        assert isinstance(result, Larger)+++@pytest.mark.parametrize('extra_behavior', ['forbid', 'ignore', 'allow'])+def test_smart_union_extra_behavior_json(extra_behavior) -> None:+    """Extra fields must not count as valid fields set in JSON mode, matching Python mode."""++    class Foo:+        foo: str = 'foo'++    class Bar:+        bar: str = 'bar'++    def member_schema(cls, field_name):+        return core_schema.model_schema(+            cls,+            core_schema.model_fields_schema(+                fields={+                    field_name: core_schema.model_field(+                        core_schema.with_default_schema(core_schema.str_schema(), default=field_name)+                    )+                }+            ),+            extra_behavior=extra_behavior,+        )++    validator = SchemaValidator(core_schema.union_schema([member_schema(Foo, 'foo'), member_schema(Bar, 'bar')]))++    assert isinstance(validator.validate_json('{"foo": "foo"}'), Foo)+    assert isinstance(validator.validate_json('{"bar": "bar"}'), Bar)+    assert isinstance(validator.validate_python({'foo': 'foo'}), Foo)+    assert isinstance(validator.validate_python({'bar': 'bar'}), Bar)tests/test_validators.pyindex 7e9f4713..f6352eb9 100644@@ -3254,3 +3254,25 @@ def test_wrap_validator_forwards_runtime_settings() -> None:     with pytest.raises(ValidationError, match='Input should be a valid dictionary'):         Outer.model_validate({'inner': InnerObj()}, context='ctx')     assert Outer.model_validate({'inner': InnerObj()}, from_attributes=True, context='ctx').inner.a == 1+++def test_smart_union_model_before_validator_json_and_python() -> None:+    """A `mode='before'` model validator must not change smart union field-count ranking in JSON mode."""++    class Smaller(BaseModel):+        a: int+        b: int++        @model_validator(mode='before')+        @classmethod+        def keep_input(cls, value: Any) -> Any:+            return value++    class Larger(BaseModel):+        a: int+        b: int+        c: int++    adapter = TypeAdapter(Smaller | Larger)+    assert type(adapter.validate_python({'a': 1, 'b': 2, 'c': 3})) is Larger+    assert type(adapter.validate_json('{"a": 1, "b": 2, "c": 3}')) is Larger 
Merged fix · PR #13731+53 −2
pydantic-core/src/validators/model_fields.rsindex 5e8fe0349..913d51292 100644@@ -179,7 +179,6 @@ impl Validator for ModelFieldsValidator {             };             self.validate_by_get_item(py, input, dict, state)?         };-        state.add_fields_set(fields_set.len());          // if we have extra=allow, but we didn't create a dict because we were validating         // from attributes, set it now so __pydantic_extra__ is always a dict if extra=allow@@ -560,6 +559,7 @@ impl ModelFieldsValidator {             (0..self.fields.len()).map(|_| None).collect();         let mut errors: Vec<ValLineError> = Vec::new();         let fields_set = PySet::empty(py)?;+        let mut fields_set_count: usize = 0;          let state = &mut state.scoped_set_data(Some(model_dict.clone()));         let state = &mut state.scoped_clear_field_error();@@ -636,6 +636,7 @@ impl ModelFieldsValidator {                 match field.validator.validate(py, field_json_value, state) {                     Ok(value) => {                         fields_set.add(&field.name)?;+                        fields_set_count += 1;                         value                     }                     Err(ValError::Omit) => continue,@@ -697,6 +698,7 @@ impl ModelFieldsValidator {             return Err(ValError::LineErrors(errors));         } +        state.add_fields_set(fields_set_count);         Ok((model_dict, model_extra_dict_op, fields_set))     } }tests/types/unions/test_union.pyindex b68d9cf19..f7a3f60f2 100644@@ -1,6 +1,6 @@ from abc import ABC, abstractmethod from enum import IntEnum-from typing import Annotated, ClassVar, Literal+from typing import Annotated, Any, ClassVar, Literal from uuid import UUID  import pytest@@ -14,6 +14,7 @@ from pydantic import (     Strict,     TypeAdapter,     ValidationError,+    model_validator, )  @@ -235,6 +236,54 @@ def test_union_typeddict():     assert ta.validate_python(dict(foo='baz')) == {'foo': 'baz'}  +@pytest.mark.parametrize('extra', ['ignore', 'allow'])+@pytest.mark.parametrize('reverse_members', [False, True])+def test_smart_union_fields_set_count_consistent_between_json_and_python(+    extra: Literal['ignore', 'allow'], reverse_members: bool+) -> None:+    """https://github.com/pydantic/pydantic/issues/13729"""++    class Smaller(BaseModel):+        model_config = ConfigDict(extra=extra)++        a: int+        b: int++        @model_validator(mode='before')+        @classmethod+        def keep_input(cls, value: Any) -> Any:+            return value++    class Larger(BaseModel):+        a: int+        b: int+        c: int++    ta = TypeAdapter(Larger | Smaller if reverse_members else Smaller | Larger)++    assert isinstance(ta.validate_python({'a': 1, 'b': 2, 'c': 3}), Larger)+    assert isinstance(ta.validate_json('{"a": 1, "b": 2, "c": 3}'), Larger)+++@pytest.mark.parametrize('reverse_members', [False, True])+def test_smart_union_fields_set_count_ignores_extra(reverse_members: bool) -> None:+    """https://github.com/pydantic/pydantic/issues/13729"""++    class WithExtra(BaseModel):+        model_config = ConfigDict(extra='allow')++        a: int++    class Explicit(BaseModel):+        a: int+        b: int++    ta = TypeAdapter(Explicit | WithExtra if reverse_members else WithExtra | Explicit)++    assert isinstance(ta.validate_python({'a': 1, 'b': 2}), Explicit)+    assert isinstance(ta.validate_json('{"a": 1, "b": 2}'), Explicit)++ def test_union_respects_local_strict() -> None:     class MyBaseModel(BaseModel):         model_config = ConfigDict(strict=True)