Backtestspydanticrun-20261007-130130#13460
Unexpected serialization schema with ValidateAs
Worse
The merged fix was judged better than our patch.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Both patches fix the root cause the same way: they generate a core schema for the annotated `source` type and attach it as the serialization schema of the after-validator. This stops serialization from following `from_type`, which caused the `list[int]` warning on an `int` value. Patch A is simpler and more robust. It always wraps the source schema in a `wrap_serializer_function_ser_schema` with `h(v)`. That still honors any custom serializer on the source schema, such as `PlainSerializer`, because the inner handler serializes through that schema. It falls back to `any_schema()` on the full `GENERATE_SCHEMA_ERRORS` tuple, matching the existing convention a few lines above in the same file (the `InstanceOf`/`SkipValidation` code). Patch B is riskier. It reuses `source_schema['serialization']` directly when present. Its own TODO admits this can produce an invalid schema when the serialization is an include-exclude dict/sequence schema, which it papers over with type-ignore comments. It catches only `PydanticSchemaGenerationError`, which is narrower than the codebase's established error set. It also calls `handler.generate_schema(source)` a second time just to supply a `return_schema`, which is redundant work. On tests, B is more thorough: it turns warnings into errors, checks both JSON schema modes, and covers a custom serializer on the source type and an arbitrary source type. A's tests are thinner but still meaningful: the JSON dump of the issue scenario, plus the unknown-schema fallback. B's better tests do not outweigh A's cleaner and less fragile implementation, which also avoids a known invalid-schema edge case.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
11m 17s
Wall-clock time in the container
Full test suite
No new failures
12,302 passed · 501 skipped · 12,803 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
80vs 56 merged
Changed lines, ours vs merged
The issue
View on GitHubInitial Checks
- I confirm that I'm using Pydantic V2
Description
From https://github.com/pydantic/pydantic/discussions/13232:
import pydantic as p
import typing as t
def first_even(nums: list[int]) -> int:
for i in nums:
if i%2 == 0:
return i
raise ValueError("Even not found.")
class Foo(p.BaseModel):
x: t.Annotated[int, p.ValidateAs(list[int], first_even)]
d = Foo(x=[1, 3, 5, 8, 6, 7])
print(d)
# x=8
print(d.model_dump())
# PydanticSerializationUnexpectedValue(Expected `list[int]` - serialized value may not be as expected [field_name='x', input_value=8, input_type=int])
# {'x': 8}
Example Code
Python, Pydantic & OS Version
mainThe two patches
Our patch+80 −0
pydantic/functional_validators.pyindex 6a38f619..5e331f2d 100644@@ -894,8 +894,29 @@ class ValidateAs: self.instantiation_hook = instantiation_hook def __get_pydantic_core_schema__(self, source: Any, handler: GetCoreSchemaHandler) -> core_schema.CoreSchema:+ # The validated value is an instance of the annotated (source) type, not of `from_type`,+ # so serialization must follow the source type's schema. If Pydantic can't generate a core schema+ # for the source type (e.g. an unsupported custom type), the default serialization behavior is kept.+ from pydantic import PydanticSchemaGenerationError++ try:+ source_schema = handler(source)+ # TODO if `source_schema['serialization']` is one of `'include-exclude-dict/sequence',+ # schema validation will fail. That's why we use 'type ignore' comments below.+ serialization = source_schema.get(+ 'serialization',+ core_schema.wrap_serializer_function_ser_schema(+ function=lambda v, h: h(v),+ schema=source_schema,+ return_schema=handler.generate_schema(source),+ ),+ )+ except PydanticSchemaGenerationError:+ serialization = None+ schema = handler(self.from_type) return core_schema.no_info_after_validator_function( self.instantiation_hook, schema=schema,+ serialization=serialization, # pyright: ignore[reportArgumentType] )tests/test_types.pyindex e70a0102..5a4a13e1 100644@@ -6096,6 +6096,65 @@ def test_validate_from() -> None: assert ta.validate_python({'a': 1}) == Arbitrary(1) +def test_validate_as_serializes_as_source_type() -> None:+ """https://github.com/pydantic/pydantic/discussions/13232"""++ def first_even(nums: list[int]) -> int:+ for i in nums:+ if i % 2 == 0:+ return i+ raise ValueError('Even not found.')++ class Foo(BaseModel):+ x: Annotated[int, ValidateAs(list[int], first_even)]++ with warnings.catch_warnings():+ warnings.simplefilter('error')+ foo = Foo(x=[1, 3, 5, 8, 6, 7])+ assert foo.x == 8+ assert foo.model_dump() == {'x': 8}+ assert foo.model_dump_json() == '{"x":8}'++ assert Foo.model_json_schema(mode='validation')['properties']['x'] == {+ 'items': {'type': 'integer'},+ 'title': 'X',+ 'type': 'array',+ }+ assert Foo.model_json_schema(mode='serialization')['properties']['x'] == {'title': 'X', 'type': 'integer'}+++def test_validate_as_uses_source_type_custom_serializer() -> None:+ class Foo(BaseModel):+ x: Annotated[+ Annotated[int, PlainSerializer(lambda v: f'#{v}')],+ ValidateAs(str, int),+ ]++ with warnings.catch_warnings():+ warnings.simplefilter('error')+ foo = Foo(x='12')+ assert foo.x == 12+ assert foo.model_dump() == {'x': '#12'}+++def test_validate_as_arbitrary_source_type_serialization() -> None:+ class Arbitrary:+ def __init__(self, a: int) -> None:+ self.a = a++ class ArbitraryModel(BaseModel):+ a: int++ class Foo(BaseModel, arbitrary_types_allowed=True):+ x: Annotated[Arbitrary, ValidateAs(ArbitraryModel, instantiation_hook=lambda v: Arbitrary(a=v.a))]++ with warnings.catch_warnings():+ warnings.simplefilter('error')+ foo = Foo(x={'a': 1})+ assert isinstance(foo.x, Arbitrary)+ assert foo.model_dump()['x'] is foo.x++ @pytest.mark.skipif(sys.version_info < (3, 12), reason="`Annotated` doesn't allow instances in <3.12") def test_skip_validation_arbitrary_type_object() -> None: """https://github.com/pydantic/pydantic/issues/11997.
Merged fix · PR #13487+52 −4
pydantic/functional_validators.pyindex 6a38f6195..336eff24d 100644@@ -794,6 +794,7 @@ else: @classmethod def __get_pydantic_core_schema__(cls, source: Any, handler: GetCoreSchemaHandler) -> core_schema.CoreSchema:+ # Lazy import to avoid cycle: from pydantic._internal._generate_schema import GENERATE_SCHEMA_ERRORS # use the generic _origin_ as the second argument to isinstance when appropriate@@ -894,8 +895,24 @@ class ValidateAs: self.instantiation_hook = instantiation_hook def __get_pydantic_core_schema__(self, source: Any, handler: GetCoreSchemaHandler) -> core_schema.CoreSchema:- schema = handler(self.from_type)- return core_schema.no_info_after_validator_function(+ # Lazy import to avoid cycle:+ from pydantic._internal._generate_schema import GENERATE_SCHEMA_ERRORS++ from_type_schema = handler(self.from_type)+ schema = core_schema.no_info_after_validator_function( self.instantiation_hook,- schema=schema,+ schema=from_type_schema,+ )+ try:+ ser_schema = handler(source)+ except GENERATE_SCHEMA_ERRORS:+ ser_schema = core_schema.any_schema()++ # TODO: pydantic-core currently doesn't accept it, but we should be able to provide an+ # arbitrary core schema for serialization (currently `simple_ser_schema()` can be used,+ # but only allow specifying a specific `'type`). Working around this with a wrap serializer+ # is already done in other parts of the code base, so grep for similar cases when fixing:+ schema['serialization'] = core_schema.wrap_serializer_function_ser_schema(+ function=lambda v, h: h(v), schema=ser_schema )+ return schematests/test_types.pyindex e70a0102e..df4d5f060 100644@@ -6080,7 +6080,7 @@ def test_skip_validation_json_schema(): } -def test_validate_from() -> None:+def test_validate_as() -> None: class Arbitrary: def __init__(self, a: int) -> None: self.a = a@@ -6096,6 +6096,37 @@ def test_validate_from() -> None: assert ta.validate_python({'a': 1}) == Arbitrary(1) +def test_validate_as_known_serialization() -> None:+ """https://github.com/pydantic/pydantic/issues/13460"""++ def first_even(nums: list[int]) -> int:+ for i in nums:+ if i % 2 == 0:+ return i+ raise ValueError('Even not found.')++ class Model(BaseModel):+ x: Annotated[int, ValidateAs(list[int], first_even)]++ m = Model(x=[1, 2])++ assert m.model_dump_json() == '{"x":2}'+++def test_validate_as_unknown_serialization() -> None:++ class Arbitrary:+ def __init__(self, a: int) -> None:+ self.a = a++ def __eq__(self, other) -> bool:+ return self.a == other.a++ ta = TypeAdapter(Annotated[Arbitrary, ValidateAs(int, instantiation_hook=Arbitrary)])++ assert ta.dump_python(Arbitrary(1)) == Arbitrary(1)++ @pytest.mark.skipif(sys.version_info < (3, 12), reason="`Annotated` doesn't allow instances in <3.12") def test_skip_validation_arbitrary_type_object() -> None: """https://github.com/pydantic/pydantic/issues/11997.