Backtestspydanticrun-20261022-024030#13579
Allow arbitrary core schemas for serialization
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Both patches give pydantic-core a way to say "serialize with this core schema", but they differ in design and completeness. Patch A changes the Rust fallthrough to `Self::build(&ser_schema)` and widens `SerSchema` to include `CoreSchema` (via a string-typed alias plus `Union[...] # noqa` workarounds). That lets a core schema be passed directly, which is why the four independent tests that pass a raw schema as `serialization` pass only with A. However, A has real gaps: (1) it does not touch `self_schema.py`, so `validate_core_schema` would still reject a raw core schema in `serialization`; (2) the design is ambiguous for `function-plain`/`function-wrap` validator schemas (they collide with the ser-function schemas), which A papers over with a new `as_ser_schema` helper that is tested but never used; (3) it leaves every identity-wrap workaround in pydantic in place and merely rewrites the `ValidateAs` TODO instead of resolving it. The issue is explicitly framed around that TODO and the workaround sites, and the one independent test checking that `ValidateAs` no longer goes through `FunctionWrap` fails on A. Patch B adds an explicit `SchemaSerSchema` (`type: 'schema'`) with a `schema_ser_schema()` helper, a dedicated Rust arm that builds a full serializer from the inner schema (so nested `serialization`, definitions, recursion, models, dataclasses all work, as its core tests show), updates the self-schema so schema validation accepts it, handles the new type in `GenerateJsonSchema.ser_schema`, and migrates all workaround sites (`ValidateAs`, `PlainValidator`, `SkipValidation`, `InstanceOf`, non-dict Mapping) with pydantic-level tests asserting the wrap serializer is gone and filtering/json-schema still work. It avoids the function-schema ambiguity entirely. The issue text itself points at the missing piece as a `SerSchema` member that "takes a schema", which is exactly what B adds; the four independent tests B fails do so only because they assume the raw-schema API shape, not because the capability is missing. On correctness B is complete end-to-end and consistent with self-schema validation; A delivers the core capability with typing hacks, an unused helper, and no caller migration. On tests both are solid, B's cover more real scenarios. On craft B is cleaner and follows existing `SerSchema` conventions. B wins.
Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.
Solve
16m 6s
Wall-clock time in the container
Full test suite
No new failures
3,473 passed · 241 skipped · 3,714 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
321vs 278 merged
Changed lines, ours vs merged
The issue
View on GitHubAs per the TODO in https://github.com/pydantic/pydantic/pull/13487:
There is currently no way to tell pydantic-core "serialize this value using this core schema". SerSchema is:
SerSchema = Union[
SimpleSerSchema,
PlainSerializerFunctionSerSchema,
WrapSerializerFunctionSerSchema,
FormatSerSchema,
ToStringSerSchema,
ModelSerSchema,
]
The only member that expresses "use this schema" is SimpleSerSchema, and it does not take a schema at all, only a name from a closed literal:
ExpectedSerializationTypes = Literal[
'none', 'int', 'bool', 'float', 'str', 'bytes', 'bytearray', 'list', 'tuple',
'set', 'frozenset', 'generator', 'dict', 'datetime', 'date', 'time', 'timedelta',
'url', 'multi-host-url', 'json', 'uuid', 'any',
]
class SimpleSerSchema(TypedDict, total=False):
type: Required[ExpectedSerializationTypes]
So an arbitrary schema (a model, a dataclass, list[int], a TypedDict, anything constructed by the schema generator) cannot be used directly. Every place in Pydantic that needs it works around the gap the same way: it wraps the schema in a wrap serializer whose function is the identity, purely so the schema can ride along in the schema argument.
From ValidateAs.__get_pydantic_core_schema__():
The two patches
pydantic-core/python/pydantic_core/core_schema.pyindex cb87ad5a..3da479fa 100644@@ -476,6 +476,36 @@ def model_ser_schema(cls: type[Any], schema: CoreSchema) -> ModelSerSchema: return ModelSerSchema(type='model', cls=cls, schema=schema) +class SchemaSerSchema(TypedDict, total=False):+ type: Required[Literal['schema']]+ schema: Required[CoreSchema]+++def schema_ser_schema(schema: CoreSchema) -> SchemaSerSchema:+ """+ Returns a schema for serialization using an arbitrary core schema.++ The value is serialized as if it was validated by `schema`, regardless of the+ schema it is attached to:++ ```py+ from pydantic_core import SchemaSerializer, core_schema++ schema = core_schema.any_schema(+ serialization=core_schema.schema_ser_schema(+ core_schema.list_schema(core_schema.int_schema())+ )+ )+ s = SchemaSerializer(schema)+ assert s.to_json([1, 2]) == b'[1,2]'+ ```++ Args:+ schema: The core schema to use to serialize the value+ """+ return SchemaSerSchema(type='schema', schema=schema)++ SerSchema: TypeAlias = ( SimpleSerSchema | PlainSerializerFunctionSerSchema@@ -483,6 +513,7 @@ SerSchema: TypeAlias = ( | FormatSerSchema | ToStringSerSchema | ModelSerSchema+ | SchemaSerSchema ) pydantic-core/src/self_schema.pyindex 18e6cc0a..c21d767a 100644@@ -6846,6 +6846,14 @@ self_schema = { }, 'extra_behavior': 'forbid', },+ 'schema': {+ 'type': 'typed-dict',+ 'fields': {+ 'type': {'schema': {'type': 'literal', 'expected': ['schema']}, 'required': True},+ 'schema': {'schema': {'type': 'definition-ref', 'schema_ref': 'root-schema'}, 'required': True},+ },+ 'extra_behavior': 'forbid',+ }, }, 'ref': 'ser-schema', },pydantic-core/src/serializers/shared.rsindex baf2dcc0..4f2c1c06 100644@@ -198,6 +198,14 @@ impl CombinedSerializer { ) .map_err(|err| py_schema_error_type!("Error building `function-wrap` serializer:\n {err}")); }+ Some("schema") => {+ // `schema` means "serialize using this (arbitrary) core schema", so we build a full+ // serializer from `schema.serialization.schema`, ignoring `schema.type`+ return ser_schema+ .get_as_req::<Bound<'_, PyDict>>(intern!(py, "schema"))+ .and_then(|inner_schema| Self::build(&inner_schema, config, definitions))+ .map_err(|err| py_schema_error_type!("Error building `schema` serializer:\n {err}"));+ } Some( // applies to lists tuples and dicts, does not override the main schema `type` "include-exclude-sequence" | "include-exclude-dict"pydantic-core/tests/serializers/test_schema_ser_schema.pynew file mode 100644index 00000000..929b25e0@@ -0,0 +1,160 @@+from dataclasses import dataclass+from typing import Any++import pytest++from pydantic_core import SchemaError, SchemaSerializer, core_schema++from ..conftest import plain_repr+++def test_schema_ser_schema_list():+ s = SchemaSerializer(+ core_schema.any_schema(+ serialization=core_schema.schema_ser_schema(core_schema.list_schema(core_schema.int_schema()))+ )+ )+ assert (+ plain_repr(s)+ == 'SchemaSerializer(serializer=List(ListSerializer{item_serializer:Int(IntSerializer),filter:SchemaFilter{include:None,exclude:None},name:"list[int]"}),definitions=[])'+ )++ assert s.to_python([1, 2]) == [1, 2]+ assert s.to_json([1, 2]) == b'[1,2]'+ assert s.to_python([1, 2, 3], include={1}) == [2]++ with pytest.warns(UserWarning, match='Expected `list\\[int\\]`'):+ assert s.to_python('foo') == 'foo'+++def test_schema_ser_schema_overrides_outer_type():+ # the outer schema would serialize with `to-string`, the attached schema serializes as a plain `str`+ s = SchemaSerializer(+ core_schema.no_info_after_validator_function(+ str,+ core_schema.int_schema(),+ serialization=core_schema.schema_ser_schema(core_schema.str_schema()),+ )+ )+ assert s.to_python('abc') == 'abc'+ assert s.to_json('abc') == b'"abc"'++ with pytest.warns(UserWarning, match='Expected `str`'):+ s.to_python(123)+++def test_schema_ser_schema_honors_inner_serialization():+ inner = core_schema.int_schema(serialization=core_schema.plain_serializer_function_ser_schema(lambda v: f'int={v}'))+ s = SchemaSerializer(core_schema.any_schema(serialization=core_schema.schema_ser_schema(inner)))+ assert s.to_python(1) == 'int=1'+ assert s.to_json(1) == b'"int=1"'+++def test_schema_ser_schema_nested():+ s = SchemaSerializer(+ core_schema.any_schema(+ serialization=core_schema.schema_ser_schema(+ core_schema.any_schema(serialization=core_schema.schema_ser_schema(core_schema.bytes_schema()))+ )+ )+ )+ assert s.to_python(b'abc', mode='json') == 'abc'+++def test_schema_ser_schema_typed_dict():+ td_schema = core_schema.typed_dict_schema(+ {+ 'a': core_schema.typed_dict_field(core_schema.int_schema()),+ 'b': core_schema.typed_dict_field(core_schema.bytes_schema(), serialization_alias='B'),+ }+ )+ s = SchemaSerializer(core_schema.any_schema(serialization=core_schema.schema_ser_schema(td_schema)))+ assert s.to_python({'a': 1, 'b': b'x'}) == {'a': 1, 'b': b'x'}+ assert s.to_python({'a': 1, 'b': b'x'}, by_alias=True, mode='json') == {'a': 1, 'B': 'x'}+ assert s.to_python({'a': 1, 'b': b'x'}, exclude={'a'}) == {'b': b'x'}+++def test_schema_ser_schema_model():+ class MyModel:+ __slots__ = '__dict__', '__pydantic_extra__', '__pydantic_fields_set__', '__pydantic_private__'+ field_a: int++ model_schema = core_schema.model_schema(+ MyModel,+ core_schema.model_fields_schema({'field_a': core_schema.model_field(core_schema.int_schema())}),+ )+ s = SchemaSerializer(core_schema.any_schema(serialization=core_schema.schema_ser_schema(model_schema)))++ m = MyModel()+ m.__dict__ = {'field_a': 1}+ m.__pydantic_extra__ = None+ m.__pydantic_fields_set__ = {'field_a'}+ m.__pydantic_private__ = None+ assert s.to_python(m) == {'field_a': 1}+ assert s.to_json(m) == b'{"field_a":1}'+++def test_schema_ser_schema_dataclass():+ @dataclass+ class Foo:+ a: int+ b: bytes++ dc_schema = core_schema.dataclass_schema(+ Foo,+ core_schema.dataclass_args_schema(+ 'Foo',+ [+ core_schema.dataclass_field(name='a', schema=core_schema.int_schema()),+ core_schema.dataclass_field(name='b', schema=core_schema.bytes_schema()),+ ],+ ),+ ['a', 'b'],+ )+ s = SchemaSerializer(core_schema.any_schema(serialization=core_schema.schema_ser_schema(dc_schema)))+ assert s.to_python(Foo(a=1, b=b'x')) == {'a': 1, 'b': b'x'}+ assert s.to_json(Foo(a=1, b=b'x')) == b'{"a":1,"b":"x"}'+++def test_schema_ser_schema_definition_ref():+ s = SchemaSerializer(+ core_schema.definitions_schema(+ core_schema.any_schema(+ serialization=core_schema.schema_ser_schema(core_schema.definition_reference_schema('my-int'))+ ),+ [core_schema.int_schema(ref='my-int', serialization=core_schema.to_string_ser_schema(when_used='always'))],+ )+ )+ assert s.to_python(1) == '1'+ assert s.to_json(1) == b'"1"'+++def test_schema_ser_schema_recursive():+ s = SchemaSerializer(+ core_schema.definitions_schema(+ core_schema.definition_reference_schema('node'),+ [+ core_schema.typed_dict_schema(+ {+ 'value': core_schema.typed_dict_field(core_schema.int_schema()),+ 'children': core_schema.typed_dict_field(+ core_schema.any_schema(+ serialization=core_schema.schema_ser_schema(+ core_schema.list_schema(core_schema.definition_reference_schema('node'))+ )+ )+ ),+ },+ ref='node',+ )+ ],+ )+ )+ value: dict[str, Any] = {'value': 1, 'children': [{'value': 2, 'children': []}]}+ assert s.to_python(value) == value+ assert s.to_json(value) == b'{"value":1,"children":[{"value":2,"children":[]}]}'+++def test_schema_ser_schema_missing_schema():+ with pytest.raises(SchemaError, match='Error building `schema` serializer'):+ SchemaSerializer(core_schema.any_schema(serialization={'type': 'schema'})) # type: ignore[typeddict-item]pydantic/_internal/_generate_schema.pyindex 3933e821..21453d76 100644@@ -706,9 +706,7 @@ class GenerateSchema: schema = core_schema.lax_or_strict_schema( lax_schema=lax_schema, strict_schema=strict_schema,- serialization=core_schema.wrap_serializer_function_ser_schema(- lambda v, h: h(v), schema=dict_schema, info_arg=False- ),+ serialization=core_schema.schema_ser_schema(dict_schema), ) return schemapydantic/functional_validators.pyindex 24f39be2..665beb7c 100644@@ -217,14 +217,7 @@ class PlainValidator: schema = handler(source_type) # TODO if `schema['serialization']` is one of `'include-exclude-dict/sequence', # schema validation will fail. That's why we use 'type ignore' comments below.- serialization = schema.get(- 'serialization',- core_schema.wrap_serializer_function_ser_schema(- function=lambda v, h: h(v),- schema=schema,- return_schema=handler.generate_schema(source_type),- ),- )+ serialization = schema.get('serialization', core_schema.schema_ser_schema(schema)) except PydanticSchemaGenerationError: serialization = None @@ -811,9 +804,7 @@ else: return instance_of_schema else: # Use the "original" approach to serialization- instance_of_schema['serialization'] = core_schema.wrap_serializer_function_ser_schema(- function=lambda v, h: h(v), schema=original_schema- )+ instance_of_schema['serialization'] = core_schema.schema_ser_schema(original_schema) return core_schema.json_or_python_schema(python_schema=instance_of_schema, json_schema=original_schema) __hash__ = object.__hash__@@ -847,9 +838,7 @@ else: metadata = {'pydantic_js_annotation_functions': [lambda _c, h: h(original_schema)]} return core_schema.any_schema( metadata=metadata,- serialization=core_schema.wrap_serializer_function_ser_schema(- function=lambda v, h: h(v), schema=original_schema- ),+ serialization=core_schema.schema_ser_schema(original_schema), ) __hash__ = object.__hash__@@ -911,11 +900,5 @@ class ValidateAs: except GENERATE_SCHEMA_ERRORS: ser_schema = core_schema.any_schema() - # TODO: pydantic-core currently doesn't accept it, but we should be able to provide an- # arbitrary core schema for serialization (currently `simple_ser_schema()` can be used,- # but only allow specifying a specific `'type`). Working around this with a wrap serializer- # is already done in other parts of the code base, so grep for similar cases when fixing:- schema['serialization'] = core_schema.wrap_serializer_function_ser_schema(- function=lambda v, h: h(v), schema=ser_schema- )+ schema['serialization'] = core_schema.schema_ser_schema(ser_schema) return schemapydantic/json_schema.pyindex e7abcad0..4b46e52a 100644@@ -2246,8 +2246,8 @@ class GenerateJsonSchema: elif schema_type == 'format' or schema_type == 'to-string': # FormatSerSchema or ToStringSerSchema return self.str_schema(core_schema.str_schema())- elif schema['type'] == 'model':- # ModelSerSchema+ elif schema_type == 'model' or schema_type == 'schema':+ # ModelSerSchema or SchemaSerSchema return self.generate_inner(schema['schema']) return None tests/test_types.pyindex d168c449..d215eb81 100644@@ -1,3 +1,4 @@+import collections import ipaddress import json import math@@ -1620,6 +1621,17 @@ def test_skip_validation_serialization(): assert A(x='abc').model_dump_json() == '{"x":"abcabc"}' +def test_skip_validation_serializes_with_original_schema() -> None:+ ta = TypeAdapter(SkipValidation[list[bytes]])++ # The original schema is used directly, without an intermediate wrap serializer:+ assert 'FunctionWrap' not in repr(ta.serializer)+ assert ta.dump_python([b'a', b'b'], mode='json') == ['a', 'b']+ assert ta.dump_json([b'a']) == b'["a"]'+ assert ta.dump_python([b'a', b'b'], include={1}) == [b'b']+ assert ta.json_schema(mode='serialization') == {'items': {'format': 'binary', 'type': 'string'}, 'type': 'array'}++ def test_skip_validation_json_schema(): class A(BaseModel): x: SkipValidation[int]@@ -1665,6 +1677,22 @@ def test_validate_as_known_serialization() -> None: assert m.model_dump_json() == '{"x":2}' +def test_validate_as_serialization_schema() -> None:+ def first_even(nums: list[int]) -> int:+ for i in nums:+ if i % 2 == 0:+ return i+ raise ValueError('Even not found.')++ ta = TypeAdapter(Annotated[int, ValidateAs(list[int], first_even)])++ assert 'FunctionWrap' not in repr(ta.serializer)+ assert ta.dump_json(ta.validate_python([1, 2])) == b'2'+ # Values are serialized using the schema of the annotated type (`int`), not `list[int]`:+ assert ta.json_schema(mode='validation') == {'items': {'type': 'integer'}, 'type': 'array'}+ assert ta.json_schema(mode='serialization') == {'type': 'integer'}++ def test_validate_as_unknown_serialization() -> None: class Arbitrary:@@ -1926,6 +1954,49 @@ def test_instanceof_serialization(): } +def test_instanceof_serializes_with_original_schema() -> None:+ ta = TypeAdapter(InstanceOf[list[bytes]])++ assert 'FunctionWrap' not in repr(ta.serializer)+ assert ta.dump_python([b'a', b'b'], mode='json') == ['a', 'b']+ assert ta.dump_json([b'a']) == b'["a"]'+ assert ta.dump_python([b'a', b'b'], exclude={0}) == [b'b']+ assert ta.json_schema(mode='serialization') == {'items': {'format': 'binary', 'type': 'string'}, 'type': 'array'}+++@pytest.mark.parametrize(+ 'mapping_type,value,json_value,values_json_schema',+ [+ (+ collections.OrderedDict[str, bytes],+ {'a': b'x', 'b': b'y'},+ {'a': 'x', 'b': 'y'},+ {'format': 'binary', 'type': 'string'},+ ),+ (+ collections.defaultdict[str, list[bytes]],+ {'a': [b'x'], 'b': [b'y']},+ {'a': ['x'], 'b': ['y']},+ {'items': {'format': 'binary', 'type': 'string'}, 'type': 'array'},+ ),+ ],+)+def test_non_dict_mapping_serializes_with_dict_schema(+ mapping_type: Any, value: Any, json_value: Any, values_json_schema: Any+) -> None:+ ta = TypeAdapter(mapping_type)+ validated = ta.validate_python(value)++ assert 'FunctionWrap' not in repr(ta.serializer)+ assert ta.dump_python(validated, mode='json') == json_value+ assert ta.dump_python(validated, exclude={'a'}) == {'b': value['b']}+ assert json.loads(ta.dump_json(validated)) == json_value+ assert ta.json_schema(mode='serialization') == {+ 'additionalProperties': values_json_schema,+ 'type': 'object',+ }++ def test_constraints_arbitrary_type() -> None: class CustomType: def __init__(self, v: Any) -> None:tests/test_validators.pyindex 7e9f4713..e84f1d67 100644@@ -2967,6 +2967,16 @@ def test_plain_validator_with_filter_dict_schema() -> None: assert Model(f={'a': 1, 'b': 1}).model_dump() == {'f': {'a': 1}} +def test_plain_validator_serializes_with_source_type_schema() -> None:+ ta = TypeAdapter(Annotated[list[bytes], PlainValidator(lambda v: [bytes(x) for x in v])])++ assert 'FunctionWrap' not in repr(ta.serializer)+ assert ta.dump_python(ta.validate_python([b'a', b'b']), mode='json') == ['a', 'b']+ assert ta.dump_json([b'a']) == b'["a"]'+ assert ta.dump_python([b'a', b'b'], include={0}) == [b'a']+ assert ta.json_schema(mode='serialization') == {'items': {'format': 'binary', 'type': 'string'}, 'type': 'array'}++ def test_plain_validator_with_unsupported_type() -> None: class UnsupportedClass: pass
pydantic-core/python/pydantic_core/core_schema.pyindex cb87ad5ae..9e847a5ad 100644@@ -12,7 +12,7 @@ from datetime import date, datetime, time, timedelta from decimal import Decimal from fractions import Fraction from re import Pattern-from typing import TYPE_CHECKING, Any, Literal+from typing import TYPE_CHECKING, Any, Literal, Union from typing_extensions import TypeVar, deprecated @@ -278,6 +278,9 @@ def simple_ser_schema(type: ExpectedSerializationTypes) -> SimpleSerSchema: """ Returns a schema for serialization with a custom type. + Note that any core schema can be used as a serialization schema, e.g. `int_schema()`+ is equivalent to `simple_ser_schema('int')`.+ Args: type: The type to use for serialization """@@ -469,6 +472,9 @@ def model_ser_schema(cls: type[Any], schema: CoreSchema) -> ModelSerSchema: """ Returns a schema for serialization using a model. + Note that any core schema can be used as a serialization schema, e.g. `model_schema()`+ can be used instead.+ Args: cls: The expected class type, used to generate warnings if the wrong type is passed schema: Internal schema to use to serialize the model dict@@ -476,14 +482,9 @@ def model_ser_schema(cls: type[Any], schema: CoreSchema) -> ModelSerSchema: return ModelSerSchema(type='model', cls=cls, schema=schema) -SerSchema: TypeAlias = (- SimpleSerSchema- | PlainSerializerFunctionSerSchema- | WrapSerializerFunctionSerSchema- | FormatSerSchema- | ToStringSerSchema- | ModelSerSchema-)+SerSchema: TypeAlias = 'SimpleSerSchema | PlainSerializerFunctionSerSchema | WrapSerializerFunctionSerSchema | FormatSerSchema | ToStringSerSchema | ModelSerSchema | CoreSchema'++"""The schemas that can be used as the `serialization` key of a core schema.""" class InvalidSchema(TypedDict, total=False):@@ -1626,7 +1627,7 @@ def filter_seq_schema(*, include: set[int] | None = None, exclude: set[int] | No return _dict_not_none(type='include-exclude-sequence', include=include, exclude=exclude) -IncExSeqOrElseSerSchema: TypeAlias = IncExSeqSerSchema | SerSchema+IncExSeqOrElseSerSchema: TypeAlias = Union[IncExSeqSerSchema, SerSchema] # noqa: UP007 (TypeError when evaluating) class ListSchema(TypedDict, total=False):@@ -2032,7 +2033,7 @@ def filter_dict_schema(*, include: IncExDict | None = None, exclude: IncExDict | return _dict_not_none(type='include-exclude-dict', include=include, exclude=exclude) -IncExDictOrElseSerSchema: TypeAlias = IncExDictSerSchema | SerSchema+IncExDictOrElseSerSchema: TypeAlias = Union[IncExDictSerSchema, SerSchema] # noqa: UP007 (TypeError when evaluating) class DictSchema(TypedDict, total=False):pydantic-core/src/serializers/shared.rsindex baf2dcc07..5e398125e 100644@@ -206,10 +206,15 @@ impl CombinedSerializer { ) // if `schema.serialization.type` is None, fall back to `schema.type` | None => (),- Some(ser_type) => {- // otherwise if `schema.serialization.type` is defined, use that with `find_serializer`- // instead of `schema.type`. In this case it's an error if a serializer isn't found.- return Self::find_serializer(ser_type, &ser_schema, config, definitions);+ Some(_) => {+ // otherwise, `schema.serialization` is an arbitrary core schema (which includes the+ // simple `{'type': ...}` ser schemas), so build a serializer from it as if it was+ // the main schema (this ensures nested `serialization` schemas, prebuilt serializers+ // and polymorphic serialization are handled). In this case, it's an error if a+ // serializer isn't found.+ // Note that as a consequence, `function-plain`/`function-wrap` *validator* schemas can't+ // be used as `schema.serialization`, as they are interpreted as the function *ser* schemas.+ return Self::build(&ser_schema, config, definitions); } } }pydantic-core/tests/serializers/test_core_schema_as_ser_schema.pynew file mode 100644index 000000000..b1110316f@@ -0,0 +1,124 @@+"""Tests for arbitrary core schemas being used as the `serialization` schema."""++from typing import Any++import pytest++from pydantic_core import SchemaError, SchemaSerializer, core_schema++from ..conftest import plain_repr+++@pytest.mark.parametrize(+ ['ser_schema', 'value', 'expected_python', 'expected_json'],+ [+ # equivalent to `simple_ser_schema('list')` (i.e. it must still work):+ (core_schema.simple_ser_schema('list'), [1, 'a'], [1, 'a'], b'[1,"a"]'),+ (core_schema.list_schema(core_schema.int_schema()), [1, 2], [1, 2], b'[1,2]'),+ (+ core_schema.tuple_schema([core_schema.int_schema(), core_schema.str_schema()]),+ (1, 'a'),+ (1, 'a'),+ b'[1,"a"]',+ ),+ (+ core_schema.dict_schema(core_schema.str_schema(), core_schema.int_schema()),+ {'a': 1},+ {'a': 1},+ b'{"a":1}',+ ),+ (+ core_schema.typed_dict_schema({'a': core_schema.typed_dict_field(core_schema.int_schema())}),+ {'a': 1, 'b': 2},+ {'a': 1},+ b'{"a":1}',+ ),+ (+ core_schema.definitions_schema(+ core_schema.definition_reference_schema('list_of_int'),+ [core_schema.list_schema(core_schema.int_schema(), ref='list_of_int')],+ ),+ [1, 2],+ [1, 2],+ b'[1,2]',+ ),+ # `'function-before'`/`'function-after'` core schemas serialize using the inner schema:+ (+ core_schema.no_info_after_validator_function(lambda v: v, core_schema.int_schema()),+ 1,+ 1,+ b'1',+ ),+ ],+)+def test_core_schema_as_ser_schema(ser_schema: Any, value: Any, expected_python: Any, expected_json: bytes) -> None:+ s = SchemaSerializer(core_schema.any_schema(serialization=ser_schema))++ assert s.to_python(value) == expected_python+ assert s.to_json(value) == expected_json+++def test_core_schema_as_ser_schema_expected_type() -> None:+ s = SchemaSerializer(core_schema.any_schema(serialization=core_schema.list_schema(core_schema.int_schema())))++ with pytest.warns(UserWarning, match=r'Expected `int` - serialized value may not be as expected'):+ assert s.to_python(['a']) == ['a']+++def test_core_schema_as_ser_schema_nested_serialization() -> None:+ """The `serialization` schema of the core schema used as a `serialization` schema is taken into account."""+ s = SchemaSerializer(+ core_schema.any_schema(+ serialization=core_schema.int_schema(+ serialization=core_schema.plain_serializer_function_ser_schema(lambda v: v * 2),+ ),+ )+ )++ assert s.to_python(3) == 6+ assert s.to_json(3) == b'6'+++def test_core_schema_as_ser_schema_unknown_type() -> None:+ with pytest.raises(SchemaError, match=r'Unknown serialization schema type: `unknown`'):+ SchemaSerializer(core_schema.any_schema(serialization={'type': 'unknown'})) # pyright: ignore[reportArgumentType]+++def test_model_schema_as_ser_schema_polymorphic() -> None:+ class Model:+ __pydantic_complete__ = True++ class SubModel(Model):+ pass++ model_schema = core_schema.model_schema(+ Model,+ schema=core_schema.model_fields_schema({'x': core_schema.model_field(core_schema.int_schema())}),+ config=core_schema.CoreConfig(polymorphic_serialization=True),+ )+ Model.__pydantic_serializer__ = SchemaSerializer(model_schema) # pyright: ignore[reportAttributeAccessIssue]++ sub_model_schema = core_schema.model_schema(+ SubModel,+ schema=core_schema.model_fields_schema(+ {+ 'x': core_schema.model_field(core_schema.int_schema()),+ 'y': core_schema.model_field(core_schema.int_schema()),+ }+ ),+ )+ SubModel.__pydantic_serializer__ = SchemaSerializer(sub_model_schema) # pyright: ignore[reportAttributeAccessIssue]++ s = SchemaSerializer(core_schema.any_schema(serialization=model_schema))+ # Using the model schema as a serialization schema behaves the same as using it as the main schema+ # (in this case, the prebuilt serializer of `Model` is used, wrapped in a polymorphism trampoline):+ assert plain_repr(s) == plain_repr(SchemaSerializer(model_schema))+ assert plain_repr(s).startswith(+ 'SchemaSerializer(serializer=PolymorphismTrampoline(PolymorphismTrampoline{class:Py(0x'+ )+ assert 'serializer:Prebuilt(PrebuiltSerializer{' in plain_repr(s)++ sub_model = SubModel()+ sub_model.__dict__ = {'x': 1, 'y': 2}+ assert s.to_python(sub_model) == {'x': 1, 'y': 2}+ assert s.to_json(sub_model) == b'{"x":1,"y":2}'pydantic-core/tests/test_typing.pyindex 94164c689..c698ccc17 100644@@ -247,6 +247,11 @@ def test_ser_function_wrap(): ) +def test_ser_core_schema():+ s = SchemaSerializer(core_schema.any_schema(serialization=core_schema.list_schema(core_schema.int_schema())))+ assert s.to_python([1, 2]) == [1, 2]++ def test_error_details() -> None: # Test that the ErrorDetails type is correctly exported. def act_on_error_details(_: ErrorDetails) -> None:pydantic/_internal/_core_utils.pyindex 56ea79bfc..79a8588f1 100644@@ -53,6 +53,29 @@ def is_function_with_inner_schema( return schema['type'] in _FUNCTION_WITH_INNER_SCHEMA_TYPES +def as_ser_schema(schema: CoreSchema) -> core_schema.SerSchema:+ """Return a schema suitable for the `'serialization'` key of a core schema, from an arbitrary core schema.++ Any core schema can be used as a serialization schema, except `'function-plain'` and `'function-wrap'`+ schemas, as these types are already used by the plain and wrap serializer *function* schemas.+ For these, mimic what pydantic-core would do if they were used as the *main* schema:++ - if the function schema has a `'serialization'` schema, use it.+ - otherwise, a `'function-plain'` schema is serialized as `'any'`, and a `'function-wrap'` schema+ is serialized using its inner schema.++ Note that `schema` is expected to be a *core* schema (e.g. as returned by the schema generator),+ not an existing `'serialization'` schema.+ """+ if schema['type'] == 'function-plain' or schema['type'] == 'function-wrap':+ if (ser_schema := schema.get('serialization')) is not None:+ return ser_schema+ if schema['type'] == 'function-plain':+ return core_schema.any_schema()+ return as_ser_schema(schema['schema'])+ return schema++ def is_list_like_schema_with_items_schema( schema: CoreSchema, ) -> TypeGuard[core_schema.ListSchema | core_schema.SetSchema | core_schema.FrozenSetSchema]:pydantic/functional_validators.pyindex 24f39be22..339e89946 100644@@ -911,10 +911,10 @@ class ValidateAs: except GENERATE_SCHEMA_ERRORS: ser_schema = core_schema.any_schema() - # TODO: pydantic-core currently doesn't accept it, but we should be able to provide an- # arbitrary core schema for serialization (currently `simple_ser_schema()` can be used,- # but only allow specifying a specific `'type`). Working around this with a wrap serializer- # is already done in other parts of the code base, so grep for similar cases when fixing:+ # TODO: pydantic-core now accepts an arbitrary core schema for serialization, so the+ # wrap serializer workaround below can be replaced with+ # `schema['serialization'] = _core_utils.as_ser_schema(ser_schema)`.+ # The same workaround is used in other parts of the code base, so grep for similar cases when fixing: schema['serialization'] = core_schema.wrap_serializer_function_ser_schema( function=lambda v, h: h(v), schema=ser_schema )pydantic/json_schema.pyindex e7abcad07..ebcd0f523 100644@@ -2237,18 +2237,18 @@ class GenerateJsonSchema: Returns: The generated JSON schema. """- schema_type = schema['type']- if schema_type == 'function-plain' or schema_type == 'function-wrap':- # PlainSerializerFunctionSerSchema or WrapSerializerFunctionSerSchema- return_schema = schema.get('return_schema')- if return_schema is not None:- return self.generate_inner(return_schema)- elif schema_type == 'format' or schema_type == 'to-string':- # FormatSerSchema or ToStringSerSchema- return self.str_schema(core_schema.str_schema())- elif schema['type'] == 'model':- # ModelSerSchema- return self.generate_inner(schema['schema'])+ match schema:+ case {'type': 'function-plain' | 'function-wrap'}:+ # PlainSerializerFunctionSerSchema or WrapSerializerFunctionSerSchema+ return_schema = schema.get('return_schema')+ if return_schema is not None:+ return self.generate_inner(return_schema)+ case {'type': 'format' | 'to-string'}:+ # FormatSerSchema or ToStringSerSchema+ return self.str_schema(core_schema.str_schema())+ case {'type': 'model'}:+ # ModelSerSchema+ return self.generate_inner(schema['schema']) return None def complex_schema(self, schema: core_schema.ComplexSchema) -> JsonSchemaValue:@@ -3016,6 +3016,6 @@ def _get_ser_schema_for_default_value(schema: CoreSchema) -> core_schema.PlainSe and ser_schema['type'] == 'function-plain' and not ser_schema.get('info_arg') ):- return ser_schema+ return cast('core_schema.PlainSerializerFunctionSerSchema', ser_schema) if _core_utils.is_function_with_inner_schema(schema): return _get_ser_schema_for_default_value(schema['schema'])tests/test_internal.pyindex acd0b46e7..c3912bc7e 100644@@ -16,6 +16,7 @@ from pydantic_core import core_schema as cs from pydantic import BaseModel, TypeAdapter from pydantic._internal._config import ConfigWrapper from pydantic._internal._core_metadata import update_core_metadata+from pydantic._internal._core_utils import as_ser_schema from pydantic._internal._fields import resolve_default_value from pydantic._internal._generate_schema import GenerateSchema from pydantic._internal._repr import Representation@@ -288,3 +289,58 @@ def test_resolve_default_value_missing_validated_data(): validated_data=None, call_default_factory=True, )+++def _identity(v: Any) -> Any:+ return v+++def _wrap_identity(v: Any, handler: Any) -> Any:+ return handler(v)+++@pytest.mark.parametrize(+ ['schema', 'expected'],+ [+ # Any non-function schema is returned as is:+ (cs.int_schema(), cs.int_schema()),+ (+ cs.no_info_after_validator_function(_identity, cs.int_schema()),+ cs.no_info_after_validator_function(_identity, cs.int_schema()),+ ),+ # The `'serialization'` schema of a plain/wrap function schema takes precedence:+ (+ cs.no_info_plain_validator_function(_identity, serialization=cs.simple_ser_schema('str')),+ cs.simple_ser_schema('str'),+ ),+ (+ cs.no_info_wrap_validator_function(+ _wrap_identity, cs.int_schema(), serialization=cs.simple_ser_schema('str')+ ),+ cs.simple_ser_schema('str'),+ ),+ # Otherwise, plain function schemas are serialized as `'any'`, wrap function schemas using the inner schema:+ (cs.no_info_plain_validator_function(_identity), cs.any_schema()),+ (cs.no_info_wrap_validator_function(_wrap_identity, cs.int_schema()), cs.int_schema()),+ # Nested plain/wrap function schemas are unwrapped:+ (+ cs.no_info_wrap_validator_function(+ _wrap_identity, cs.no_info_wrap_validator_function(_wrap_identity, cs.int_schema())+ ),+ cs.int_schema(),+ ),+ (+ cs.no_info_wrap_validator_function(_wrap_identity, cs.no_info_plain_validator_function(_identity)),+ cs.any_schema(),+ ),+ (+ cs.no_info_wrap_validator_function(+ _wrap_identity,+ cs.no_info_plain_validator_function(_identity, serialization=cs.simple_ser_schema('str')),+ ),+ cs.simple_ser_schema('str'),+ ),+ ],+)+def test_as_ser_schema(schema: CoreSchema, expected: cs.SerSchema) -> None:+ assert as_ser_schema(schema) == expected