Backtestspydanticrun-20261009-110034#13579
Allow arbitrary core schemas for serialization
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Patch B is the better fix on correctness, tests and craft.
**Correctness.** Patch B adds an explicit `SchemaSerSchema` (`{'type': 'schema', 'schema': CoreSchema}`) with a `schema_ser_schema()` helper. Its Rust handling is small and unambiguous: it builds the inner schema as a full serializer and gives a clear error when `schema` is missing. B then does what the issue asks:
- It replaces the identity wrap-serializer workaround everywhere it appears: `ValidateAs`, `PlainValidator`, `InstanceOf`, `SkipValidation`, and the mapping lax_or_strict case in `_generate_schema`.
- It updates JSON-schema generation (`ser_schema` and `_get_ser_schema_for_default_value`) so the new type is reflected in serialization JSON schemas and default serialization.
Patch A takes a different route: every non-special `serialization` dict is treated as a core schema and passed to `Self::build`. This creates a namespace collision. `function-plain`/`function-wrap` validator schemas cannot be used, because those type names are already taken by the ser-function schemas, and A admits this in a comment. `model` also overlaps with `ModelSerSchema`. A tries to paper over the collision with a Python-side `as_ser_schema` helper, but never uses it. The `ValidateAs` workaround stays in place with only the TODO reworded.
A also leaves JSON-schema generation unaware of core schemas used as serialization schemas: `ser_schema()` returns None for a `list_schema`. And it declares `SerSchema` as a string forward-reference alias, which forces `Union` workarounds and casts elsewhere. A's switch from `find_serializer` to `build` for every existing ser type also changes behaviour broadly, which adds risk.
**Tests.** Both add solid pydantic-core tests. B also adds end-to-end pydantic tests (ValidateAs, SkipValidation, InstanceOf, PlainValidator, Counter/defaultdict/OrderedDict). These cover serialization, include/exclude, and serialization JSON schema through the code paths that actually changed. A's pydantic-side test only exercises an unused helper.
**Overall.** B resolves the gap cleanly, removes the workarounds the issue points to, and keeps JSON schema consistent. A leaves the issue's concrete workaround in place and has an ambiguous design.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
17m 1s
Wall-clock time in the container
Full test suite
No new failures
12,390 passed · 500 skipped · 12,890 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
322vs 278 merged
Changed lines, ours vs merged
The issue
View on GitHubAs per the TODO in https://github.com/pydantic/pydantic/pull/13487:
There is currently no way to tell pydantic-core "serialize this value using this core schema". SerSchema is:
SerSchema = Union[
SimpleSerSchema,
PlainSerializerFunctionSerSchema,
WrapSerializerFunctionSerSchema,
FormatSerSchema,
ToStringSerSchema,
ModelSerSchema,
]
The only member that expresses "use this schema" is SimpleSerSchema, and it does not take a schema at all, only a name from a closed literal:
ExpectedSerializationTypes = Literal[
'none', 'int', 'bool', 'float', 'str', 'bytes', 'bytearray', 'list', 'tuple',
'set', 'frozenset', 'generator', 'dict', 'datetime', 'date', 'time', 'timedelta',
'url', 'multi-host-url', 'json', 'uuid', 'any',
]
class SimpleSerSchema(TypedDict, total=False):
type: Required[ExpectedSerializationTypes]
So an arbitrary schema (a model, a dataclass, list[int], a TypedDict, anything constructed by the schema generator) cannot be used directly. Every place in Pydantic that needs it works around the gap the same way: it wraps the schema in a wrap serializer whose function is the identity, purely so the schema can ride along in the schema argument.
From ValidateAs.__get_pydantic_core_schema__():
The two patches
pydantic-core/python/pydantic_core/core_schema.pyindex cb87ad5a..169f42d3 100644@@ -476,6 +476,34 @@ def model_ser_schema(cls: type[Any], schema: CoreSchema) -> ModelSerSchema: return ModelSerSchema(type='model', cls=cls, schema=schema) +class SchemaSerSchema(TypedDict, total=False):+ type: Required[Literal['schema']]+ schema: Required[CoreSchema]+++def schema_ser_schema(schema: CoreSchema) -> SchemaSerSchema:+ """+ Returns a schema for serialization using an arbitrary core schema.++ The serializer built from `schema` entirely replaces the serializer that would otherwise be+ built from the schema this serialization schema is attached to.++ ```py+ from pydantic_core import SchemaSerializer, core_schema++ list_of_ints = core_schema.list_schema(core_schema.int_schema())+ ser_schema = core_schema.schema_ser_schema(list_of_ints)+ ser = SchemaSerializer(core_schema.any_schema(serialization=ser_schema))+ assert ser.to_json([1, 2]) == b'[1,2]'+ assert ser.to_python([1, 2, 3], exclude={1}) == [1, 3]+ ```++ Args:+ schema: The core schema to use to serialize the value+ """+ return SchemaSerSchema(type='schema', schema=schema)++ SerSchema: TypeAlias = ( SimpleSerSchema | PlainSerializerFunctionSerSchema@@ -483,6 +511,7 @@ SerSchema: TypeAlias = ( | FormatSerSchema | ToStringSerSchema | ModelSerSchema+ | SchemaSerSchema ) pydantic-core/src/serializers/shared.rsindex baf2dcc0..27de0c10 100644@@ -198,6 +198,14 @@ impl CombinedSerializer { ) .map_err(|err| py_schema_error_type!("Error building `function-wrap` serializer:\n {err}")); }+ Some("schema") => {+ // `schema` means "serialize using this arbitrary core schema", the inner schema is built+ // as a standalone serializer and entirely replaces serialization of the outer schema+ return ser_schema+ .get_as_req::<Bound<'_, PyDict>>(intern!(py, "schema"))+ .and_then(|inner_schema| Self::build(&inner_schema, config, definitions))+ .map_err(|err| py_schema_error_type!("Error building `schema` serializer:\n {err}"));+ } Some( // applies to lists tuples and dicts, does not override the main schema `type` "include-exclude-sequence" | "include-exclude-dict"pydantic-core/tests/serializers/test_schema_ser.pynew file mode 100644index 00000000..33600197@@ -0,0 +1,130 @@+"""Tests for `core_schema.schema_ser_schema()`: serializing a value using an arbitrary core schema."""++import dataclasses++import pytest++from pydantic_core import SchemaError, SchemaSerializer, core_schema++from ..conftest import plain_repr+++def test_schema_ser_list_of_int():+ s = SchemaSerializer(+ core_schema.any_schema(+ serialization=core_schema.schema_ser_schema(core_schema.list_schema(core_schema.int_schema()))+ )+ )+ assert plain_repr(s).startswith('SchemaSerializer(serializer=List(ListSerializer')++ assert s.to_python([1, 2]) == [1, 2]+ assert s.to_python([1, 2], mode='json') == [1, 2]+ assert s.to_json([1, 2]) == b'[1,2]'+ # include/exclude are forwarded to the inner serializer+ assert s.to_python([1, 2, 3], include={0, 2}) == [1, 3]+ assert s.to_python([1, 2, 3], exclude={1}) == [1, 3]+++def test_schema_ser_overrides_outer_schema():+ s = SchemaSerializer(+ core_schema.int_schema(serialization=core_schema.schema_ser_schema(core_schema.str_schema()))+ )+ assert s.to_python('foo') == 'foo'+ with pytest.warns(UserWarning, match=r'Expected `str` - serialized value may not be as expected'):+ assert s.to_python(1) == 1+++def test_schema_ser_typed_dict():+ inner = core_schema.typed_dict_schema(+ {+ 'a': core_schema.typed_dict_field(core_schema.int_schema()),+ 'b': core_schema.typed_dict_field(core_schema.bytes_schema(), serialization_alias='B'),+ }+ )+ s = SchemaSerializer(core_schema.any_schema(serialization=core_schema.schema_ser_schema(inner)))+ assert s.to_python({'a': 1, 'b': b'x'}, by_alias=True) == {'a': 1, 'B': b'x'}+ assert s.to_json({'a': 1, 'b': b'x'}, by_alias=True) == b'{"a":1,"B":"x"}'+ assert s.to_python({'a': 1, 'b': b'x'}, exclude={'b'}) == {'a': 1}+++def test_schema_ser_model():+ class MyModel:+ __slots__ = '__dict__', '__pydantic_fields_set__', '__pydantic_extra__', '__pydantic_private__'++ model_schema = core_schema.model_schema(+ MyModel,+ core_schema.model_fields_schema(+ {+ 'x': core_schema.model_field(core_schema.int_schema()),+ 'y': core_schema.model_field(core_schema.str_schema(), serialization_alias='Y'),+ }+ ),+ )+ s = SchemaSerializer(core_schema.any_schema(serialization=core_schema.schema_ser_schema(model_schema)))++ m = MyModel()+ m.__dict__ = {'x': 1, 'y': 'a'}+ m.__pydantic_fields_set__ = {'x', 'y'}+ m.__pydantic_extra__ = None+ m.__pydantic_private__ = None++ assert s.to_python(m) == {'x': 1, 'y': 'a'}+ assert s.to_python(m, by_alias=True) == {'x': 1, 'Y': 'a'}+ assert s.to_json(m) == b'{"x":1,"y":"a"}'+++def test_schema_ser_dataclass():+ @dataclasses.dataclass+ class Foo:+ a: int+ b: str++ dc_schema = core_schema.dataclass_schema(+ Foo,+ core_schema.dataclass_args_schema(+ 'Foo',+ [+ core_schema.dataclass_field(name='a', schema=core_schema.int_schema()),+ core_schema.dataclass_field(name='b', schema=core_schema.str_schema()),+ ],+ ),+ ['a', 'b'],+ )+ s = SchemaSerializer(core_schema.any_schema(serialization=core_schema.schema_ser_schema(dc_schema)))+ assert s.to_python(Foo(a=1, b='x')) == {'a': 1, 'b': 'x'}+ assert s.to_json(Foo(a=1, b='x')) == b'{"a":1,"b":"x"}'+++def test_schema_ser_definition_ref():+ s = SchemaSerializer(+ core_schema.definitions_schema(+ core_schema.any_schema(+ serialization=core_schema.schema_ser_schema(+ core_schema.list_schema(core_schema.definition_reference_schema('Bytes'))+ )+ ),+ [core_schema.bytes_schema(ref='Bytes')],+ )+ )+ assert s.to_python([b'a'], mode='json') == ['a']+ assert s.to_json([b'a']) == b'["a"]'+++def test_schema_ser_nested_serialization():+ # the inner schema can itself have a custom serialization schema+ inner = core_schema.int_schema(+ serialization=core_schema.plain_serializer_function_ser_schema(lambda v: v * 2, when_used='always')+ )+ s = SchemaSerializer(core_schema.any_schema(serialization=core_schema.schema_ser_schema(inner)))+ assert s.to_python(2) == 4+ assert s.to_json(2) == b'4'+++def test_schema_ser_missing_schema():+ with pytest.raises(SchemaError):+ SchemaSerializer(core_schema.any_schema(serialization={'type': 'schema'}))+++def test_schema_ser_invalid_inner_schema():+ with pytest.raises(SchemaError):+ SchemaSerializer(core_schema.any_schema(serialization={'type': 'schema', 'schema': {'type': 'not-a-type'}}))pydantic/_internal/_generate_schema.pyindex 3933e821..21453d76 100644@@ -706,9 +706,7 @@ class GenerateSchema: schema = core_schema.lax_or_strict_schema( lax_schema=lax_schema, strict_schema=strict_schema,- serialization=core_schema.wrap_serializer_function_ser_schema(- lambda v, h: h(v), schema=dict_schema, info_arg=False- ),+ serialization=core_schema.schema_ser_schema(dict_schema), ) return schemapydantic/functional_validators.pyindex 24f39be2..6d8bca36 100644@@ -217,14 +217,7 @@ class PlainValidator: schema = handler(source_type) # TODO if `schema['serialization']` is one of `'include-exclude-dict/sequence', # schema validation will fail. That's why we use 'type ignore' comments below.- serialization = schema.get(- 'serialization',- core_schema.wrap_serializer_function_ser_schema(- function=lambda v, h: h(v),- schema=schema,- return_schema=handler.generate_schema(source_type),- ),- )+ serialization = schema.get('serialization', core_schema.schema_ser_schema(schema)) except PydanticSchemaGenerationError: serialization = None @@ -811,9 +804,7 @@ else: return instance_of_schema else: # Use the "original" approach to serialization- instance_of_schema['serialization'] = core_schema.wrap_serializer_function_ser_schema(- function=lambda v, h: h(v), schema=original_schema- )+ instance_of_schema['serialization'] = core_schema.schema_ser_schema(original_schema) return core_schema.json_or_python_schema(python_schema=instance_of_schema, json_schema=original_schema) __hash__ = object.__hash__@@ -847,9 +838,7 @@ else: metadata = {'pydantic_js_annotation_functions': [lambda _c, h: h(original_schema)]} return core_schema.any_schema( metadata=metadata,- serialization=core_schema.wrap_serializer_function_ser_schema(- function=lambda v, h: h(v), schema=original_schema- ),+ serialization=core_schema.schema_ser_schema(original_schema), ) __hash__ = object.__hash__@@ -910,12 +899,5 @@ class ValidateAs: ser_schema = handler(source) except GENERATE_SCHEMA_ERRORS: ser_schema = core_schema.any_schema()-- # TODO: pydantic-core currently doesn't accept it, but we should be able to provide an- # arbitrary core schema for serialization (currently `simple_ser_schema()` can be used,- # but only allow specifying a specific `'type`). Working around this with a wrap serializer- # is already done in other parts of the code base, so grep for similar cases when fixing:- schema['serialization'] = core_schema.wrap_serializer_function_ser_schema(- function=lambda v, h: h(v), schema=ser_schema- )+ schema['serialization'] = core_schema.schema_ser_schema(ser_schema) return schemapydantic/json_schema.pyindex e7abcad0..dfb5167a 100644@@ -2246,8 +2246,8 @@ class GenerateJsonSchema: elif schema_type == 'format' or schema_type == 'to-string': # FormatSerSchema or ToStringSerSchema return self.str_schema(core_schema.str_schema())- elif schema['type'] == 'model':- # ModelSerSchema+ elif schema['type'] == 'model' or schema['type'] == 'schema':+ # ModelSerSchema or SchemaSerSchema return self.generate_inner(schema['schema']) return None @@ -3011,11 +3011,10 @@ def _get_ser_schema_for_default_value(schema: CoreSchema) -> core_schema.PlainSe This takes into account having the serialization schema nested under validation schema(s). """- if (- (ser_schema := schema.get('serialization'))- and ser_schema['type'] == 'function-plain'- and not ser_schema.get('info_arg')- ):+ if (ser_schema := schema.get('serialization')) and ser_schema['type'] == 'schema':+ # The value is serialized using an arbitrary core schema:+ return _get_ser_schema_for_default_value(ser_schema['schema'])+ if ser_schema and ser_schema['type'] == 'function-plain' and not ser_schema.get('info_arg'): return ser_schema if _core_utils.is_function_with_inner_schema(schema): return _get_ser_schema_for_default_value(schema['schema'])tests/test_types.pyindex d168c449..6d9ae475 100644@@ -1,3 +1,4 @@+import collections import ipaddress import json import math@@ -1679,6 +1680,67 @@ def test_validate_as_unknown_serialization() -> None: assert ta.dump_python(Arbitrary(1)) == Arbitrary(1) +def test_validate_as_serialization_json_schema() -> None:+ """The serialization JSON Schema of `ValidateAs` reflects the annotated type, not the type validated from."""++ def first_even(nums: list[int]) -> int:+ return next(i for i in nums if i % 2 == 0)++ ta = TypeAdapter(Annotated[int, ValidateAs(list[int], first_even)])++ assert ta.json_schema(mode='validation') == {'items': {'type': 'integer'}, 'type': 'array'}+ assert ta.json_schema(mode='serialization') == {'type': 'integer'}+ assert ta.dump_json(ta.validate_python([1, 4])) == b'4'+++def test_validate_as_model_serialization() -> None:+ class Inner(BaseModel):+ a: int+ b: bytes++ class Sub(Inner):+ c: int++ ta = TypeAdapter(Annotated[Inner, ValidateAs(dict[str, Any], lambda d: Sub(**d))])++ assert ta.dump_python(ta.validate_python({'a': 1, 'b': b'x', 'c': 2})) == {'a': 1, 'b': b'x'}+ assert ta.dump_json(Inner(a=1, b=b'x')) == b'{"a":1,"b":"x"}'+ assert ta.dump_python(Inner(a=1, b=b'x'), exclude={'b'}) == {'a': 1}+++def test_skip_validation_serialization_uses_type() -> None:+ class Inner(BaseModel):+ x: int++ class Model(BaseModel):+ a: SkipValidation[tuple[int, ...]]+ b: SkipValidation[Inner]++ m = Model(a=(1, 2), b=Inner(x=1))+ assert m.model_dump() == {'a': (1, 2), 'b': {'x': 1}}+ assert m.model_dump(include={'a': {0}, 'b': True}) == {'a': (1,), 'b': {'x': 1}}+ assert m.model_dump_json() == '{"a":[1,2],"b":{"x":1}}'+ assert Model.model_json_schema(mode='serialization')['properties']['a'] == {+ 'items': {'type': 'integer'},+ 'title': 'A',+ 'type': 'array',+ }+++def test_skip_validation_serialization_json_schema_default() -> None:+ """Defaults are serialized in the serialization JSON Schema using the serializer of the skipped type."""++ class Model(BaseModel):+ a: SkipValidation[Annotated[int, PlainSerializer(str)]] = 1++ assert Model().model_dump() == {'a': '1'}+ assert Model.model_json_schema(mode='serialization')['properties']['a'] == {+ 'default': '1',+ 'title': 'A',+ 'type': 'string',+ }++ @pytest.mark.skipif(sys.version_info < (3, 12), reason="`Annotated` doesn't allow instances in <3.12") def test_skip_validation_arbitrary_type_object() -> None: """https://github.com/pydantic/pydantic/issues/11997.@@ -1926,6 +1988,25 @@ def test_instanceof_serialization(): } +def test_instanceof_serialization_uses_type() -> None:+ ta = TypeAdapter(InstanceOf[list[bytes]])++ assert ta.dump_python([b'a']) == [b'a']+ assert ta.dump_python([b'a'], mode='json') == ['a']+ assert ta.dump_json([b'a', b'b']) == b'["a","b"]'+ assert ta.dump_python([b'a', b'b'], exclude={0}) == [b'b']+ assert ta.json_schema(mode='serialization') == {'items': {'format': 'binary', 'type': 'string'}, 'type': 'array'}+++def test_plain_validator_serialization_uses_type() -> None:+ ta = TypeAdapter(Annotated[list[bytes], PlainValidator(lambda v: v)])++ assert ta.dump_python(ta.validate_python([b'a'])) == [b'a']+ assert ta.dump_json([b'a', b'b']) == b'["a","b"]'+ assert ta.dump_python([b'a', b'b'], exclude={0}) == [b'b']+ assert ta.json_schema(mode='serialization') == {'items': {'format': 'binary', 'type': 'string'}, 'type': 'array'}++ def test_constraints_arbitrary_type() -> None: class CustomType: def __init__(self, v: Any) -> None:@@ -2424,3 +2505,34 @@ def test_custom_serializer_override_secret_str() -> None: u = User(name='sam', password='hi') assert u.model_dump()['password'] == 'secret: **********'+++@pytest.mark.parametrize(+ 'tp,value',+ [+ (collections.Counter[str], collections.Counter(a=1, b=2)),+ (collections.defaultdict[str, int], collections.defaultdict(int, a=1, b=2)),+ ],+)+def test_mapping_subclass_serialization(tp: Any, value: Any) -> None:+ ta = TypeAdapter(tp)++ assert ta.dump_python(value) == {'a': 1, 'b': 2}+ assert type(ta.dump_python(value)) is dict+ assert ta.dump_json(value) == b'{"a":1,"b":2}'+ assert ta.dump_python(value, exclude={'a'}) == {'b': 2}+ assert ta.json_schema(mode='serialization') == {'additionalProperties': {'type': 'integer'}, 'type': 'object'}+++def test_ordered_dict_serialization() -> None:+ ta = TypeAdapter(collections.OrderedDict[str, bytes])+ value = collections.OrderedDict(a=b'x', b=b'y')++ assert ta.dump_python(value) == {'a': b'x', 'b': b'y'}+ assert ta.dump_python(value, mode='json') == {'a': 'x', 'b': 'y'}+ assert ta.dump_json(value) == b'{"a":"x","b":"y"}'+ assert ta.dump_python(value, exclude={'a'}) == {'b': b'y'}+ assert ta.json_schema(mode='serialization') == {+ 'additionalProperties': {'format': 'binary', 'type': 'string'},+ 'type': 'object',+ }
pydantic-core/python/pydantic_core/core_schema.pyindex cb87ad5ae..9e847a5ad 100644@@ -12,7 +12,7 @@ from datetime import date, datetime, time, timedelta from decimal import Decimal from fractions import Fraction from re import Pattern-from typing import TYPE_CHECKING, Any, Literal+from typing import TYPE_CHECKING, Any, Literal, Union from typing_extensions import TypeVar, deprecated @@ -278,6 +278,9 @@ def simple_ser_schema(type: ExpectedSerializationTypes) -> SimpleSerSchema: """ Returns a schema for serialization with a custom type. + Note that any core schema can be used as a serialization schema, e.g. `int_schema()`+ is equivalent to `simple_ser_schema('int')`.+ Args: type: The type to use for serialization """@@ -469,6 +472,9 @@ def model_ser_schema(cls: type[Any], schema: CoreSchema) -> ModelSerSchema: """ Returns a schema for serialization using a model. + Note that any core schema can be used as a serialization schema, e.g. `model_schema()`+ can be used instead.+ Args: cls: The expected class type, used to generate warnings if the wrong type is passed schema: Internal schema to use to serialize the model dict@@ -476,14 +482,9 @@ def model_ser_schema(cls: type[Any], schema: CoreSchema) -> ModelSerSchema: return ModelSerSchema(type='model', cls=cls, schema=schema) -SerSchema: TypeAlias = (- SimpleSerSchema- | PlainSerializerFunctionSerSchema- | WrapSerializerFunctionSerSchema- | FormatSerSchema- | ToStringSerSchema- | ModelSerSchema-)+SerSchema: TypeAlias = 'SimpleSerSchema | PlainSerializerFunctionSerSchema | WrapSerializerFunctionSerSchema | FormatSerSchema | ToStringSerSchema | ModelSerSchema | CoreSchema'++"""The schemas that can be used as the `serialization` key of a core schema.""" class InvalidSchema(TypedDict, total=False):@@ -1626,7 +1627,7 @@ def filter_seq_schema(*, include: set[int] | None = None, exclude: set[int] | No return _dict_not_none(type='include-exclude-sequence', include=include, exclude=exclude) -IncExSeqOrElseSerSchema: TypeAlias = IncExSeqSerSchema | SerSchema+IncExSeqOrElseSerSchema: TypeAlias = Union[IncExSeqSerSchema, SerSchema] # noqa: UP007 (TypeError when evaluating) class ListSchema(TypedDict, total=False):@@ -2032,7 +2033,7 @@ def filter_dict_schema(*, include: IncExDict | None = None, exclude: IncExDict | return _dict_not_none(type='include-exclude-dict', include=include, exclude=exclude) -IncExDictOrElseSerSchema: TypeAlias = IncExDictSerSchema | SerSchema+IncExDictOrElseSerSchema: TypeAlias = Union[IncExDictSerSchema, SerSchema] # noqa: UP007 (TypeError when evaluating) class DictSchema(TypedDict, total=False):pydantic-core/src/serializers/shared.rsindex baf2dcc07..5e398125e 100644@@ -206,10 +206,15 @@ impl CombinedSerializer { ) // if `schema.serialization.type` is None, fall back to `schema.type` | None => (),- Some(ser_type) => {- // otherwise if `schema.serialization.type` is defined, use that with `find_serializer`- // instead of `schema.type`. In this case it's an error if a serializer isn't found.- return Self::find_serializer(ser_type, &ser_schema, config, definitions);+ Some(_) => {+ // otherwise, `schema.serialization` is an arbitrary core schema (which includes the+ // simple `{'type': ...}` ser schemas), so build a serializer from it as if it was+ // the main schema (this ensures nested `serialization` schemas, prebuilt serializers+ // and polymorphic serialization are handled). In this case, it's an error if a+ // serializer isn't found.+ // Note that as a consequence, `function-plain`/`function-wrap` *validator* schemas can't+ // be used as `schema.serialization`, as they are interpreted as the function *ser* schemas.+ return Self::build(&ser_schema, config, definitions); } } }pydantic-core/tests/serializers/test_core_schema_as_ser_schema.pynew file mode 100644index 000000000..b1110316f@@ -0,0 +1,124 @@+"""Tests for arbitrary core schemas being used as the `serialization` schema."""++from typing import Any++import pytest++from pydantic_core import SchemaError, SchemaSerializer, core_schema++from ..conftest import plain_repr+++@pytest.mark.parametrize(+ ['ser_schema', 'value', 'expected_python', 'expected_json'],+ [+ # equivalent to `simple_ser_schema('list')` (i.e. it must still work):+ (core_schema.simple_ser_schema('list'), [1, 'a'], [1, 'a'], b'[1,"a"]'),+ (core_schema.list_schema(core_schema.int_schema()), [1, 2], [1, 2], b'[1,2]'),+ (+ core_schema.tuple_schema([core_schema.int_schema(), core_schema.str_schema()]),+ (1, 'a'),+ (1, 'a'),+ b'[1,"a"]',+ ),+ (+ core_schema.dict_schema(core_schema.str_schema(), core_schema.int_schema()),+ {'a': 1},+ {'a': 1},+ b'{"a":1}',+ ),+ (+ core_schema.typed_dict_schema({'a': core_schema.typed_dict_field(core_schema.int_schema())}),+ {'a': 1, 'b': 2},+ {'a': 1},+ b'{"a":1}',+ ),+ (+ core_schema.definitions_schema(+ core_schema.definition_reference_schema('list_of_int'),+ [core_schema.list_schema(core_schema.int_schema(), ref='list_of_int')],+ ),+ [1, 2],+ [1, 2],+ b'[1,2]',+ ),+ # `'function-before'`/`'function-after'` core schemas serialize using the inner schema:+ (+ core_schema.no_info_after_validator_function(lambda v: v, core_schema.int_schema()),+ 1,+ 1,+ b'1',+ ),+ ],+)+def test_core_schema_as_ser_schema(ser_schema: Any, value: Any, expected_python: Any, expected_json: bytes) -> None:+ s = SchemaSerializer(core_schema.any_schema(serialization=ser_schema))++ assert s.to_python(value) == expected_python+ assert s.to_json(value) == expected_json+++def test_core_schema_as_ser_schema_expected_type() -> None:+ s = SchemaSerializer(core_schema.any_schema(serialization=core_schema.list_schema(core_schema.int_schema())))++ with pytest.warns(UserWarning, match=r'Expected `int` - serialized value may not be as expected'):+ assert s.to_python(['a']) == ['a']+++def test_core_schema_as_ser_schema_nested_serialization() -> None:+ """The `serialization` schema of the core schema used as a `serialization` schema is taken into account."""+ s = SchemaSerializer(+ core_schema.any_schema(+ serialization=core_schema.int_schema(+ serialization=core_schema.plain_serializer_function_ser_schema(lambda v: v * 2),+ ),+ )+ )++ assert s.to_python(3) == 6+ assert s.to_json(3) == b'6'+++def test_core_schema_as_ser_schema_unknown_type() -> None:+ with pytest.raises(SchemaError, match=r'Unknown serialization schema type: `unknown`'):+ SchemaSerializer(core_schema.any_schema(serialization={'type': 'unknown'})) # pyright: ignore[reportArgumentType]+++def test_model_schema_as_ser_schema_polymorphic() -> None:+ class Model:+ __pydantic_complete__ = True++ class SubModel(Model):+ pass++ model_schema = core_schema.model_schema(+ Model,+ schema=core_schema.model_fields_schema({'x': core_schema.model_field(core_schema.int_schema())}),+ config=core_schema.CoreConfig(polymorphic_serialization=True),+ )+ Model.__pydantic_serializer__ = SchemaSerializer(model_schema) # pyright: ignore[reportAttributeAccessIssue]++ sub_model_schema = core_schema.model_schema(+ SubModel,+ schema=core_schema.model_fields_schema(+ {+ 'x': core_schema.model_field(core_schema.int_schema()),+ 'y': core_schema.model_field(core_schema.int_schema()),+ }+ ),+ )+ SubModel.__pydantic_serializer__ = SchemaSerializer(sub_model_schema) # pyright: ignore[reportAttributeAccessIssue]++ s = SchemaSerializer(core_schema.any_schema(serialization=model_schema))+ # Using the model schema as a serialization schema behaves the same as using it as the main schema+ # (in this case, the prebuilt serializer of `Model` is used, wrapped in a polymorphism trampoline):+ assert plain_repr(s) == plain_repr(SchemaSerializer(model_schema))+ assert plain_repr(s).startswith(+ 'SchemaSerializer(serializer=PolymorphismTrampoline(PolymorphismTrampoline{class:Py(0x'+ )+ assert 'serializer:Prebuilt(PrebuiltSerializer{' in plain_repr(s)++ sub_model = SubModel()+ sub_model.__dict__ = {'x': 1, 'y': 2}+ assert s.to_python(sub_model) == {'x': 1, 'y': 2}+ assert s.to_json(sub_model) == b'{"x":1,"y":2}'pydantic-core/tests/test_typing.pyindex 94164c689..c698ccc17 100644@@ -247,6 +247,11 @@ def test_ser_function_wrap(): ) +def test_ser_core_schema():+ s = SchemaSerializer(core_schema.any_schema(serialization=core_schema.list_schema(core_schema.int_schema())))+ assert s.to_python([1, 2]) == [1, 2]++ def test_error_details() -> None: # Test that the ErrorDetails type is correctly exported. def act_on_error_details(_: ErrorDetails) -> None:pydantic/_internal/_core_utils.pyindex 56ea79bfc..79a8588f1 100644@@ -53,6 +53,29 @@ def is_function_with_inner_schema( return schema['type'] in _FUNCTION_WITH_INNER_SCHEMA_TYPES +def as_ser_schema(schema: CoreSchema) -> core_schema.SerSchema:+ """Return a schema suitable for the `'serialization'` key of a core schema, from an arbitrary core schema.++ Any core schema can be used as a serialization schema, except `'function-plain'` and `'function-wrap'`+ schemas, as these types are already used by the plain and wrap serializer *function* schemas.+ For these, mimic what pydantic-core would do if they were used as the *main* schema:++ - if the function schema has a `'serialization'` schema, use it.+ - otherwise, a `'function-plain'` schema is serialized as `'any'`, and a `'function-wrap'` schema+ is serialized using its inner schema.++ Note that `schema` is expected to be a *core* schema (e.g. as returned by the schema generator),+ not an existing `'serialization'` schema.+ """+ if schema['type'] == 'function-plain' or schema['type'] == 'function-wrap':+ if (ser_schema := schema.get('serialization')) is not None:+ return ser_schema+ if schema['type'] == 'function-plain':+ return core_schema.any_schema()+ return as_ser_schema(schema['schema'])+ return schema++ def is_list_like_schema_with_items_schema( schema: CoreSchema, ) -> TypeGuard[core_schema.ListSchema | core_schema.SetSchema | core_schema.FrozenSetSchema]:pydantic/functional_validators.pyindex 24f39be22..339e89946 100644@@ -911,10 +911,10 @@ class ValidateAs: except GENERATE_SCHEMA_ERRORS: ser_schema = core_schema.any_schema() - # TODO: pydantic-core currently doesn't accept it, but we should be able to provide an- # arbitrary core schema for serialization (currently `simple_ser_schema()` can be used,- # but only allow specifying a specific `'type`). Working around this with a wrap serializer- # is already done in other parts of the code base, so grep for similar cases when fixing:+ # TODO: pydantic-core now accepts an arbitrary core schema for serialization, so the+ # wrap serializer workaround below can be replaced with+ # `schema['serialization'] = _core_utils.as_ser_schema(ser_schema)`.+ # The same workaround is used in other parts of the code base, so grep for similar cases when fixing: schema['serialization'] = core_schema.wrap_serializer_function_ser_schema( function=lambda v, h: h(v), schema=ser_schema )pydantic/json_schema.pyindex e7abcad07..ebcd0f523 100644@@ -2237,18 +2237,18 @@ class GenerateJsonSchema: Returns: The generated JSON schema. """- schema_type = schema['type']- if schema_type == 'function-plain' or schema_type == 'function-wrap':- # PlainSerializerFunctionSerSchema or WrapSerializerFunctionSerSchema- return_schema = schema.get('return_schema')- if return_schema is not None:- return self.generate_inner(return_schema)- elif schema_type == 'format' or schema_type == 'to-string':- # FormatSerSchema or ToStringSerSchema- return self.str_schema(core_schema.str_schema())- elif schema['type'] == 'model':- # ModelSerSchema- return self.generate_inner(schema['schema'])+ match schema:+ case {'type': 'function-plain' | 'function-wrap'}:+ # PlainSerializerFunctionSerSchema or WrapSerializerFunctionSerSchema+ return_schema = schema.get('return_schema')+ if return_schema is not None:+ return self.generate_inner(return_schema)+ case {'type': 'format' | 'to-string'}:+ # FormatSerSchema or ToStringSerSchema+ return self.str_schema(core_schema.str_schema())+ case {'type': 'model'}:+ # ModelSerSchema+ return self.generate_inner(schema['schema']) return None def complex_schema(self, schema: core_schema.ComplexSchema) -> JsonSchemaValue:@@ -3016,6 +3016,6 @@ def _get_ser_schema_for_default_value(schema: CoreSchema) -> core_schema.PlainSe and ser_schema['type'] == 'function-plain' and not ser_schema.get('info_arg') ):- return ser_schema+ return cast('core_schema.PlainSerializerFunctionSerSchema', ser_schema) if _core_utils.is_function_with_inner_schema(schema): return _get_ser_schema_for_default_value(schema['schema'])tests/test_internal.pyindex acd0b46e7..c3912bc7e 100644@@ -16,6 +16,7 @@ from pydantic_core import core_schema as cs from pydantic import BaseModel, TypeAdapter from pydantic._internal._config import ConfigWrapper from pydantic._internal._core_metadata import update_core_metadata+from pydantic._internal._core_utils import as_ser_schema from pydantic._internal._fields import resolve_default_value from pydantic._internal._generate_schema import GenerateSchema from pydantic._internal._repr import Representation@@ -288,3 +289,58 @@ def test_resolve_default_value_missing_validated_data(): validated_data=None, call_default_factory=True, )+++def _identity(v: Any) -> Any:+ return v+++def _wrap_identity(v: Any, handler: Any) -> Any:+ return handler(v)+++@pytest.mark.parametrize(+ ['schema', 'expected'],+ [+ # Any non-function schema is returned as is:+ (cs.int_schema(), cs.int_schema()),+ (+ cs.no_info_after_validator_function(_identity, cs.int_schema()),+ cs.no_info_after_validator_function(_identity, cs.int_schema()),+ ),+ # The `'serialization'` schema of a plain/wrap function schema takes precedence:+ (+ cs.no_info_plain_validator_function(_identity, serialization=cs.simple_ser_schema('str')),+ cs.simple_ser_schema('str'),+ ),+ (+ cs.no_info_wrap_validator_function(+ _wrap_identity, cs.int_schema(), serialization=cs.simple_ser_schema('str')+ ),+ cs.simple_ser_schema('str'),+ ),+ # Otherwise, plain function schemas are serialized as `'any'`, wrap function schemas using the inner schema:+ (cs.no_info_plain_validator_function(_identity), cs.any_schema()),+ (cs.no_info_wrap_validator_function(_wrap_identity, cs.int_schema()), cs.int_schema()),+ # Nested plain/wrap function schemas are unwrapped:+ (+ cs.no_info_wrap_validator_function(+ _wrap_identity, cs.no_info_wrap_validator_function(_wrap_identity, cs.int_schema())+ ),+ cs.int_schema(),+ ),+ (+ cs.no_info_wrap_validator_function(_wrap_identity, cs.no_info_plain_validator_function(_identity)),+ cs.any_schema(),+ ),+ (+ cs.no_info_wrap_validator_function(+ _wrap_identity,+ cs.no_info_plain_validator_function(_identity, serialization=cs.simple_ser_schema('str')),+ ),+ cs.simple_ser_schema('str'),+ ),+ ],+)+def test_as_ser_schema(schema: CoreSchema, expected: cs.SerSchema) -> None:+ assert as_ser_schema(schema) == expected