Backtestspydanticrun-20261007-130130#13579
Allow arbitrary core schemas for serialization
Our patch was judged better than the merged fix.
In this judgment, Patch A = our agent, Patch B = the merged fix.
Patch A fixes the issue directly. It adds an explicit `SchemaSerSchema` (`{'type': 'schema', 'schema': CoreSchema}`) with a `schema_ser_schema()` helper and registers it in `SerSchema`. It also adds it to the self-schema in `self_schema.py`, so schema validation accepts it. In Rust it builds a full serializer from the inner schema via `CombinedSerializer::build`, so nested `serialization` keys, definitions and polymorphism are respected. It then does what the issue's TODO asks: it replaces the identity wrap-serializer workarounds in `ValidateAs`, `SkipValidation`, `InstanceOf`, `PlainValidator` and the OrderedDict/defaultdict mapping schema. It also teaches `GenerateJsonSchema.ser_schema` about the new type, so serialization-mode JSON schemas keep working. Tests cover pydantic-core behaviour (lists, models with exclude, definition refs, nested serialization, typed-dict field exclusion, wrong-type warnings, missing or invalid inner schema). On the pydantic side they check that the new ser type is emitted and that dumps and JSON schemas still match, and they would fail before the change.
Patch B instead makes any core schema a valid `serialization` value by routing every non-special type through `build`. This creates an inherent ambiguity: `function-plain`/`function-wrap` validator schemas collide with the function ser schemas. B acknowledges this and needs a separate `as_ser_schema` helper to work around it. B does not update `self_schema.py`, so with schema validation active, arbitrary core schemas in `serialization` would likely be rejected. It also does not update `GenerateJsonSchema.ser_schema` to handle arbitrary core schemas, so serialization-mode JSON schema would silently fall back. Crucially, B never replaces the workaround: it only rewrites the TODO comment and leaves the wrap serializer in `ValidateAs` and elsewhere. It also adds unrelated churn and risk: a stringified `SerSchema` TypeAlias, a `Union` with noqa, a rewrite to `match` statements, and a `cast`.
On correctness, completeness and scope, A is the cleaner and more complete fix.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
17m 5s
Wall-clock time in the container
Full test suite
No new failures
12,388 passed · 500 skipped · 12,888 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
300vs 278 merged
Changed lines, ours vs merged
The issue
View on GitHubAs per the TODO in https://github.com/pydantic/pydantic/pull/13487:
There is currently no way to tell pydantic-core "serialize this value using this core schema". SerSchema is:
SerSchema = Union[
SimpleSerSchema,
PlainSerializerFunctionSerSchema,
WrapSerializerFunctionSerSchema,
FormatSerSchema,
ToStringSerSchema,
ModelSerSchema,
]
The only member that expresses "use this schema" is SimpleSerSchema, and it does not take a schema at all, only a name from a closed literal:
ExpectedSerializationTypes = Literal[
'none', 'int', 'bool', 'float', 'str', 'bytes', 'bytearray', 'list', 'tuple',
'set', 'frozenset', 'generator', 'dict', 'datetime', 'date', 'time', 'timedelta',
'url', 'multi-host-url', 'json', 'uuid', 'any',
]
class SimpleSerSchema(TypedDict, total=False):
type: Required[ExpectedSerializationTypes]
So an arbitrary schema (a model, a dataclass, list[int], a TypedDict, anything constructed by the schema generator) cannot be used directly. Every place in Pydantic that needs it works around the gap the same way: it wraps the schema in a wrap serializer whose function is the identity, purely so the schema can ride along in the schema argument.
From ValidateAs.__get_pydantic_core_schema__():
The two patches
pydantic-core/python/pydantic_core/core_schema.pyindex cb87ad5a..343da5a9 100644@@ -476,6 +476,35 @@ def model_ser_schema(cls: type[Any], schema: CoreSchema) -> ModelSerSchema: return ModelSerSchema(type='model', cls=cls, schema=schema) +class SchemaSerSchema(TypedDict, total=False):+ type: Required[Literal['schema']]+ schema: Required[CoreSchema]+++def schema_ser_schema(schema: CoreSchema) -> SchemaSerSchema:+ """+ Returns a schema for serialization using an arbitrary core schema.++ This is useful when the value should be serialized according to a different core schema+ than the one used for validation, e.g.:++ ```py+ from pydantic_core import SchemaSerializer, core_schema++ int_list_schema = core_schema.list_schema(core_schema.int_schema())+ schema = core_schema.any_schema(+ serialization=core_schema.schema_ser_schema(int_list_schema)+ )+ v = SchemaSerializer(schema)+ assert v.to_json([1, 2]) == b'[1,2]'+ ```++ Args:+ schema: The core schema to use to serialize the value+ """+ return SchemaSerSchema(type='schema', schema=schema)++ SerSchema: TypeAlias = ( SimpleSerSchema | PlainSerializerFunctionSerSchema@@ -483,6 +512,7 @@ SerSchema: TypeAlias = ( | FormatSerSchema | ToStringSerSchema | ModelSerSchema+ | SchemaSerSchema ) pydantic-core/src/self_schema.pyindex 18e6cc0a..c21d767a 100644@@ -6846,6 +6846,14 @@ self_schema = { }, 'extra_behavior': 'forbid', },+ 'schema': {+ 'type': 'typed-dict',+ 'fields': {+ 'type': {'schema': {'type': 'literal', 'expected': ['schema']}, 'required': True},+ 'schema': {'schema': {'type': 'definition-ref', 'schema_ref': 'root-schema'}, 'required': True},+ },+ 'extra_behavior': 'forbid',+ }, }, 'ref': 'ser-schema', },pydantic-core/src/serializers/shared.rsindex baf2dcc0..8ba9d57c 100644@@ -198,6 +198,15 @@ impl CombinedSerializer { ) .map_err(|err| py_schema_error_type!("Error building `function-wrap` serializer:\n {err}")); }+ Some("schema") => {+ // `schema` means "serialize using this arbitrary core schema", so build a full+ // serializer from the inner core schema (including its own `serialization` key,+ // prebuilt serializers and polymorphism)+ return ser_schema+ .get_as_req::<Bound<'_, PyDict>>(intern!(py, "schema"))+ .and_then(|inner_schema| Self::build(&inner_schema, config, definitions))+ .map_err(|err| py_schema_error_type!("Error building `schema` serializer:\n {err}"));+ } Some( // applies to lists tuples and dicts, does not override the main schema `type` "include-exclude-sequence" | "include-exclude-dict"pydantic-core/tests/serializers/test_schema_ser_schema.pynew file mode 100644index 00000000..2af86d61@@ -0,0 +1,114 @@+import json++import pytest++from pydantic_core import SchemaError, SchemaSerializer, core_schema+++class BasicModel:+ def __init__(self, **kwargs):+ for key, value in kwargs.items():+ setattr(self, key, value)+++def test_schema_ser_schema_list():+ s = SchemaSerializer(+ core_schema.any_schema(+ serialization=core_schema.schema_ser_schema(core_schema.list_schema(core_schema.bytes_schema()))+ ),+ config={'ser_json_bytes': 'base64'},+ )+ assert s.to_python([b'foo']) == [b'foo']+ assert s.to_python([b'foo'], mode='json') == ['Zm9v']+ assert s.to_json([b'foo']) == b'["Zm9v"]'+++def test_schema_ser_schema_overrides_validation_schema():+ # validation schema is `int`, serialization uses the `str` schema of the inner schema+ s = SchemaSerializer(+ core_schema.int_schema(+ serialization=core_schema.schema_ser_schema(+ core_schema.str_schema(serialization=core_schema.to_string_ser_schema(when_used='always'))+ )+ )+ )+ assert s.to_python(1) == '1'+ assert s.to_json(1) == b'"1"'+++def test_schema_ser_schema_nested_serialization_respected():+ # the `serialization` key of the inner schema is used too+ s = SchemaSerializer(+ core_schema.any_schema(+ serialization=core_schema.schema_ser_schema(+ core_schema.int_schema(serialization=core_schema.plain_serializer_function_ser_schema(lambda v: v * 2))+ )+ )+ )+ assert s.to_python(3) == 6+ assert s.to_json(3) == b'6'+++def test_schema_ser_schema_model():+ model_schema = core_schema.model_schema(+ BasicModel,+ core_schema.model_fields_schema(+ {+ 'a': core_schema.model_field(core_schema.int_schema()),+ 'b': core_schema.model_field(core_schema.bytes_schema()),+ }+ ),+ )+ s = SchemaSerializer(core_schema.any_schema(serialization=core_schema.schema_ser_schema(model_schema)))+ m = BasicModel(a=1, b=b'x')+ assert s.to_python(m) == {'a': 1, 'b': b'x'}+ assert s.to_python(m, exclude={'b'}) == {'a': 1}+ assert json.loads(s.to_json(m)) == {'a': 1, 'b': 'x'}+++def test_schema_ser_schema_definition_ref():+ s = SchemaSerializer(+ core_schema.definitions_schema(+ core_schema.any_schema(+ serialization=core_schema.schema_ser_schema(+ core_schema.list_schema(core_schema.definition_reference_schema('inner'))+ )+ ),+ [core_schema.int_schema(ref='inner', serialization=core_schema.to_string_ser_schema(when_used='always'))],+ )+ )+ assert s.to_python([1, 2]) == ['1', '2']+ assert s.to_json([1, 2]) == b'["1","2"]'+++def test_schema_ser_schema_in_typed_dict_field():+ s = SchemaSerializer(+ core_schema.typed_dict_schema(+ {+ 'x': core_schema.typed_dict_field(+ core_schema.any_schema(+ serialization=core_schema.schema_ser_schema(+ core_schema.dict_schema(core_schema.str_schema(), core_schema.int_schema())+ )+ )+ )+ }+ )+ )+ assert s.to_python({'x': {'a': 1, 'b': 2}}, exclude={'x': {'b'}}) == {'x': {'a': 1}}+++def test_schema_ser_schema_wrong_type_warns():+ s = SchemaSerializer(core_schema.any_schema(serialization=core_schema.schema_ser_schema(core_schema.int_schema())))+ with pytest.warns(UserWarning, match='Expected `int`'):+ assert s.to_python('a') == 'a'+++def test_schema_ser_schema_missing_schema():+ with pytest.raises(SchemaError, match="Error building `schema` serializer:\n KeyError: 'schema'"):+ SchemaSerializer(core_schema.any_schema(serialization={'type': 'schema'}))+++def test_schema_ser_schema_invalid_inner_schema():+ with pytest.raises(SchemaError, match='Error building `schema` serializer'):+ SchemaSerializer(core_schema.any_schema(serialization=core_schema.schema_ser_schema({'type': 'unknown'})))pydantic/_internal/_generate_schema.pyindex 3933e821..21453d76 100644@@ -706,9 +706,7 @@ class GenerateSchema: schema = core_schema.lax_or_strict_schema( lax_schema=lax_schema, strict_schema=strict_schema,- serialization=core_schema.wrap_serializer_function_ser_schema(- lambda v, h: h(v), schema=dict_schema, info_arg=False- ),+ serialization=core_schema.schema_ser_schema(dict_schema), ) return schemapydantic/functional_validators.pyindex 24f39be2..665beb7c 100644@@ -217,14 +217,7 @@ class PlainValidator: schema = handler(source_type) # TODO if `schema['serialization']` is one of `'include-exclude-dict/sequence', # schema validation will fail. That's why we use 'type ignore' comments below.- serialization = schema.get(- 'serialization',- core_schema.wrap_serializer_function_ser_schema(- function=lambda v, h: h(v),- schema=schema,- return_schema=handler.generate_schema(source_type),- ),- )+ serialization = schema.get('serialization', core_schema.schema_ser_schema(schema)) except PydanticSchemaGenerationError: serialization = None @@ -811,9 +804,7 @@ else: return instance_of_schema else: # Use the "original" approach to serialization- instance_of_schema['serialization'] = core_schema.wrap_serializer_function_ser_schema(- function=lambda v, h: h(v), schema=original_schema- )+ instance_of_schema['serialization'] = core_schema.schema_ser_schema(original_schema) return core_schema.json_or_python_schema(python_schema=instance_of_schema, json_schema=original_schema) __hash__ = object.__hash__@@ -847,9 +838,7 @@ else: metadata = {'pydantic_js_annotation_functions': [lambda _c, h: h(original_schema)]} return core_schema.any_schema( metadata=metadata,- serialization=core_schema.wrap_serializer_function_ser_schema(- function=lambda v, h: h(v), schema=original_schema- ),+ serialization=core_schema.schema_ser_schema(original_schema), ) __hash__ = object.__hash__@@ -911,11 +900,5 @@ class ValidateAs: except GENERATE_SCHEMA_ERRORS: ser_schema = core_schema.any_schema() - # TODO: pydantic-core currently doesn't accept it, but we should be able to provide an- # arbitrary core schema for serialization (currently `simple_ser_schema()` can be used,- # but only allow specifying a specific `'type`). Working around this with a wrap serializer- # is already done in other parts of the code base, so grep for similar cases when fixing:- schema['serialization'] = core_schema.wrap_serializer_function_ser_schema(- function=lambda v, h: h(v), schema=ser_schema- )+ schema['serialization'] = core_schema.schema_ser_schema(ser_schema) return schemapydantic/json_schema.pyindex e7abcad0..8335c6d2 100644@@ -2249,6 +2249,9 @@ class GenerateJsonSchema: elif schema['type'] == 'model': # ModelSerSchema return self.generate_inner(schema['schema'])+ elif schema['type'] == 'schema':+ # SchemaSerSchema+ return self.generate_inner(schema['schema']) return None def complex_schema(self, schema: core_schema.ComplexSchema) -> JsonSchemaValue:tests/test_json_schema.pyindex f7533631..70874e68 100644@@ -6854,7 +6854,7 @@ def test_min_and_max_in_schema() -> None: def test_plain_field_validator_serialization() -> None:- """`PlainValidator` internally creates a wrap ser. schema. This tests that we can+ """`PlainValidator` internally creates a `schema` ser. schema. This tests that we can still generate a JSON Schema in `'serialization'` mode. """ tests/test_types.pyindex d168c449..944ea30f 100644@@ -5,6 +5,7 @@ import os import re import sys import warnings+from collections import OrderedDict, defaultdict from collections.abc import Callable, Sequence from dataclasses import dataclass from datetime import date, datetime, timedelta@@ -1679,6 +1680,110 @@ def test_validate_as_unknown_serialization() -> None: assert ta.dump_python(Arbitrary(1)) == Arbitrary(1) +def test_validate_as_serialization_uses_source_type_schema() -> None:+ """The source type schema is used for serialization (and serialization JSON Schema), without any wrap serializer."""++ class Model(BaseModel):+ a: int+ b: bytes++ ta = TypeAdapter(Annotated[Model, ValidateAs(int, lambda v: Model(a=v, b=b'x'))])++ assert ta.core_schema['serialization']['type'] == 'schema'+ assert ta.dump_python(ta.validate_python(1)) == {'a': 1, 'b': b'x'}+ assert ta.dump_python(ta.validate_python(1), exclude={'b'}) == {'a': 1}+ assert ta.dump_json(ta.validate_python(1)) == b'{"a":1,"b":"x"}'+ assert ta.json_schema(mode='validation') == {'type': 'integer'}+ assert ta.json_schema(mode='serialization') == {+ 'properties': {+ 'a': {'title': 'A', 'type': 'integer'},+ 'b': {'format': 'binary', 'title': 'B', 'type': 'string'},+ },+ 'required': ['a', 'b'],+ 'title': 'Model',+ 'type': 'object',+ }+++def test_validate_as_serialization_json_schema() -> None:+ def first_even(nums: list[int]) -> int:+ return next(i for i in nums if i % 2 == 0)++ ta = TypeAdapter(Annotated[int, ValidateAs(list[int], first_even)])++ assert ta.json_schema(mode='validation') == {'items': {'type': 'integer'}, 'type': 'array'}+ assert ta.json_schema(mode='serialization') == {'type': 'integer'}+++def test_skip_validation_serialization_schema() -> None:+ class Model(BaseModel):+ x: SkipValidation[list[int]]++ ser_schema = Model.__pydantic_core_schema__['schema']['fields']['x']['schema']['serialization']+ assert ser_schema == {'type': 'schema', 'schema': {'type': 'list', 'items_schema': {'type': 'int'}}}+ assert Model(x=[1, 2]).model_dump() == {'x': [1, 2]}+ assert Model(x=[1, 2]).model_dump_json() == '{"x":[1,2]}'+ with pytest.warns(UserWarning, match='Expected `list\\[int\\]`'):+ assert Model(x='abc').model_dump() == {'x': 'abc'}+ assert Model.model_json_schema(mode='serialization') == {+ 'properties': {'x': {'items': {'type': 'integer'}, 'title': 'X', 'type': 'array'}},+ 'required': ['x'],+ 'title': 'Model',+ 'type': 'object',+ }+++def test_instance_of_serialization_schema() -> None:+ class Inner(BaseModel):+ a: int+ b: int++ class Model(BaseModel):+ x: InstanceOf[Inner]++ ta_schema = TypeAdapter(InstanceOf[Inner]).core_schema+ assert ta_schema['type'] == 'definitions'+ x_schema = ta_schema['schema']+ assert x_schema['type'] == 'json-or-python'+ assert x_schema['python_schema']['serialization']['type'] == 'schema'++ m = Model(x=Inner(a=1, b=2))+ assert m.model_dump() == {'x': {'a': 1, 'b': 2}}+ assert m.model_dump(exclude={'x': {'b'}}) == {'x': {'a': 1}}+ assert m.model_dump_json() == '{"x":{"a":1,"b":2}}'+++def test_plain_validator_serialization_schema() -> None:+ ta = TypeAdapter(Annotated[list[bytes], PlainValidator(lambda v: [b'x'])])++ assert ta.core_schema['serialization'] == {+ 'type': 'schema',+ 'schema': {'type': 'list', 'items_schema': {'type': 'bytes'}},+ }+ assert ta.dump_python(ta.validate_python(None)) == [b'x']+ assert ta.dump_json(ta.validate_python(None)) == b'["x"]'+ assert ta.json_schema(mode='serialization') == {'items': {'format': 'binary', 'type': 'string'}, 'type': 'array'}+++@pytest.mark.parametrize('tp', [OrderedDict, defaultdict])+def test_mapping_serialization_schema(tp) -> None:+ ta = TypeAdapter(tp[str, list[bytes]])++ assert ta.core_schema['serialization'] == {+ 'type': 'schema',+ 'schema': {+ 'type': 'dict',+ 'keys_schema': {'type': 'str'},+ 'values_schema': {'type': 'list', 'items_schema': {'type': 'bytes'}},+ 'strict': False,+ },+ }+ value = ta.validate_python({'a': [b'x'], 'b': [b'y']})+ assert ta.dump_python(value) == {'a': [b'x'], 'b': [b'y']}+ assert ta.dump_python(value, exclude={'b'}) == {'a': [b'x']}+ assert ta.dump_json(value) == b'{"a":["x"],"b":["y"]}'++ @pytest.mark.skipif(sys.version_info < (3, 12), reason="`Annotated` doesn't allow instances in <3.12") def test_skip_validation_arbitrary_type_object() -> None: """https://github.com/pydantic/pydantic/issues/11997.
pydantic-core/python/pydantic_core/core_schema.pyindex cb87ad5ae..9e847a5ad 100644@@ -12,7 +12,7 @@ from datetime import date, datetime, time, timedelta from decimal import Decimal from fractions import Fraction from re import Pattern-from typing import TYPE_CHECKING, Any, Literal+from typing import TYPE_CHECKING, Any, Literal, Union from typing_extensions import TypeVar, deprecated @@ -278,6 +278,9 @@ def simple_ser_schema(type: ExpectedSerializationTypes) -> SimpleSerSchema: """ Returns a schema for serialization with a custom type. + Note that any core schema can be used as a serialization schema, e.g. `int_schema()`+ is equivalent to `simple_ser_schema('int')`.+ Args: type: The type to use for serialization """@@ -469,6 +472,9 @@ def model_ser_schema(cls: type[Any], schema: CoreSchema) -> ModelSerSchema: """ Returns a schema for serialization using a model. + Note that any core schema can be used as a serialization schema, e.g. `model_schema()`+ can be used instead.+ Args: cls: The expected class type, used to generate warnings if the wrong type is passed schema: Internal schema to use to serialize the model dict@@ -476,14 +482,9 @@ def model_ser_schema(cls: type[Any], schema: CoreSchema) -> ModelSerSchema: return ModelSerSchema(type='model', cls=cls, schema=schema) -SerSchema: TypeAlias = (- SimpleSerSchema- | PlainSerializerFunctionSerSchema- | WrapSerializerFunctionSerSchema- | FormatSerSchema- | ToStringSerSchema- | ModelSerSchema-)+SerSchema: TypeAlias = 'SimpleSerSchema | PlainSerializerFunctionSerSchema | WrapSerializerFunctionSerSchema | FormatSerSchema | ToStringSerSchema | ModelSerSchema | CoreSchema'++"""The schemas that can be used as the `serialization` key of a core schema.""" class InvalidSchema(TypedDict, total=False):@@ -1626,7 +1627,7 @@ def filter_seq_schema(*, include: set[int] | None = None, exclude: set[int] | No return _dict_not_none(type='include-exclude-sequence', include=include, exclude=exclude) -IncExSeqOrElseSerSchema: TypeAlias = IncExSeqSerSchema | SerSchema+IncExSeqOrElseSerSchema: TypeAlias = Union[IncExSeqSerSchema, SerSchema] # noqa: UP007 (TypeError when evaluating) class ListSchema(TypedDict, total=False):@@ -2032,7 +2033,7 @@ def filter_dict_schema(*, include: IncExDict | None = None, exclude: IncExDict | return _dict_not_none(type='include-exclude-dict', include=include, exclude=exclude) -IncExDictOrElseSerSchema: TypeAlias = IncExDictSerSchema | SerSchema+IncExDictOrElseSerSchema: TypeAlias = Union[IncExDictSerSchema, SerSchema] # noqa: UP007 (TypeError when evaluating) class DictSchema(TypedDict, total=False):pydantic-core/src/serializers/shared.rsindex baf2dcc07..5e398125e 100644@@ -206,10 +206,15 @@ impl CombinedSerializer { ) // if `schema.serialization.type` is None, fall back to `schema.type` | None => (),- Some(ser_type) => {- // otherwise if `schema.serialization.type` is defined, use that with `find_serializer`- // instead of `schema.type`. In this case it's an error if a serializer isn't found.- return Self::find_serializer(ser_type, &ser_schema, config, definitions);+ Some(_) => {+ // otherwise, `schema.serialization` is an arbitrary core schema (which includes the+ // simple `{'type': ...}` ser schemas), so build a serializer from it as if it was+ // the main schema (this ensures nested `serialization` schemas, prebuilt serializers+ // and polymorphic serialization are handled). In this case, it's an error if a+ // serializer isn't found.+ // Note that as a consequence, `function-plain`/`function-wrap` *validator* schemas can't+ // be used as `schema.serialization`, as they are interpreted as the function *ser* schemas.+ return Self::build(&ser_schema, config, definitions); } } }pydantic-core/tests/serializers/test_core_schema_as_ser_schema.pynew file mode 100644index 000000000..b1110316f@@ -0,0 +1,124 @@+"""Tests for arbitrary core schemas being used as the `serialization` schema."""++from typing import Any++import pytest++from pydantic_core import SchemaError, SchemaSerializer, core_schema++from ..conftest import plain_repr+++@pytest.mark.parametrize(+ ['ser_schema', 'value', 'expected_python', 'expected_json'],+ [+ # equivalent to `simple_ser_schema('list')` (i.e. it must still work):+ (core_schema.simple_ser_schema('list'), [1, 'a'], [1, 'a'], b'[1,"a"]'),+ (core_schema.list_schema(core_schema.int_schema()), [1, 2], [1, 2], b'[1,2]'),+ (+ core_schema.tuple_schema([core_schema.int_schema(), core_schema.str_schema()]),+ (1, 'a'),+ (1, 'a'),+ b'[1,"a"]',+ ),+ (+ core_schema.dict_schema(core_schema.str_schema(), core_schema.int_schema()),+ {'a': 1},+ {'a': 1},+ b'{"a":1}',+ ),+ (+ core_schema.typed_dict_schema({'a': core_schema.typed_dict_field(core_schema.int_schema())}),+ {'a': 1, 'b': 2},+ {'a': 1},+ b'{"a":1}',+ ),+ (+ core_schema.definitions_schema(+ core_schema.definition_reference_schema('list_of_int'),+ [core_schema.list_schema(core_schema.int_schema(), ref='list_of_int')],+ ),+ [1, 2],+ [1, 2],+ b'[1,2]',+ ),+ # `'function-before'`/`'function-after'` core schemas serialize using the inner schema:+ (+ core_schema.no_info_after_validator_function(lambda v: v, core_schema.int_schema()),+ 1,+ 1,+ b'1',+ ),+ ],+)+def test_core_schema_as_ser_schema(ser_schema: Any, value: Any, expected_python: Any, expected_json: bytes) -> None:+ s = SchemaSerializer(core_schema.any_schema(serialization=ser_schema))++ assert s.to_python(value) == expected_python+ assert s.to_json(value) == expected_json+++def test_core_schema_as_ser_schema_expected_type() -> None:+ s = SchemaSerializer(core_schema.any_schema(serialization=core_schema.list_schema(core_schema.int_schema())))++ with pytest.warns(UserWarning, match=r'Expected `int` - serialized value may not be as expected'):+ assert s.to_python(['a']) == ['a']+++def test_core_schema_as_ser_schema_nested_serialization() -> None:+ """The `serialization` schema of the core schema used as a `serialization` schema is taken into account."""+ s = SchemaSerializer(+ core_schema.any_schema(+ serialization=core_schema.int_schema(+ serialization=core_schema.plain_serializer_function_ser_schema(lambda v: v * 2),+ ),+ )+ )++ assert s.to_python(3) == 6+ assert s.to_json(3) == b'6'+++def test_core_schema_as_ser_schema_unknown_type() -> None:+ with pytest.raises(SchemaError, match=r'Unknown serialization schema type: `unknown`'):+ SchemaSerializer(core_schema.any_schema(serialization={'type': 'unknown'})) # pyright: ignore[reportArgumentType]+++def test_model_schema_as_ser_schema_polymorphic() -> None:+ class Model:+ __pydantic_complete__ = True++ class SubModel(Model):+ pass++ model_schema = core_schema.model_schema(+ Model,+ schema=core_schema.model_fields_schema({'x': core_schema.model_field(core_schema.int_schema())}),+ config=core_schema.CoreConfig(polymorphic_serialization=True),+ )+ Model.__pydantic_serializer__ = SchemaSerializer(model_schema) # pyright: ignore[reportAttributeAccessIssue]++ sub_model_schema = core_schema.model_schema(+ SubModel,+ schema=core_schema.model_fields_schema(+ {+ 'x': core_schema.model_field(core_schema.int_schema()),+ 'y': core_schema.model_field(core_schema.int_schema()),+ }+ ),+ )+ SubModel.__pydantic_serializer__ = SchemaSerializer(sub_model_schema) # pyright: ignore[reportAttributeAccessIssue]++ s = SchemaSerializer(core_schema.any_schema(serialization=model_schema))+ # Using the model schema as a serialization schema behaves the same as using it as the main schema+ # (in this case, the prebuilt serializer of `Model` is used, wrapped in a polymorphism trampoline):+ assert plain_repr(s) == plain_repr(SchemaSerializer(model_schema))+ assert plain_repr(s).startswith(+ 'SchemaSerializer(serializer=PolymorphismTrampoline(PolymorphismTrampoline{class:Py(0x'+ )+ assert 'serializer:Prebuilt(PrebuiltSerializer{' in plain_repr(s)++ sub_model = SubModel()+ sub_model.__dict__ = {'x': 1, 'y': 2}+ assert s.to_python(sub_model) == {'x': 1, 'y': 2}+ assert s.to_json(sub_model) == b'{"x":1,"y":2}'pydantic-core/tests/test_typing.pyindex 94164c689..c698ccc17 100644@@ -247,6 +247,11 @@ def test_ser_function_wrap(): ) +def test_ser_core_schema():+ s = SchemaSerializer(core_schema.any_schema(serialization=core_schema.list_schema(core_schema.int_schema())))+ assert s.to_python([1, 2]) == [1, 2]++ def test_error_details() -> None: # Test that the ErrorDetails type is correctly exported. def act_on_error_details(_: ErrorDetails) -> None:pydantic/_internal/_core_utils.pyindex 56ea79bfc..79a8588f1 100644@@ -53,6 +53,29 @@ def is_function_with_inner_schema( return schema['type'] in _FUNCTION_WITH_INNER_SCHEMA_TYPES +def as_ser_schema(schema: CoreSchema) -> core_schema.SerSchema:+ """Return a schema suitable for the `'serialization'` key of a core schema, from an arbitrary core schema.++ Any core schema can be used as a serialization schema, except `'function-plain'` and `'function-wrap'`+ schemas, as these types are already used by the plain and wrap serializer *function* schemas.+ For these, mimic what pydantic-core would do if they were used as the *main* schema:++ - if the function schema has a `'serialization'` schema, use it.+ - otherwise, a `'function-plain'` schema is serialized as `'any'`, and a `'function-wrap'` schema+ is serialized using its inner schema.++ Note that `schema` is expected to be a *core* schema (e.g. as returned by the schema generator),+ not an existing `'serialization'` schema.+ """+ if schema['type'] == 'function-plain' or schema['type'] == 'function-wrap':+ if (ser_schema := schema.get('serialization')) is not None:+ return ser_schema+ if schema['type'] == 'function-plain':+ return core_schema.any_schema()+ return as_ser_schema(schema['schema'])+ return schema++ def is_list_like_schema_with_items_schema( schema: CoreSchema, ) -> TypeGuard[core_schema.ListSchema | core_schema.SetSchema | core_schema.FrozenSetSchema]:pydantic/functional_validators.pyindex 24f39be22..339e89946 100644@@ -911,10 +911,10 @@ class ValidateAs: except GENERATE_SCHEMA_ERRORS: ser_schema = core_schema.any_schema() - # TODO: pydantic-core currently doesn't accept it, but we should be able to provide an- # arbitrary core schema for serialization (currently `simple_ser_schema()` can be used,- # but only allow specifying a specific `'type`). Working around this with a wrap serializer- # is already done in other parts of the code base, so grep for similar cases when fixing:+ # TODO: pydantic-core now accepts an arbitrary core schema for serialization, so the+ # wrap serializer workaround below can be replaced with+ # `schema['serialization'] = _core_utils.as_ser_schema(ser_schema)`.+ # The same workaround is used in other parts of the code base, so grep for similar cases when fixing: schema['serialization'] = core_schema.wrap_serializer_function_ser_schema( function=lambda v, h: h(v), schema=ser_schema )pydantic/json_schema.pyindex e7abcad07..ebcd0f523 100644@@ -2237,18 +2237,18 @@ class GenerateJsonSchema: Returns: The generated JSON schema. """- schema_type = schema['type']- if schema_type == 'function-plain' or schema_type == 'function-wrap':- # PlainSerializerFunctionSerSchema or WrapSerializerFunctionSerSchema- return_schema = schema.get('return_schema')- if return_schema is not None:- return self.generate_inner(return_schema)- elif schema_type == 'format' or schema_type == 'to-string':- # FormatSerSchema or ToStringSerSchema- return self.str_schema(core_schema.str_schema())- elif schema['type'] == 'model':- # ModelSerSchema- return self.generate_inner(schema['schema'])+ match schema:+ case {'type': 'function-plain' | 'function-wrap'}:+ # PlainSerializerFunctionSerSchema or WrapSerializerFunctionSerSchema+ return_schema = schema.get('return_schema')+ if return_schema is not None:+ return self.generate_inner(return_schema)+ case {'type': 'format' | 'to-string'}:+ # FormatSerSchema or ToStringSerSchema+ return self.str_schema(core_schema.str_schema())+ case {'type': 'model'}:+ # ModelSerSchema+ return self.generate_inner(schema['schema']) return None def complex_schema(self, schema: core_schema.ComplexSchema) -> JsonSchemaValue:@@ -3016,6 +3016,6 @@ def _get_ser_schema_for_default_value(schema: CoreSchema) -> core_schema.PlainSe and ser_schema['type'] == 'function-plain' and not ser_schema.get('info_arg') ):- return ser_schema+ return cast('core_schema.PlainSerializerFunctionSerSchema', ser_schema) if _core_utils.is_function_with_inner_schema(schema): return _get_ser_schema_for_default_value(schema['schema'])tests/test_internal.pyindex acd0b46e7..c3912bc7e 100644@@ -16,6 +16,7 @@ from pydantic_core import core_schema as cs from pydantic import BaseModel, TypeAdapter from pydantic._internal._config import ConfigWrapper from pydantic._internal._core_metadata import update_core_metadata+from pydantic._internal._core_utils import as_ser_schema from pydantic._internal._fields import resolve_default_value from pydantic._internal._generate_schema import GenerateSchema from pydantic._internal._repr import Representation@@ -288,3 +289,58 @@ def test_resolve_default_value_missing_validated_data(): validated_data=None, call_default_factory=True, )+++def _identity(v: Any) -> Any:+ return v+++def _wrap_identity(v: Any, handler: Any) -> Any:+ return handler(v)+++@pytest.mark.parametrize(+ ['schema', 'expected'],+ [+ # Any non-function schema is returned as is:+ (cs.int_schema(), cs.int_schema()),+ (+ cs.no_info_after_validator_function(_identity, cs.int_schema()),+ cs.no_info_after_validator_function(_identity, cs.int_schema()),+ ),+ # The `'serialization'` schema of a plain/wrap function schema takes precedence:+ (+ cs.no_info_plain_validator_function(_identity, serialization=cs.simple_ser_schema('str')),+ cs.simple_ser_schema('str'),+ ),+ (+ cs.no_info_wrap_validator_function(+ _wrap_identity, cs.int_schema(), serialization=cs.simple_ser_schema('str')+ ),+ cs.simple_ser_schema('str'),+ ),+ # Otherwise, plain function schemas are serialized as `'any'`, wrap function schemas using the inner schema:+ (cs.no_info_plain_validator_function(_identity), cs.any_schema()),+ (cs.no_info_wrap_validator_function(_wrap_identity, cs.int_schema()), cs.int_schema()),+ # Nested plain/wrap function schemas are unwrapped:+ (+ cs.no_info_wrap_validator_function(+ _wrap_identity, cs.no_info_wrap_validator_function(_wrap_identity, cs.int_schema())+ ),+ cs.int_schema(),+ ),+ (+ cs.no_info_wrap_validator_function(_wrap_identity, cs.no_info_plain_validator_function(_identity)),+ cs.any_schema(),+ ),+ (+ cs.no_info_wrap_validator_function(+ _wrap_identity,+ cs.no_info_plain_validator_function(_identity, serialization=cs.simple_ser_schema('str')),+ ),+ cs.simple_ser_schema('str'),+ ),+ ],+)+def test_as_ser_schema(schema: CoreSchema, expected: cs.SerSchema) -> None:+ assert as_ser_schema(schema) == expected