Backtestspydanticrun-20261003-150343#13467
Support Ellipsis
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Both patches make `EllipsisType` usable as a field type. That covers the issue's use case: a `tuple[int | EllipsisType, ...]` field holding Python values. Patch A adds a new core schema, a Rust validator, a Rust serializer and a new error type. - `validate_json` can never succeed, because the validator only accepts `input.as_python()`. - JSON serialization always errors. - JSON schema generation raises. - So a model with an Ellipsis field cannot round-trip through JSON or produce a JSON schema. That is a significant limitation for a general-purpose `BaseModel` field. - It touches many Rust and Python files to deliver Python-only behavior, which adds risk. - Its tests are thin. One TypeAdapter test checks the happy path for validation and dump, and checks that dumping invalid values or dumping to JSON raises. There is no model-level or union test. Patch B is implemented entirely in `_generate_schema.py` using existing core primitives. - In Python mode it uses `literal_schema([...])`. - In JSON mode it accepts the `'...'` string and maps it back to `Ellipsis`. - A JSON-only plain serializer emits `'...'` and raises `PydanticSerializationUnexpectedValue` for non-Ellipsis values, so unions serialize correctly. - JSON schema output is a sensible `const '...'` string. - Its tests are more thorough. They use the issue's own tuple example and cover Python and JSON dumps, the JSON round-trip, strict mode and JSON schema in both modes. They also cover union serialization with int and str, plus error output for invalid Python and JSON input. - One debatable design choice is representing Ellipsis as the `'...'` string in JSON, which could be ambiguous in `str | EllipsisType` unions. Even so, it gives working end-to-end behavior where Patch A gives errors. Overall, Patch B delivers more complete functionality with lower implementation risk and better test coverage.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
10m 22s
Wall-clock time in the container
Full test suite
No new failures
12,308 passed · 499 skipped · 12,807 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
128vs 225 merged
Changed lines, ours vs merged
The issue
View on GitHubInitial Checks
- I have searched Google & GitHub for similar requests and couldn't find anything
- I have read and followed the docs and still think this feature is missing
Description
Ellipsis is a builtin type that pydantic does not support. Its usage is infrequent, but it can be useful to express placeholders:
This is a bit of a contrived example, but:
class Model(pydantic.BaseModel):
m: list[list[float]]
subset: tuple[int | Ellipsis, int | Ellipsis]
def subset_arr(self) -> np.NDArray:
return np.asarray(self.m)[*self.subset]
Here's how I implement this right now with an adapter: https://github.com/psalvaggio/scientific_pydantic/blob/main/src/scientific_pydantic/ellipsis.py
However, since this is a standard library type, I think this would be a good thing to live in pydantic itself.
Affected Components
- Compatibility between releases
- Data validation/parsing
- Data serialization -
.model_dump()and.model_dump_json() - JSON Schema
- Dataclasses
- Model Config
- Field Types - adding or changing a particular data type
- Function validation decorator
- Generic Models
- Other Model behaviour -
model_construct(), pickling, private attributes, ORM mode - Plugins and integration with other tools - mypy, FastAPI, python-devtools, Hypothesis, VS Code, PyCharm, etc.
The two patches
docs/api/standard_library_types.mdindex 4739bdb4..d871ef3b 100644@@ -756,6 +756,34 @@ Supported types: [`None`][], [`NoneType`][types.NoneType] or `Literal[None]` (th Allows only `None` as a value. +## Ellipsis++Supported type: [`EllipsisType`][types.EllipsisType].++* Validation: in Python mode, allows only the [`Ellipsis`][] (`...`) value. As JSON has no equivalent value,+ the `'...'` string is expected in JSON mode.+* Serialization: in JSON mode, the `'...'` string is used. In Python mode, the `Ellipsis` value is kept.+* JSON Schema: a `string` with a constant value of `'...'`.++```python+from types import EllipsisType++from pydantic import BaseModel+++class Model(BaseModel):+ subset: tuple[int | EllipsisType, int | EllipsisType]+++m = Model(subset=(1, ...))+print(m.subset)+#> (1, Ellipsis)+print(m.model_dump_json())+#> {"subset":[1,"..."]}+print(Model.model_validate_json('{"subset": ["...", 2]}').subset)+#> (Ellipsis, 2)+```+ ## Generic collection types Pydantic supports a wide variety of generic collection types, both built-ins (such as [`list`][]) and abstract base classespydantic/_internal/_generate_schema.pyindex 3fba8dc3..45ffc2a8 100644@@ -23,7 +23,7 @@ from inspect import Parameter, _ParameterKind from ipaddress import IPv4Address, IPv4Interface, IPv4Network, IPv6Address, IPv6Interface, IPv6Network from itertools import chain from operator import attrgetter-from types import FunctionType, GenericAlias, LambdaType, MethodType, NoneType+from types import EllipsisType, FunctionType, GenericAlias, LambdaType, MethodType, NoneType from typing import ( TYPE_CHECKING, Any,@@ -363,6 +363,7 @@ class GenerateSchema: MultiHostUrl: lambda self, obj: core_schema.multi_host_url_schema(), None: lambda self, obj: core_schema.none_schema(), NoneType: lambda self, obj: core_schema.none_schema(),+ EllipsisType: lambda self, obj: self._ellipsis_schema(), MISSING: lambda self, obj: core_schema.missing_sentinel_schema(), type: lambda self, obj: self._type_schema(), dict: lambda self, obj: self._dict_schema(Any, Any),@@ -1808,6 +1809,25 @@ class GenerateSchema: else: raise PydanticSchemaGenerationError(f'Unable to generate pydantic-core schema for {pattern_type!r}.') + def _ellipsis_schema(self) -> core_schema.CoreSchema:+ def ser_ellipsis(value: Any) -> str:+ if value is not ...:+ raise PydanticSerializationUnexpectedValue(+ f"Expected `Ellipsis` but got `{type(value)}` with value `'{value}'` - serialized value may not be as expected."+ )+ return '...'++ # JSON has no representation of `...`, so the `'...'` string is used in JSON mode:+ return core_schema.json_or_python_schema(+ json_schema=core_schema.no_info_after_validator_function(+ lambda _: ..., core_schema.literal_schema(['...'])+ ),+ python_schema=core_schema.literal_schema([...]),+ serialization=core_schema.plain_serializer_function_ser_schema(+ ser_ellipsis, return_schema=core_schema.literal_schema(['...']), when_used='json'+ ),+ )+ def _hashable_schema(self) -> core_schema.CoreSchema: return core_schema.custom_error_schema( schema=core_schema.json_or_python_schema(tests/test_types.pyindex df4d5f06..a1183e9f 100644@@ -20,6 +20,7 @@ from enum import Enum, IntEnum from numbers import Number from pathlib import Path from re import Pattern+from types import EllipsisType from typing import ( Annotated, Any,@@ -5326,6 +5327,83 @@ def test_none(value_type): ] +def test_ellipsis_type() -> None:+ class Model(BaseModel):+ subset: tuple[int | EllipsisType, int | EllipsisType]+ my_ellipsis: EllipsisType++ m = Model(subset=(1, ...), my_ellipsis=...)+ assert m.subset == (1, ...)+ assert m.my_ellipsis is ...+ assert m.model_dump() == {'subset': (1, ...), 'my_ellipsis': ...}+ assert m.model_dump(mode='json') == {'subset': [1, '...'], 'my_ellipsis': '...'}+ assert m.model_dump_json() == '{"subset":[1,"..."],"my_ellipsis":"..."}'+ assert Model.model_validate_json(m.model_dump_json()) == m+ assert Model.model_validate({'subset': (..., 2), 'my_ellipsis': ...}, strict=True).subset == (..., 2)++ # insert_assert(Model.model_json_schema())+ assert Model.model_json_schema() == {+ 'properties': {+ 'subset': {+ 'maxItems': 2,+ 'minItems': 2,+ 'prefixItems': [+ {'anyOf': [{'type': 'integer'}, {'const': '...', 'type': 'string'}]},+ {'anyOf': [{'type': 'integer'}, {'const': '...', 'type': 'string'}]},+ ],+ 'title': 'Subset',+ 'type': 'array',+ },+ 'my_ellipsis': {'const': '...', 'title': 'My Ellipsis', 'type': 'string'},+ },+ 'required': ['subset', 'my_ellipsis'],+ 'title': 'Model',+ 'type': 'object',+ }+ assert Model.model_json_schema(mode='serialization') == Model.model_json_schema()+++def test_ellipsis_type_union_serialization() -> None:+ class Model(BaseModel):+ x: EllipsisType | int+ y: EllipsisType | str++ m = Model(x=5, y='hello')+ assert m.model_dump(mode='json') == {'x': 5, 'y': 'hello'}+ assert m.model_dump_json() == '{"x":5,"y":"hello"}'+ assert Model(x=..., y=...).model_dump_json() == '{"x":"...","y":"..."}'+++def test_ellipsis_type_invalid() -> None:+ ta = TypeAdapter(EllipsisType)++ with pytest.raises(ValidationError) as exc_info:+ ta.validate_python('...')+ # insert_assert(exc_info.value.errors(include_url=False))+ assert exc_info.value.errors(include_url=False) == [+ {+ 'type': 'literal_error',+ 'loc': (),+ 'msg': 'Input should be Ellipsis',+ 'input': '...',+ 'ctx': {'expected': 'Ellipsis'},+ }+ ]++ with pytest.raises(ValidationError) as exc_info:+ ta.validate_json('null')+ # insert_assert(exc_info.value.errors(include_url=False))+ assert exc_info.value.errors(include_url=False) == [+ {+ 'type': 'literal_error',+ 'loc': (),+ 'msg': "Input should be '...'",+ 'input': None,+ 'ctx': {'expected': "'...'"},+ }+ ]++ def test_none_literal(): class Model(BaseModel): my_none: Literal[None]
docs/errors/validation_errors.mdindex b0d509b97..fc0539fef 100644@@ -724,6 +724,27 @@ except ValidationError as exc: #> 'dict_type' ``` +## `ellipsis_error`++This error is raised when the input isn't the [`Ellipsis`][] literal:++```python+from types import EllipsisType++from pydantic import BaseModel, ValidationError+++class Model(BaseModel):+ e: EllipsisType+++try:+ Model(e=1)+except ValidationError as exc:+ print(repr(exc.errors()[0]['type']))+ #> 'ellipsis_error'+```+ ## `enum` This error is raised when the input value does not exist in an `enum` field members:pydantic-core/python/pydantic_core/core_schema.pyindex c6118a082..965cf28e1 100644@@ -1453,6 +1453,25 @@ def missing_sentinel_schema( ) +class EllipsisSchema(TypedDict, total=False):+ type: Required[Literal['ellipsis']]+ metadata: dict[str, Any]+ serialization: SerSchema+++def ellipsis_schema(+ metadata: dict[str, Any] | None = None,+ serialization: SerSchema | None = None,+) -> EllipsisSchema:+ """Returns a schema for the [`Ellipsis`][] literal."""++ return _dict_not_none(+ type='ellipsis',+ metadata=metadata,+ serialization=serialization,+ )++ # must match input/parse_json.rs::JsonType::try_from JsonType = Literal['null', 'bool', 'int', 'float', 'str', 'list', 'dict'] @@ -4201,6 +4220,7 @@ if not MYPY: | TimedeltaSchema | LiteralSchema | MissingSentinelSchema+ | EllipsisSchema | EnumSchema | IsInstanceSchema | IsSubclassSchema@@ -4261,6 +4281,7 @@ CoreSchemaType: TypeAlias = Literal[ 'timedelta', 'literal', 'missing-sentinel',+ 'ellipsis', 'enum', 'is-instance', 'is-subclass',@@ -4363,6 +4384,7 @@ ErrorType: TypeAlias = Literal[ 'assertion_error', 'literal_error', 'missing_sentinel_error',+ 'ellipsis_error', 'date_type', 'date_parsing', 'date_from_datetime_parsing',pydantic-core/src/errors/types.rsindex da9cdae0c..8f8e936ce 100644@@ -324,6 +324,9 @@ error_types! { // --------------------- // missing sentinel MissingSentinelError {},+ // ---------------------+ // ellipsis+ EllipsisError {}, // date errors DateType {}, DateParsing {@@ -552,6 +555,7 @@ impl ErrorType { Self::CustomError { .. } => "", // custom errors are handled separately Self::LiteralError { .. } => "Input should be {expected}", Self::MissingSentinelError { .. } => "Input should be the 'MISSING' sentinel",+ Self::EllipsisError { .. } => "Input should be the 'Ellipsis' literal", Self::DateType { .. } => "Input should be a valid date", Self::DateParsing { .. } => "Input should be a valid date in the format YYYY-MM-DD, {error}", Self::DateFromDatetimeParsing { .. } => "Input should be a valid date or datetime, {error}",pydantic-core/src/serializers/shared.rsindex 5f7efee37..f6342c1ec 100644@@ -153,6 +153,7 @@ combined_serializer! { TaggedUnion: super::type_serializers::union::TaggedUnionSerializer; Literal: super::type_serializers::literal::LiteralSerializer; MissingSentinel: super::type_serializers::missing_sentinel::MissingSentinelSerializer;+ Ellipsis: super::type_serializers::ellipsis::EllipsisSerializer; Enum: super::type_serializers::enum_::EnumSerializer; Recursive: super::type_serializers::definitions::DefinitionRefSerializer; Tuple: super::type_serializers::tuple::TupleSerializer;@@ -385,6 +386,7 @@ impl PyGcTraverse for CombinedSerializer { CombinedSerializer::TaggedUnion(inner) => inner.py_gc_traverse(visit), CombinedSerializer::Literal(inner) => inner.py_gc_traverse(visit), CombinedSerializer::MissingSentinel(inner) => inner.py_gc_traverse(visit),+ CombinedSerializer::Ellipsis(inner) => inner.py_gc_traverse(visit), CombinedSerializer::Enum(inner) => inner.py_gc_traverse(visit), CombinedSerializer::Recursive(inner) => inner.py_gc_traverse(visit), CombinedSerializer::Tuple(inner) => inner.py_gc_traverse(visit),pydantic-core/src/serializers/type_serializers/ellipsis.rsnew file mode 100644index 000000000..634916c5d@@ -0,0 +1,74 @@+// This serializer is defined so that building a schema serializer containing an+// 'ellipsis' core schema doesn't crash. In practice, the serializer can be used+// if the 'ellipsis' core schema is used standalone (e.g. with a Pydantic type+// adapter), but this isn't something we explicitly support.++use std::borrow::Cow;+use std::sync::{Arc, LazyLock};++use pyo3::prelude::*;+use pyo3::types::{PyDict, PyEllipsis};++use serde::ser::Error;++use crate::PydanticSerializationUnexpectedValue;+use crate::definitions::DefinitionsBuilder;+use crate::serializers::SerializationState;++use super::{BuildSerializer, CombinedSerializer, TypeSerializer};++#[derive(Debug)]+pub struct EllipsisSerializer {}++static ELLIPSIS_SERIALIZER: LazyLock<Arc<CombinedSerializer>> =+ LazyLock::new(|| Arc::new(EllipsisSerializer {}.into()));++impl BuildSerializer for EllipsisSerializer {+ const EXPECTED_TYPE: &'static str = "ellipsis";++ fn build(+ _schema: &Bound<'_, PyDict>,+ _config: Option<&Bound<'_, PyDict>>,+ _definitions: &mut DefinitionsBuilder<Arc<CombinedSerializer>>,+ ) -> PyResult<Arc<CombinedSerializer>> {+ Ok(ELLIPSIS_SERIALIZER.clone())+ }+}++impl_py_gc_traverse!(EllipsisSerializer {});++impl TypeSerializer for EllipsisSerializer {+ fn to_python(&self, value: &Bound<'_, PyAny>, _state: &mut SerializationState<'_>) -> PyResult<Py<PyAny>> {+ let ellipsis = PyEllipsis::get(value.py());++ if value.is(ellipsis) {+ Ok(ellipsis.to_owned().into_any().unbind())+ } else {+ Err(+ PydanticSerializationUnexpectedValue::new_from_msg(Some("Expected 'Ellipsis' object".to_string()))+ .to_py_err(),+ )+ }+ }++ fn json_key<'a, 'py>(+ &self,+ key: &'a Bound<'py, PyAny>,+ state: &mut SerializationState<'py>,+ ) -> PyResult<Cow<'a, str>> {+ self.invalid_as_json_key(key, state, Self::EXPECTED_TYPE)+ }++ fn serde_serialize<S: serde::ser::Serializer>(+ &self,+ _value: &Bound<'_, PyAny>,+ _serializer: S,+ _state: &mut SerializationState<'_>,+ ) -> Result<S::Ok, S::Error> {+ Err(Error::custom("'Ellipsis' can't be serialized to JSON".to_string()))+ }++ fn get_name(&self) -> &str {+ Self::EXPECTED_TYPE+ }+}pydantic-core/src/serializers/type_serializers/mod.rsindex da95113ea..ad664e8f8 100644@@ -6,6 +6,7 @@ pub mod datetime_etc; pub mod decimal; pub mod definitions; pub mod dict;+pub mod ellipsis; pub mod enum_; pub mod float; pub mod format;pydantic-core/src/validators/ellipsis.rsnew file mode 100644index 000000000..d3dfa9faf@@ -0,0 +1,48 @@+use core::fmt::Debug;+use std::sync::{Arc, LazyLock};++use pyo3::prelude::*;+use pyo3::types::{PyDict, PyEllipsis};++use crate::errors::{ErrorType, ValError, ValResult};+use crate::input::Input;++use super::{BuildValidator, CombinedValidator, DefinitionsBuilder, ValidationState, Validator};++#[derive(Debug, Clone)]+pub struct EllipsisValidator {}++static ELLIPSIS_VALIDATOR: LazyLock<Arc<CombinedValidator>> =+ LazyLock::new(|| CombinedValidator::Ellipsis(EllipsisValidator {}).into());++impl BuildValidator for EllipsisValidator {+ const EXPECTED_TYPE: &'static str = "ellipsis";++ fn build(+ _schema: &Bound<'_, PyDict>,+ _config: Option<&Bound<'_, PyDict>>,+ _definitions: &mut DefinitionsBuilder<Arc<CombinedValidator>>,+ ) -> PyResult<Arc<CombinedValidator>> {+ Ok(ELLIPSIS_VALIDATOR.clone())+ }+}++impl_py_gc_traverse!(EllipsisValidator {});++impl Validator for EllipsisValidator {+ fn validate<'py>(+ &self,+ py: Python<'py>,+ input: &(impl Input<'py> + ?Sized),+ _state: &mut ValidationState<'_, 'py>,+ ) -> ValResult<Py<PyAny>> {+ match input.as_python() {+ Some(v) if v.is(PyEllipsis::get(py)) => Ok(v.to_owned().into()),+ _ => Err(ValError::new(ErrorType::EllipsisError { context: None }, input)),+ }+ }++ fn get_name(&self) -> &str {+ Self::EXPECTED_TYPE+ }+}pydantic-core/src/validators/mod.rsindex 2ab41e6b0..7a6875e3d 100644@@ -37,6 +37,7 @@ mod datetime; pub(crate) mod decimal; mod definitions; mod dict;+mod ellipsis; mod enum_; mod float; pub(crate) mod fraction;@@ -610,6 +611,8 @@ fn build_validator_inner( literal::LiteralValidator, // missing sentinel missing_sentinel::MissingSentinelValidator,+ // ellipsis+ ellipsis::EllipsisValidator, // enums enum_::BuildEnumValidator, // any@@ -775,6 +778,8 @@ pub enum CombinedValidator { Literal(literal::LiteralValidator), // Missing sentinel MissingSentinel(missing_sentinel::MissingSentinelValidator),+ // Ellipsis+ Ellipsis(ellipsis::EllipsisValidator), // enums IntEnum(enum_::EnumValidator<enum_::IntEnumValidator>), StrEnum(enum_::EnumValidator<enum_::StrEnumValidator>),pydantic-core/tests/test_errors.pyindex 274ecb3f6..334e281bd 100644@@ -347,6 +347,7 @@ all_errors = [ ('literal_error', 'Input should be foo', {'expected': 'foo'}), ('literal_error', 'Input should be foo or bar', {'expected': 'foo or bar'}), ('missing_sentinel_error', "Input should be the 'MISSING' sentinel", None),+ ('ellipsis_error', "Input should be the 'Ellipsis' literal", None), ('date_type', 'Input should be a valid date', None), ('date_parsing', 'Input should be a valid date in the format YYYY-MM-DD, foobar', {'error': 'foobar'}), ('date_from_datetime_parsing', 'Input should be a valid date or datetime, foobar', {'error': 'foobar'}),pydantic-core/tests/test_schema_functions.pyindex 299b08e3f..65a015a6f 100644@@ -83,6 +83,7 @@ all_schema_functions = [ ), (core_schema.literal_schema, args(['a', 'b']), {'type': 'literal', 'expected': ['a', 'b']}), (core_schema.missing_sentinel_schema, args(), {'type': 'missing-sentinel'}),+ (core_schema.ellipsis_schema, args(), {'type': 'ellipsis'}), ( core_schema.enum_schema, args(MyEnum, list(MyEnum.__members__.values())),pydantic/_internal/_generate_schema.pyindex 3fba8dc3e..3436338cc 100644@@ -23,7 +23,7 @@ from inspect import Parameter, _ParameterKind from ipaddress import IPv4Address, IPv4Interface, IPv4Network, IPv6Address, IPv6Interface, IPv6Network from itertools import chain from operator import attrgetter-from types import FunctionType, GenericAlias, LambdaType, MethodType, NoneType+from types import EllipsisType, FunctionType, GenericAlias, LambdaType, MethodType, NoneType from typing import ( TYPE_CHECKING, Any,@@ -364,6 +364,7 @@ class GenerateSchema: None: lambda self, obj: core_schema.none_schema(), NoneType: lambda self, obj: core_schema.none_schema(), MISSING: lambda self, obj: core_schema.missing_sentinel_schema(),+ EllipsisType: lambda self, obj: core_schema.ellipsis_schema(), type: lambda self, obj: self._type_schema(), dict: lambda self, obj: self._dict_schema(Any, Any), tuple: lambda self, obj: self._tuple_schema(obj),pydantic/json_schema.pyindex 27756c099..c6adb6947 100644@@ -907,6 +907,19 @@ class GenerateJsonSchema: """ raise PydanticOmit + def ellipsis_schema(self, schema: core_schema.EllipsisSchema) -> JsonSchemaValue:+ """Handles JSON schema generation for a core schema that checks if a value is the [`Ellipsis`][] literal.++ Unless overridden in a subclass, this raises an error.++ Args:+ schema: The core schema.++ Returns:+ The generated JSON schema.+ """+ return self.handle_invalid_for_json_schema(schema, 'core_schema.EllipsisSchema')+ def enum_schema(self, schema: core_schema.EnumSchema) -> JsonSchemaValue: """Generates a JSON schema that matches an Enum value. tests/test_json_schema.pyindex 2486dfdd8..633abea77 100644@@ -12,6 +12,7 @@ from enum import Enum, IntEnum from ipaddress import IPv4Address, IPv4Interface, IPv4Network, IPv6Address, IPv6Interface, IPv6Network from pathlib import Path from re import Pattern+from types import EllipsisType from typing import ( Annotated, Any,@@ -1319,6 +1320,14 @@ def test_callable_type(type_, default_value, base_json_schema, properties): assert model_schema['properties'] == properties +def test_ellipsis_schema() -> None:+ class Model(BaseModel):+ e: EllipsisType++ with pytest.raises(PydanticInvalidForJsonSchema):+ Model.model_json_schema()++ @pytest.mark.parametrize( 'default_value,properties', (tests/types/test_ellipsis.pynew file mode 100644index 000000000..11b7fdea7@@ -0,0 +1,21 @@+from types import EllipsisType++import pytest+from pydantic_core import PydanticSerializationError, PydanticSerializationUnexpectedValue++from pydantic import TypeAdapter+++def test_ellipsis_literal() -> None:+ ta = TypeAdapter(EllipsisType)++ assert ta.validate_python(...) is ...+ assert ta.dump_python(...) is ...++ with pytest.raises(PydanticSerializationUnexpectedValue):+ ta.dump_python(1)++ with pytest.raises(+ PydanticSerializationError, match="Error serializing to JSON: 'Ellipsis' can't be serialized to JSON"+ ):+ ta.dump_json(...)