Backtestspydanticrun-20261007-130130#13467
Support Ellipsis
Our patch was judged better than the merged fix.
In this judgment, Patch A = our agent, Patch B = the merged fix.
Both patches make `EllipsisType` a supported type that accepts only the `...` singleton in Python mode, which is the core ask. Patch A goes further and makes the type usable end to end. It maps Ellipsis to the `'...'` string in JSON for validation and serialization, and produces a sensible JSON schema (`const '...'`). So the issue's model (`tuple[int | EllipsisType, ...]`) round-trips through `model_dump_json`/`model_validate_json`, and `model_json_schema()` works. Patch B adds a new core schema type with a Rust validator and serializer. However, it rejects all JSON input, raises on `dump_json`, and raises `PydanticInvalidForJsonSchema` for JSON schema. The issue's model therefore fails on JSON serialization or schema generation, which is a significant usability gap for a pydantic model. B is also much larger and riskier: new Rust modules, a new error type, a new core schema type and core_schema API surface. That needs a pydantic-core rebuild, all for behavior A achieves with existing primitives. On tests, A is thorough: TypeAdapter behavior, error shapes, model usage, JSON schema, union serialization with the expected warning, and rejection of `...` as a bare type argument. B's tests are minimal: one TypeAdapter test plus a JSON schema test asserting failure. A's design choice is defensible, documented, and well tested. B's is more invasive and less functional.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
11m 19s
Wall-clock time in the container
Full test suite
No new failures
12,309 passed · 499 skipped · 12,808 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
151vs 225 merged
Changed lines, ours vs merged
The issue
View on GitHubInitial Checks
- I have searched Google & GitHub for similar requests and couldn't find anything
- I have read and followed the docs and still think this feature is missing
Description
Ellipsis is a builtin type that pydantic does not support. Its usage is infrequent, but it can be useful to express placeholders:
This is a bit of a contrived example, but:
class Model(pydantic.BaseModel):
m: list[list[float]]
subset: tuple[int | Ellipsis, int | Ellipsis]
def subset_arr(self) -> np.NDArray:
return np.asarray(self.m)[*self.subset]
Here's how I implement this right now with an adapter: https://github.com/psalvaggio/scientific_pydantic/blob/main/src/scientific_pydantic/ellipsis.py
However, since this is a standard library type, I think this would be a good thing to live in pydantic itself.
Affected Components
- Compatibility between releases
- Data validation/parsing
- Data serialization -
.model_dump()and.model_dump_json() - JSON Schema
- Dataclasses
- Model Config
- Field Types - adding or changing a particular data type
- Function validation decorator
- Generic Models
- Other Model behaviour -
model_construct(), pickling, private attributes, ORM mode - Plugins and integration with other tools - mypy, FastAPI, python-devtools, Hypothesis, VS Code, PyCharm, etc.
The two patches
docs/api/standard_library_types.mdindex 4739bdb4..737cb72e 100644@@ -756,6 +756,34 @@ Supported types: [`None`][], [`NoneType`][types.NoneType] or `Literal[None]` (th Allows only `None` as a value. +## Ellipsis++Supported types: [`EllipsisType`][types.EllipsisType].++Allows only the [`Ellipsis`][Ellipsis] (`...`) singleton as a value. As it has no JSON equivalent,+it is represented as the `'...'` string in JSON (both for validation and serialization).++<h3>Example</h3>++```python+from types import EllipsisType++from pydantic import BaseModel+++class Model(BaseModel):+ subset: tuple[int | EllipsisType, int | EllipsisType]+++m = Model(subset=(1, ...))+print(m.subset)+#> (1, Ellipsis)+print(m.model_dump_json())+#> {"subset":[1,"..."]}+print(Model.model_validate_json('{"subset":["...",2]}'))+#> subset=(Ellipsis, 2)+```+ ## Generic collection types Pydantic supports a wide variety of generic collection types, both built-ins (such as [`list`][]) and abstract base classespydantic/_internal/_generate_schema.pyindex 3fba8dc3..c5e28eb8 100644@@ -23,7 +23,7 @@ from inspect import Parameter, _ParameterKind from ipaddress import IPv4Address, IPv4Interface, IPv4Network, IPv6Address, IPv6Interface, IPv6Network from itertools import chain from operator import attrgetter-from types import FunctionType, GenericAlias, LambdaType, MethodType, NoneType+from types import EllipsisType, FunctionType, GenericAlias, LambdaType, MethodType, NoneType from typing import ( TYPE_CHECKING, Any,@@ -363,6 +363,7 @@ class GenerateSchema: MultiHostUrl: lambda self, obj: core_schema.multi_host_url_schema(), None: lambda self, obj: core_schema.none_schema(), NoneType: lambda self, obj: core_schema.none_schema(),+ EllipsisType: lambda self, obj: self._ellipsis_schema(), MISSING: lambda self, obj: core_schema.missing_sentinel_schema(), type: lambda self, obj: self._type_schema(), dict: lambda self, obj: self._dict_schema(Any, Any),@@ -1808,6 +1809,31 @@ class GenerateSchema: else: raise PydanticSchemaGenerationError(f'Unable to generate pydantic-core schema for {pattern_type!r}.') + def _ellipsis_schema(self) -> core_schema.CoreSchema:+ """Generate schema for `EllipsisType`.++ In Python mode, only the `Ellipsis` singleton is accepted. As `Ellipsis` has no JSON+ representation, it is represented as the `'...'` string in JSON mode.+ """+ json_repr = '...'++ def ser_ellipsis(value: Any) -> str:+ if value is not Ellipsis:+ raise PydanticSerializationUnexpectedValue(+ f'Expected `ellipsis` but got `{type(value)}` with value `{value!r}` - serialized value may not be as expected.'+ )+ return json_repr++ return core_schema.json_or_python_schema(+ json_schema=core_schema.no_info_after_validator_function(+ lambda _: Ellipsis, core_schema.literal_schema([json_repr])+ ),+ python_schema=core_schema.is_instance_schema(EllipsisType),+ serialization=core_schema.plain_serializer_function_ser_schema(+ ser_ellipsis, return_schema=core_schema.literal_schema([json_repr]), when_used='json'+ ),+ )+ def _hashable_schema(self) -> core_schema.CoreSchema: return core_schema.custom_error_schema( schema=core_schema.json_or_python_schema(tests/test_types.pyindex df4d5f06..2a739993 100644@@ -20,6 +20,7 @@ from enum import Enum, IntEnum from numbers import Number from pathlib import Path from re import Pattern+from types import EllipsisType from typing import ( Annotated, Any,@@ -5326,6 +5327,100 @@ def test_none(value_type): ] +def test_ellipsis() -> None:+ ta = TypeAdapter(EllipsisType)++ assert ta.validate_python(...) is ...+ assert ta.validate_json('"..."') is ...+ assert ta.validate_strings('...') is ...+ assert ta.dump_python(...) is ...+ assert ta.dump_python(..., mode='json') == '...'+ assert ta.dump_json(...) == b'"..."'+ assert ta.json_schema() == {'const': '...', 'type': 'string'}+ assert ta.json_schema(mode='serialization') == {'const': '...', 'type': 'string'}++ for value in ('...', 'Ellipsis', None, 1):+ with pytest.raises(ValidationError) as exc_info:+ ta.validate_python(value)+ assert exc_info.value.errors(include_url=False) == [+ {+ 'type': 'is_instance_of',+ 'loc': (),+ 'msg': 'Input should be an instance of ellipsis',+ 'input': value,+ 'ctx': {'class': 'ellipsis'},+ }+ ]++ with pytest.raises(ValidationError) as exc_info:+ ta.validate_json('"Ellipsis"')+ assert exc_info.value.errors(include_url=False) == [+ {+ 'type': 'literal_error',+ 'loc': (),+ 'msg': "Input should be '...'",+ 'input': 'Ellipsis',+ 'ctx': {'expected': "'...'"},+ }+ ]+++def test_ellipsis_in_model() -> None:+ class Model(BaseModel):+ subset: tuple[int | EllipsisType, int | EllipsisType]+ optional: EllipsisType | None = None++ m = Model(subset=(1, ...), optional=...)+ assert m.subset == (1, ...)+ assert m.optional is ...+ assert m.model_dump() == {'subset': (1, ...), 'optional': ...}+ assert m.model_dump_json() == '{"subset":[1,"..."],"optional":"..."}'+ assert Model.model_validate_json(m.model_dump_json()) == m++ assert Model.model_json_schema() == {+ 'properties': {+ 'subset': {+ 'maxItems': 2,+ 'minItems': 2,+ 'prefixItems': [+ {'anyOf': [{'type': 'integer'}, {'const': '...', 'type': 'string'}]},+ {'anyOf': [{'type': 'integer'}, {'const': '...', 'type': 'string'}]},+ ],+ 'title': 'Subset',+ 'type': 'array',+ },+ 'optional': {+ 'anyOf': [{'const': '...', 'type': 'string'}, {'type': 'null'}],+ 'default': None,+ 'title': 'Optional',+ },+ },+ 'required': ['subset'],+ 'title': 'Model',+ 'type': 'object',+ }+++def test_ellipsis_union_serialization() -> None:+ class Model(BaseModel):+ a: EllipsisType | str+ b: EllipsisType | int | None+ c: list[EllipsisType | float]++ m = Model(a='hello', b=3, c=[1.5, ...])+ assert m.model_dump(mode='json') == {'a': 'hello', 'b': 3, 'c': [1.5, '...']}+ assert m.model_dump_json() == '{"a":"hello","b":3,"c":[1.5,"..."]}'+ assert Model.model_validate_json(m.model_dump_json()) == m++ with pytest.warns(UserWarning, match='Expected `ellipsis`'):+ assert TypeAdapter(EllipsisType).dump_json(1) == b'1'+++def test_ellipsis_not_a_type_argument() -> None:+ with pytest.raises(PydanticSchemaGenerationError):+ TypeAdapter(tuple[int, ..., str])++ def test_none_literal(): class Model(BaseModel): my_none: Literal[None]
docs/errors/validation_errors.mdindex b0d509b97..fc0539fef 100644@@ -724,6 +724,27 @@ except ValidationError as exc: #> 'dict_type' ``` +## `ellipsis_error`++This error is raised when the input isn't the [`Ellipsis`][] literal:++```python+from types import EllipsisType++from pydantic import BaseModel, ValidationError+++class Model(BaseModel):+ e: EllipsisType+++try:+ Model(e=1)+except ValidationError as exc:+ print(repr(exc.errors()[0]['type']))+ #> 'ellipsis_error'+```+ ## `enum` This error is raised when the input value does not exist in an `enum` field members:pydantic-core/python/pydantic_core/core_schema.pyindex c6118a082..965cf28e1 100644@@ -1453,6 +1453,25 @@ def missing_sentinel_schema( ) +class EllipsisSchema(TypedDict, total=False):+ type: Required[Literal['ellipsis']]+ metadata: dict[str, Any]+ serialization: SerSchema+++def ellipsis_schema(+ metadata: dict[str, Any] | None = None,+ serialization: SerSchema | None = None,+) -> EllipsisSchema:+ """Returns a schema for the [`Ellipsis`][] literal."""++ return _dict_not_none(+ type='ellipsis',+ metadata=metadata,+ serialization=serialization,+ )++ # must match input/parse_json.rs::JsonType::try_from JsonType = Literal['null', 'bool', 'int', 'float', 'str', 'list', 'dict'] @@ -4201,6 +4220,7 @@ if not MYPY: | TimedeltaSchema | LiteralSchema | MissingSentinelSchema+ | EllipsisSchema | EnumSchema | IsInstanceSchema | IsSubclassSchema@@ -4261,6 +4281,7 @@ CoreSchemaType: TypeAlias = Literal[ 'timedelta', 'literal', 'missing-sentinel',+ 'ellipsis', 'enum', 'is-instance', 'is-subclass',@@ -4363,6 +4384,7 @@ ErrorType: TypeAlias = Literal[ 'assertion_error', 'literal_error', 'missing_sentinel_error',+ 'ellipsis_error', 'date_type', 'date_parsing', 'date_from_datetime_parsing',pydantic-core/src/errors/types.rsindex da9cdae0c..8f8e936ce 100644@@ -324,6 +324,9 @@ error_types! { // --------------------- // missing sentinel MissingSentinelError {},+ // ---------------------+ // ellipsis+ EllipsisError {}, // date errors DateType {}, DateParsing {@@ -552,6 +555,7 @@ impl ErrorType { Self::CustomError { .. } => "", // custom errors are handled separately Self::LiteralError { .. } => "Input should be {expected}", Self::MissingSentinelError { .. } => "Input should be the 'MISSING' sentinel",+ Self::EllipsisError { .. } => "Input should be the 'Ellipsis' literal", Self::DateType { .. } => "Input should be a valid date", Self::DateParsing { .. } => "Input should be a valid date in the format YYYY-MM-DD, {error}", Self::DateFromDatetimeParsing { .. } => "Input should be a valid date or datetime, {error}",pydantic-core/src/serializers/shared.rsindex 5f7efee37..f6342c1ec 100644@@ -153,6 +153,7 @@ combined_serializer! { TaggedUnion: super::type_serializers::union::TaggedUnionSerializer; Literal: super::type_serializers::literal::LiteralSerializer; MissingSentinel: super::type_serializers::missing_sentinel::MissingSentinelSerializer;+ Ellipsis: super::type_serializers::ellipsis::EllipsisSerializer; Enum: super::type_serializers::enum_::EnumSerializer; Recursive: super::type_serializers::definitions::DefinitionRefSerializer; Tuple: super::type_serializers::tuple::TupleSerializer;@@ -385,6 +386,7 @@ impl PyGcTraverse for CombinedSerializer { CombinedSerializer::TaggedUnion(inner) => inner.py_gc_traverse(visit), CombinedSerializer::Literal(inner) => inner.py_gc_traverse(visit), CombinedSerializer::MissingSentinel(inner) => inner.py_gc_traverse(visit),+ CombinedSerializer::Ellipsis(inner) => inner.py_gc_traverse(visit), CombinedSerializer::Enum(inner) => inner.py_gc_traverse(visit), CombinedSerializer::Recursive(inner) => inner.py_gc_traverse(visit), CombinedSerializer::Tuple(inner) => inner.py_gc_traverse(visit),pydantic-core/src/serializers/type_serializers/ellipsis.rsnew file mode 100644index 000000000..634916c5d@@ -0,0 +1,74 @@+// This serializer is defined so that building a schema serializer containing an+// 'ellipsis' core schema doesn't crash. In practice, the serializer can be used+// if the 'ellipsis' core schema is used standalone (e.g. with a Pydantic type+// adapter), but this isn't something we explicitly support.++use std::borrow::Cow;+use std::sync::{Arc, LazyLock};++use pyo3::prelude::*;+use pyo3::types::{PyDict, PyEllipsis};++use serde::ser::Error;++use crate::PydanticSerializationUnexpectedValue;+use crate::definitions::DefinitionsBuilder;+use crate::serializers::SerializationState;++use super::{BuildSerializer, CombinedSerializer, TypeSerializer};++#[derive(Debug)]+pub struct EllipsisSerializer {}++static ELLIPSIS_SERIALIZER: LazyLock<Arc<CombinedSerializer>> =+ LazyLock::new(|| Arc::new(EllipsisSerializer {}.into()));++impl BuildSerializer for EllipsisSerializer {+ const EXPECTED_TYPE: &'static str = "ellipsis";++ fn build(+ _schema: &Bound<'_, PyDict>,+ _config: Option<&Bound<'_, PyDict>>,+ _definitions: &mut DefinitionsBuilder<Arc<CombinedSerializer>>,+ ) -> PyResult<Arc<CombinedSerializer>> {+ Ok(ELLIPSIS_SERIALIZER.clone())+ }+}++impl_py_gc_traverse!(EllipsisSerializer {});++impl TypeSerializer for EllipsisSerializer {+ fn to_python(&self, value: &Bound<'_, PyAny>, _state: &mut SerializationState<'_>) -> PyResult<Py<PyAny>> {+ let ellipsis = PyEllipsis::get(value.py());++ if value.is(ellipsis) {+ Ok(ellipsis.to_owned().into_any().unbind())+ } else {+ Err(+ PydanticSerializationUnexpectedValue::new_from_msg(Some("Expected 'Ellipsis' object".to_string()))+ .to_py_err(),+ )+ }+ }++ fn json_key<'a, 'py>(+ &self,+ key: &'a Bound<'py, PyAny>,+ state: &mut SerializationState<'py>,+ ) -> PyResult<Cow<'a, str>> {+ self.invalid_as_json_key(key, state, Self::EXPECTED_TYPE)+ }++ fn serde_serialize<S: serde::ser::Serializer>(+ &self,+ _value: &Bound<'_, PyAny>,+ _serializer: S,+ _state: &mut SerializationState<'_>,+ ) -> Result<S::Ok, S::Error> {+ Err(Error::custom("'Ellipsis' can't be serialized to JSON".to_string()))+ }++ fn get_name(&self) -> &str {+ Self::EXPECTED_TYPE+ }+}pydantic-core/src/serializers/type_serializers/mod.rsindex da95113ea..ad664e8f8 100644@@ -6,6 +6,7 @@ pub mod datetime_etc; pub mod decimal; pub mod definitions; pub mod dict;+pub mod ellipsis; pub mod enum_; pub mod float; pub mod format;pydantic-core/src/validators/ellipsis.rsnew file mode 100644index 000000000..d3dfa9faf@@ -0,0 +1,48 @@+use core::fmt::Debug;+use std::sync::{Arc, LazyLock};++use pyo3::prelude::*;+use pyo3::types::{PyDict, PyEllipsis};++use crate::errors::{ErrorType, ValError, ValResult};+use crate::input::Input;++use super::{BuildValidator, CombinedValidator, DefinitionsBuilder, ValidationState, Validator};++#[derive(Debug, Clone)]+pub struct EllipsisValidator {}++static ELLIPSIS_VALIDATOR: LazyLock<Arc<CombinedValidator>> =+ LazyLock::new(|| CombinedValidator::Ellipsis(EllipsisValidator {}).into());++impl BuildValidator for EllipsisValidator {+ const EXPECTED_TYPE: &'static str = "ellipsis";++ fn build(+ _schema: &Bound<'_, PyDict>,+ _config: Option<&Bound<'_, PyDict>>,+ _definitions: &mut DefinitionsBuilder<Arc<CombinedValidator>>,+ ) -> PyResult<Arc<CombinedValidator>> {+ Ok(ELLIPSIS_VALIDATOR.clone())+ }+}++impl_py_gc_traverse!(EllipsisValidator {});++impl Validator for EllipsisValidator {+ fn validate<'py>(+ &self,+ py: Python<'py>,+ input: &(impl Input<'py> + ?Sized),+ _state: &mut ValidationState<'_, 'py>,+ ) -> ValResult<Py<PyAny>> {+ match input.as_python() {+ Some(v) if v.is(PyEllipsis::get(py)) => Ok(v.to_owned().into()),+ _ => Err(ValError::new(ErrorType::EllipsisError { context: None }, input)),+ }+ }++ fn get_name(&self) -> &str {+ Self::EXPECTED_TYPE+ }+}pydantic-core/src/validators/mod.rsindex 2ab41e6b0..7a6875e3d 100644@@ -37,6 +37,7 @@ mod datetime; pub(crate) mod decimal; mod definitions; mod dict;+mod ellipsis; mod enum_; mod float; pub(crate) mod fraction;@@ -610,6 +611,8 @@ fn build_validator_inner( literal::LiteralValidator, // missing sentinel missing_sentinel::MissingSentinelValidator,+ // ellipsis+ ellipsis::EllipsisValidator, // enums enum_::BuildEnumValidator, // any@@ -775,6 +778,8 @@ pub enum CombinedValidator { Literal(literal::LiteralValidator), // Missing sentinel MissingSentinel(missing_sentinel::MissingSentinelValidator),+ // Ellipsis+ Ellipsis(ellipsis::EllipsisValidator), // enums IntEnum(enum_::EnumValidator<enum_::IntEnumValidator>), StrEnum(enum_::EnumValidator<enum_::StrEnumValidator>),pydantic-core/tests/test_errors.pyindex 274ecb3f6..334e281bd 100644@@ -347,6 +347,7 @@ all_errors = [ ('literal_error', 'Input should be foo', {'expected': 'foo'}), ('literal_error', 'Input should be foo or bar', {'expected': 'foo or bar'}), ('missing_sentinel_error', "Input should be the 'MISSING' sentinel", None),+ ('ellipsis_error', "Input should be the 'Ellipsis' literal", None), ('date_type', 'Input should be a valid date', None), ('date_parsing', 'Input should be a valid date in the format YYYY-MM-DD, foobar', {'error': 'foobar'}), ('date_from_datetime_parsing', 'Input should be a valid date or datetime, foobar', {'error': 'foobar'}),pydantic-core/tests/test_schema_functions.pyindex 299b08e3f..65a015a6f 100644@@ -83,6 +83,7 @@ all_schema_functions = [ ), (core_schema.literal_schema, args(['a', 'b']), {'type': 'literal', 'expected': ['a', 'b']}), (core_schema.missing_sentinel_schema, args(), {'type': 'missing-sentinel'}),+ (core_schema.ellipsis_schema, args(), {'type': 'ellipsis'}), ( core_schema.enum_schema, args(MyEnum, list(MyEnum.__members__.values())),pydantic/_internal/_generate_schema.pyindex 3fba8dc3e..3436338cc 100644@@ -23,7 +23,7 @@ from inspect import Parameter, _ParameterKind from ipaddress import IPv4Address, IPv4Interface, IPv4Network, IPv6Address, IPv6Interface, IPv6Network from itertools import chain from operator import attrgetter-from types import FunctionType, GenericAlias, LambdaType, MethodType, NoneType+from types import EllipsisType, FunctionType, GenericAlias, LambdaType, MethodType, NoneType from typing import ( TYPE_CHECKING, Any,@@ -364,6 +364,7 @@ class GenerateSchema: None: lambda self, obj: core_schema.none_schema(), NoneType: lambda self, obj: core_schema.none_schema(), MISSING: lambda self, obj: core_schema.missing_sentinel_schema(),+ EllipsisType: lambda self, obj: core_schema.ellipsis_schema(), type: lambda self, obj: self._type_schema(), dict: lambda self, obj: self._dict_schema(Any, Any), tuple: lambda self, obj: self._tuple_schema(obj),pydantic/json_schema.pyindex 27756c099..c6adb6947 100644@@ -907,6 +907,19 @@ class GenerateJsonSchema: """ raise PydanticOmit + def ellipsis_schema(self, schema: core_schema.EllipsisSchema) -> JsonSchemaValue:+ """Handles JSON schema generation for a core schema that checks if a value is the [`Ellipsis`][] literal.++ Unless overridden in a subclass, this raises an error.++ Args:+ schema: The core schema.++ Returns:+ The generated JSON schema.+ """+ return self.handle_invalid_for_json_schema(schema, 'core_schema.EllipsisSchema')+ def enum_schema(self, schema: core_schema.EnumSchema) -> JsonSchemaValue: """Generates a JSON schema that matches an Enum value. tests/test_json_schema.pyindex 2486dfdd8..633abea77 100644@@ -12,6 +12,7 @@ from enum import Enum, IntEnum from ipaddress import IPv4Address, IPv4Interface, IPv4Network, IPv6Address, IPv6Interface, IPv6Network from pathlib import Path from re import Pattern+from types import EllipsisType from typing import ( Annotated, Any,@@ -1319,6 +1320,14 @@ def test_callable_type(type_, default_value, base_json_schema, properties): assert model_schema['properties'] == properties +def test_ellipsis_schema() -> None:+ class Model(BaseModel):+ e: EllipsisType++ with pytest.raises(PydanticInvalidForJsonSchema):+ Model.model_json_schema()++ @pytest.mark.parametrize( 'default_value,properties', (tests/types/test_ellipsis.pynew file mode 100644index 000000000..11b7fdea7@@ -0,0 +1,21 @@+from types import EllipsisType++import pytest+from pydantic_core import PydanticSerializationError, PydanticSerializationUnexpectedValue++from pydantic import TypeAdapter+++def test_ellipsis_literal() -> None:+ ta = TypeAdapter(EllipsisType)++ assert ta.validate_python(...) is ...+ assert ta.dump_python(...) is ...++ with pytest.raises(PydanticSerializationUnexpectedValue):+ ta.dump_python(1)++ with pytest.raises(+ PydanticSerializationError, match="Error serializing to JSON: 'Ellipsis' can't be serialized to JSON"+ ):+ ta.dump_json(...)