Backtestspydanticrun-20261009-110034#13467
Support Ellipsis
Our patch was judged better than the merged fix.
In this judgment, Patch A = our agent, Patch B = the merged fix.
Patch A is the more complete and useful fix. It maps `EllipsisType` to a schema built from existing pydantic-core primitives. Validation accepts the `Ellipsis` singleton in Python mode, plus the `'...'` string in lax mode, and accepts `'...'` from JSON. Serialization keeps `Ellipsis` in Python mode and emits `'...'` in JSON mode. JSON schema output is `{'const': '...', 'type': 'string'}`. As a result, the issue's scenario (`tuple[int | EllipsisType, int | EllipsisType]` in a model) fully round-trips: `model_dump`, `model_dump_json`, `model_validate_json` and `model_json_schema` all work. Its tests exercise these paths, including:
- strict vs lax behaviour
- custom error type and message
- union serialization with `int` and `str`
- the warning on unexpected values
The tests would fail before the change. It also documents the type and warns about the `...`-as-default pitfall.
Patch B adds a new Rust validator/serializer and core schema type, but support is Python-only:
- JSON validation always fails, because the validator only matches `as_python()`.
- `dump_json` raises an error.
- `model_json_schema()` raises `PydanticInvalidForJsonSchema`.
A model like the one in the issue therefore can't be serialized to JSON or have its schema generated. That is a significant gap for a pydantic type. B's tests mostly confirm these limitations, plus error and schema-function registration. B's approach also brings much more churn and risk (new core schema type, error type, Rust code) for less functionality.
A's approach is lighter, consistent with how other special types like `Hashable` are built in `_generate_schema.py`, and covers the issue's needs more fully.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
8m 21s
Wall-clock time in the container
Full test suite
No new failures
12,308 passed · 499 skipped · 12,807 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
141vs 225 merged
Changed lines, ours vs merged
The issue
View on GitHubInitial Checks
- I have searched Google & GitHub for similar requests and couldn't find anything
- I have read and followed the docs and still think this feature is missing
Description
Ellipsis is a builtin type that pydantic does not support. Its usage is infrequent, but it can be useful to express placeholders:
This is a bit of a contrived example, but:
class Model(pydantic.BaseModel):
m: list[list[float]]
subset: tuple[int | Ellipsis, int | Ellipsis]
def subset_arr(self) -> np.NDArray:
return np.asarray(self.m)[*self.subset]
Here's how I implement this right now with an adapter: https://github.com/psalvaggio/scientific_pydantic/blob/main/src/scientific_pydantic/ellipsis.py
However, since this is a standard library type, I think this would be a good thing to live in pydantic itself.
Affected Components
- Compatibility between releases
- Data validation/parsing
- Data serialization -
.model_dump()and.model_dump_json() - JSON Schema
- Dataclasses
- Model Config
- Field Types - adding or changing a particular data type
- Function validation decorator
- Generic Models
- Other Model behaviour -
model_construct(), pickling, private attributes, ORM mode - Plugins and integration with other tools - mypy, FastAPI, python-devtools, Hypothesis, VS Code, PyCharm, etc.
The two patches
docs/api/standard_library_types.mdindex 4739bdb4..7225fb3a 100644@@ -756,6 +756,41 @@ Supported types: [`None`][], [`NoneType`][types.NoneType] or `Literal[None]` (th Allows only `None` as a value. +## Ellipsis++Supported type: [`EllipsisType`][types.EllipsisType].++<h4>Validation</h4>++* In Python mode, the [`Ellipsis`][] (`...`) singleton is accepted. In [lax mode](../concepts/strict_mode.md),+ the `'...'` string is also accepted.+* In JSON mode, the `'...'` string is accepted.++<h4>Serialization</h4>++In Python mode, the `Ellipsis` singleton is returned as is. In JSON mode, it is serialized as the `'...'` string.++```python+from types import EllipsisType++from pydantic import BaseModel+++class Model(BaseModel):+ subset: tuple[int | EllipsisType, int | EllipsisType]+++m = Model(subset=(..., 1))+print(m.model_dump_json())+#> {"subset":["...",1]}+print(Model.model_validate_json('{"subset":[0,"..."]}'))+#> subset=(0, Ellipsis)+```++!!! warning+ Using `...` as the default value of a field marks the field as required (see [default values](../concepts/fields.md#default-values)),+ so `Ellipsis` can't be used as a default value for an `EllipsisType` field.+ ## Generic collection types Pydantic supports a wide variety of generic collection types, both built-ins (such as [`list`][]) and abstract base classespydantic/_internal/_generate_schema.pyindex 3fba8dc3..22bed789 100644@@ -23,7 +23,7 @@ from inspect import Parameter, _ParameterKind from ipaddress import IPv4Address, IPv4Interface, IPv4Network, IPv6Address, IPv6Interface, IPv6Network from itertools import chain from operator import attrgetter-from types import FunctionType, GenericAlias, LambdaType, MethodType, NoneType+from types import EllipsisType, FunctionType, GenericAlias, LambdaType, MethodType, NoneType from typing import ( TYPE_CHECKING, Any,@@ -363,6 +363,7 @@ class GenerateSchema: MultiHostUrl: lambda self, obj: core_schema.multi_host_url_schema(), None: lambda self, obj: core_schema.none_schema(), NoneType: lambda self, obj: core_schema.none_schema(),+ EllipsisType: lambda self, obj: self._ellipsis_schema(), MISSING: lambda self, obj: core_schema.missing_sentinel_schema(), type: lambda self, obj: self._type_schema(), dict: lambda self, obj: self._dict_schema(Any, Any),@@ -1808,6 +1809,35 @@ class GenerateSchema: else: raise PydanticSchemaGenerationError(f'Unable to generate pydantic-core schema for {pattern_type!r}.') + def _ellipsis_schema(self) -> core_schema.CoreSchema:+ """Generate schema for `EllipsisType`, represented as the `'...'` string in JSON."""+ str_schema = core_schema.no_info_after_validator_function(+ lambda _: Ellipsis, core_schema.literal_schema(['...'])+ )+ instance_schema = core_schema.is_instance_schema(EllipsisType)++ def ser_ellipsis(value: Any) -> str:+ if value is not Ellipsis:+ raise PydanticSerializationUnexpectedValue(+ f"Expected `{EllipsisType}` but got `{type(value)}` with value `'{value}'` - serialized value may not be as expected."+ )+ return '...'++ return core_schema.custom_error_schema(+ schema=core_schema.json_or_python_schema(+ json_schema=str_schema,+ python_schema=core_schema.lax_or_strict_schema(+ lax_schema=core_schema.union_schema([instance_schema, str_schema]),+ strict_schema=instance_schema,+ ),+ serialization=core_schema.plain_serializer_function_ser_schema(+ ser_ellipsis, when_used='json', return_schema=core_schema.literal_schema(['...'])+ ),+ ),+ custom_error_type='ellipsis_type',+ custom_error_message='Input should be an ellipsis',+ )+ def _hashable_schema(self) -> core_schema.CoreSchema: return core_schema.custom_error_schema( schema=core_schema.json_or_python_schema(tests/test_types.pyindex df4d5f06..4565e898 100644@@ -7323,3 +7323,77 @@ def test_string_constraints_ascii_only() -> None: 'msg': 'String should contain only ASCII characters', 'input': 'caf\xe9', }+++def test_ellipsis_type() -> None:+ from types import EllipsisType++ class Model(BaseModel):+ subset: tuple[int | EllipsisType, int | EllipsisType]++ m = Model(subset=(..., 1))+ assert m.subset == (..., 1)+ assert m.model_dump() == {'subset': (..., 1)}+ assert m.model_dump(mode='json') == {'subset': ['...', 1]}+ assert m.model_dump_json() == '{"subset":["...",1]}'+ assert Model.model_validate_json(m.model_dump_json()).subset == (..., 1)+ assert Model.model_validate(m.model_dump(mode='json')).subset == (..., 1)++ assert Model.model_json_schema() == {+ 'properties': {+ 'subset': {+ 'maxItems': 2,+ 'minItems': 2,+ 'prefixItems': [+ {'anyOf': [{'type': 'integer'}, {'const': '...', 'type': 'string'}]},+ {'anyOf': [{'type': 'integer'}, {'const': '...', 'type': 'string'}]},+ ],+ 'title': 'Subset',+ 'type': 'array',+ }+ },+ 'required': ['subset'],+ 'title': 'Model',+ 'type': 'object',+ }+++def test_ellipsis_type_validation_errors() -> None:+ from types import EllipsisType++ ta = TypeAdapter(EllipsisType)+ assert ta.validate_python(...) is Ellipsis+ assert ta.validate_python('...') is Ellipsis+ assert ta.validate_json('"..."') is Ellipsis+ assert ta.validate_python(..., strict=True) is Ellipsis+ assert ta.validate_json('"..."', strict=True) is Ellipsis+ assert ta.dump_python(...) is Ellipsis+ assert ta.dump_json(...) == b'"..."'+ assert ta.json_schema(mode='serialization') == {'const': '...', 'type': 'string'}++ for validate in (+ lambda: ta.validate_python('..'),+ lambda: ta.validate_python(None),+ lambda: ta.validate_python('...', strict=True),+ lambda: ta.validate_json('null'),+ ):+ with pytest.raises(ValidationError) as exc_info:+ validate()+ assert exc_info.value.errors(include_url=False)[0]['type'] == 'ellipsis_type'+ assert exc_info.value.errors(include_url=False)[0]['msg'] == 'Input should be an ellipsis'+++def test_ellipsis_type_union_serialization() -> None:+ from types import EllipsisType++ ta = TypeAdapter(EllipsisType | int)+ assert ta.dump_json(1) == b'1'+ assert ta.dump_json(...) == b'"..."'+ assert ta.dump_python(1, mode='json') == 1++ ta_str = TypeAdapter(EllipsisType | str)+ assert ta_str.dump_json('x') == b'"x"'+ assert ta_str.dump_json(...) == b'"..."'++ with pytest.warns(UserWarning, match='Expected `<class \'ellipsis\'>`'):+ assert TypeAdapter(EllipsisType).dump_json(1) == b'1'
docs/errors/validation_errors.mdindex b0d509b97..fc0539fef 100644@@ -724,6 +724,27 @@ except ValidationError as exc: #> 'dict_type' ``` +## `ellipsis_error`++This error is raised when the input isn't the [`Ellipsis`][] literal:++```python+from types import EllipsisType++from pydantic import BaseModel, ValidationError+++class Model(BaseModel):+ e: EllipsisType+++try:+ Model(e=1)+except ValidationError as exc:+ print(repr(exc.errors()[0]['type']))+ #> 'ellipsis_error'+```+ ## `enum` This error is raised when the input value does not exist in an `enum` field members:pydantic-core/python/pydantic_core/core_schema.pyindex c6118a082..965cf28e1 100644@@ -1453,6 +1453,25 @@ def missing_sentinel_schema( ) +class EllipsisSchema(TypedDict, total=False):+ type: Required[Literal['ellipsis']]+ metadata: dict[str, Any]+ serialization: SerSchema+++def ellipsis_schema(+ metadata: dict[str, Any] | None = None,+ serialization: SerSchema | None = None,+) -> EllipsisSchema:+ """Returns a schema for the [`Ellipsis`][] literal."""++ return _dict_not_none(+ type='ellipsis',+ metadata=metadata,+ serialization=serialization,+ )++ # must match input/parse_json.rs::JsonType::try_from JsonType = Literal['null', 'bool', 'int', 'float', 'str', 'list', 'dict'] @@ -4201,6 +4220,7 @@ if not MYPY: | TimedeltaSchema | LiteralSchema | MissingSentinelSchema+ | EllipsisSchema | EnumSchema | IsInstanceSchema | IsSubclassSchema@@ -4261,6 +4281,7 @@ CoreSchemaType: TypeAlias = Literal[ 'timedelta', 'literal', 'missing-sentinel',+ 'ellipsis', 'enum', 'is-instance', 'is-subclass',@@ -4363,6 +4384,7 @@ ErrorType: TypeAlias = Literal[ 'assertion_error', 'literal_error', 'missing_sentinel_error',+ 'ellipsis_error', 'date_type', 'date_parsing', 'date_from_datetime_parsing',pydantic-core/src/errors/types.rsindex da9cdae0c..8f8e936ce 100644@@ -324,6 +324,9 @@ error_types! { // --------------------- // missing sentinel MissingSentinelError {},+ // ---------------------+ // ellipsis+ EllipsisError {}, // date errors DateType {}, DateParsing {@@ -552,6 +555,7 @@ impl ErrorType { Self::CustomError { .. } => "", // custom errors are handled separately Self::LiteralError { .. } => "Input should be {expected}", Self::MissingSentinelError { .. } => "Input should be the 'MISSING' sentinel",+ Self::EllipsisError { .. } => "Input should be the 'Ellipsis' literal", Self::DateType { .. } => "Input should be a valid date", Self::DateParsing { .. } => "Input should be a valid date in the format YYYY-MM-DD, {error}", Self::DateFromDatetimeParsing { .. } => "Input should be a valid date or datetime, {error}",pydantic-core/src/serializers/shared.rsindex 5f7efee37..f6342c1ec 100644@@ -153,6 +153,7 @@ combined_serializer! { TaggedUnion: super::type_serializers::union::TaggedUnionSerializer; Literal: super::type_serializers::literal::LiteralSerializer; MissingSentinel: super::type_serializers::missing_sentinel::MissingSentinelSerializer;+ Ellipsis: super::type_serializers::ellipsis::EllipsisSerializer; Enum: super::type_serializers::enum_::EnumSerializer; Recursive: super::type_serializers::definitions::DefinitionRefSerializer; Tuple: super::type_serializers::tuple::TupleSerializer;@@ -385,6 +386,7 @@ impl PyGcTraverse for CombinedSerializer { CombinedSerializer::TaggedUnion(inner) => inner.py_gc_traverse(visit), CombinedSerializer::Literal(inner) => inner.py_gc_traverse(visit), CombinedSerializer::MissingSentinel(inner) => inner.py_gc_traverse(visit),+ CombinedSerializer::Ellipsis(inner) => inner.py_gc_traverse(visit), CombinedSerializer::Enum(inner) => inner.py_gc_traverse(visit), CombinedSerializer::Recursive(inner) => inner.py_gc_traverse(visit), CombinedSerializer::Tuple(inner) => inner.py_gc_traverse(visit),pydantic-core/src/serializers/type_serializers/ellipsis.rsnew file mode 100644index 000000000..634916c5d@@ -0,0 +1,74 @@+// This serializer is defined so that building a schema serializer containing an+// 'ellipsis' core schema doesn't crash. In practice, the serializer can be used+// if the 'ellipsis' core schema is used standalone (e.g. with a Pydantic type+// adapter), but this isn't something we explicitly support.++use std::borrow::Cow;+use std::sync::{Arc, LazyLock};++use pyo3::prelude::*;+use pyo3::types::{PyDict, PyEllipsis};++use serde::ser::Error;++use crate::PydanticSerializationUnexpectedValue;+use crate::definitions::DefinitionsBuilder;+use crate::serializers::SerializationState;++use super::{BuildSerializer, CombinedSerializer, TypeSerializer};++#[derive(Debug)]+pub struct EllipsisSerializer {}++static ELLIPSIS_SERIALIZER: LazyLock<Arc<CombinedSerializer>> =+ LazyLock::new(|| Arc::new(EllipsisSerializer {}.into()));++impl BuildSerializer for EllipsisSerializer {+ const EXPECTED_TYPE: &'static str = "ellipsis";++ fn build(+ _schema: &Bound<'_, PyDict>,+ _config: Option<&Bound<'_, PyDict>>,+ _definitions: &mut DefinitionsBuilder<Arc<CombinedSerializer>>,+ ) -> PyResult<Arc<CombinedSerializer>> {+ Ok(ELLIPSIS_SERIALIZER.clone())+ }+}++impl_py_gc_traverse!(EllipsisSerializer {});++impl TypeSerializer for EllipsisSerializer {+ fn to_python(&self, value: &Bound<'_, PyAny>, _state: &mut SerializationState<'_>) -> PyResult<Py<PyAny>> {+ let ellipsis = PyEllipsis::get(value.py());++ if value.is(ellipsis) {+ Ok(ellipsis.to_owned().into_any().unbind())+ } else {+ Err(+ PydanticSerializationUnexpectedValue::new_from_msg(Some("Expected 'Ellipsis' object".to_string()))+ .to_py_err(),+ )+ }+ }++ fn json_key<'a, 'py>(+ &self,+ key: &'a Bound<'py, PyAny>,+ state: &mut SerializationState<'py>,+ ) -> PyResult<Cow<'a, str>> {+ self.invalid_as_json_key(key, state, Self::EXPECTED_TYPE)+ }++ fn serde_serialize<S: serde::ser::Serializer>(+ &self,+ _value: &Bound<'_, PyAny>,+ _serializer: S,+ _state: &mut SerializationState<'_>,+ ) -> Result<S::Ok, S::Error> {+ Err(Error::custom("'Ellipsis' can't be serialized to JSON".to_string()))+ }++ fn get_name(&self) -> &str {+ Self::EXPECTED_TYPE+ }+}pydantic-core/src/serializers/type_serializers/mod.rsindex da95113ea..ad664e8f8 100644@@ -6,6 +6,7 @@ pub mod datetime_etc; pub mod decimal; pub mod definitions; pub mod dict;+pub mod ellipsis; pub mod enum_; pub mod float; pub mod format;pydantic-core/src/validators/ellipsis.rsnew file mode 100644index 000000000..d3dfa9faf@@ -0,0 +1,48 @@+use core::fmt::Debug;+use std::sync::{Arc, LazyLock};++use pyo3::prelude::*;+use pyo3::types::{PyDict, PyEllipsis};++use crate::errors::{ErrorType, ValError, ValResult};+use crate::input::Input;++use super::{BuildValidator, CombinedValidator, DefinitionsBuilder, ValidationState, Validator};++#[derive(Debug, Clone)]+pub struct EllipsisValidator {}++static ELLIPSIS_VALIDATOR: LazyLock<Arc<CombinedValidator>> =+ LazyLock::new(|| CombinedValidator::Ellipsis(EllipsisValidator {}).into());++impl BuildValidator for EllipsisValidator {+ const EXPECTED_TYPE: &'static str = "ellipsis";++ fn build(+ _schema: &Bound<'_, PyDict>,+ _config: Option<&Bound<'_, PyDict>>,+ _definitions: &mut DefinitionsBuilder<Arc<CombinedValidator>>,+ ) -> PyResult<Arc<CombinedValidator>> {+ Ok(ELLIPSIS_VALIDATOR.clone())+ }+}++impl_py_gc_traverse!(EllipsisValidator {});++impl Validator for EllipsisValidator {+ fn validate<'py>(+ &self,+ py: Python<'py>,+ input: &(impl Input<'py> + ?Sized),+ _state: &mut ValidationState<'_, 'py>,+ ) -> ValResult<Py<PyAny>> {+ match input.as_python() {+ Some(v) if v.is(PyEllipsis::get(py)) => Ok(v.to_owned().into()),+ _ => Err(ValError::new(ErrorType::EllipsisError { context: None }, input)),+ }+ }++ fn get_name(&self) -> &str {+ Self::EXPECTED_TYPE+ }+}pydantic-core/src/validators/mod.rsindex 2ab41e6b0..7a6875e3d 100644@@ -37,6 +37,7 @@ mod datetime; pub(crate) mod decimal; mod definitions; mod dict;+mod ellipsis; mod enum_; mod float; pub(crate) mod fraction;@@ -610,6 +611,8 @@ fn build_validator_inner( literal::LiteralValidator, // missing sentinel missing_sentinel::MissingSentinelValidator,+ // ellipsis+ ellipsis::EllipsisValidator, // enums enum_::BuildEnumValidator, // any@@ -775,6 +778,8 @@ pub enum CombinedValidator { Literal(literal::LiteralValidator), // Missing sentinel MissingSentinel(missing_sentinel::MissingSentinelValidator),+ // Ellipsis+ Ellipsis(ellipsis::EllipsisValidator), // enums IntEnum(enum_::EnumValidator<enum_::IntEnumValidator>), StrEnum(enum_::EnumValidator<enum_::StrEnumValidator>),pydantic-core/tests/test_errors.pyindex 274ecb3f6..334e281bd 100644@@ -347,6 +347,7 @@ all_errors = [ ('literal_error', 'Input should be foo', {'expected': 'foo'}), ('literal_error', 'Input should be foo or bar', {'expected': 'foo or bar'}), ('missing_sentinel_error', "Input should be the 'MISSING' sentinel", None),+ ('ellipsis_error', "Input should be the 'Ellipsis' literal", None), ('date_type', 'Input should be a valid date', None), ('date_parsing', 'Input should be a valid date in the format YYYY-MM-DD, foobar', {'error': 'foobar'}), ('date_from_datetime_parsing', 'Input should be a valid date or datetime, foobar', {'error': 'foobar'}),pydantic-core/tests/test_schema_functions.pyindex 299b08e3f..65a015a6f 100644@@ -83,6 +83,7 @@ all_schema_functions = [ ), (core_schema.literal_schema, args(['a', 'b']), {'type': 'literal', 'expected': ['a', 'b']}), (core_schema.missing_sentinel_schema, args(), {'type': 'missing-sentinel'}),+ (core_schema.ellipsis_schema, args(), {'type': 'ellipsis'}), ( core_schema.enum_schema, args(MyEnum, list(MyEnum.__members__.values())),pydantic/_internal/_generate_schema.pyindex 3fba8dc3e..3436338cc 100644@@ -23,7 +23,7 @@ from inspect import Parameter, _ParameterKind from ipaddress import IPv4Address, IPv4Interface, IPv4Network, IPv6Address, IPv6Interface, IPv6Network from itertools import chain from operator import attrgetter-from types import FunctionType, GenericAlias, LambdaType, MethodType, NoneType+from types import EllipsisType, FunctionType, GenericAlias, LambdaType, MethodType, NoneType from typing import ( TYPE_CHECKING, Any,@@ -364,6 +364,7 @@ class GenerateSchema: None: lambda self, obj: core_schema.none_schema(), NoneType: lambda self, obj: core_schema.none_schema(), MISSING: lambda self, obj: core_schema.missing_sentinel_schema(),+ EllipsisType: lambda self, obj: core_schema.ellipsis_schema(), type: lambda self, obj: self._type_schema(), dict: lambda self, obj: self._dict_schema(Any, Any), tuple: lambda self, obj: self._tuple_schema(obj),pydantic/json_schema.pyindex 27756c099..c6adb6947 100644@@ -907,6 +907,19 @@ class GenerateJsonSchema: """ raise PydanticOmit + def ellipsis_schema(self, schema: core_schema.EllipsisSchema) -> JsonSchemaValue:+ """Handles JSON schema generation for a core schema that checks if a value is the [`Ellipsis`][] literal.++ Unless overridden in a subclass, this raises an error.++ Args:+ schema: The core schema.++ Returns:+ The generated JSON schema.+ """+ return self.handle_invalid_for_json_schema(schema, 'core_schema.EllipsisSchema')+ def enum_schema(self, schema: core_schema.EnumSchema) -> JsonSchemaValue: """Generates a JSON schema that matches an Enum value. tests/test_json_schema.pyindex 2486dfdd8..633abea77 100644@@ -12,6 +12,7 @@ from enum import Enum, IntEnum from ipaddress import IPv4Address, IPv4Interface, IPv4Network, IPv6Address, IPv6Interface, IPv6Network from pathlib import Path from re import Pattern+from types import EllipsisType from typing import ( Annotated, Any,@@ -1319,6 +1320,14 @@ def test_callable_type(type_, default_value, base_json_schema, properties): assert model_schema['properties'] == properties +def test_ellipsis_schema() -> None:+ class Model(BaseModel):+ e: EllipsisType++ with pytest.raises(PydanticInvalidForJsonSchema):+ Model.model_json_schema()++ @pytest.mark.parametrize( 'default_value,properties', (tests/types/test_ellipsis.pynew file mode 100644index 000000000..11b7fdea7@@ -0,0 +1,21 @@+from types import EllipsisType++import pytest+from pydantic_core import PydanticSerializationError, PydanticSerializationUnexpectedValue++from pydantic import TypeAdapter+++def test_ellipsis_literal() -> None:+ ta = TypeAdapter(EllipsisType)++ assert ta.validate_python(...) is ...+ assert ta.dump_python(...) is ...++ with pytest.raises(PydanticSerializationUnexpectedValue):+ ta.dump_python(1)++ with pytest.raises(+ PydanticSerializationError, match="Error serializing to JSON: 'Ellipsis' can't be serialized to JSON"+ ):+ ta.dump_json(...)