Backtestspydanticrun-20261003-150343#13520
Generic model parametrization drops field defaults and typevar substitution after __annotations__ is read on a metaclass (Python <= 3.13)
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Both patches make the same code change. In `safe_get_annotations()` on Python < 3.14, they read `obj.__dict__.get('__annotations__', {})` for classes and keep `getattr` for non-class objects. This matches the issue's proposed fix and addresses the root cause: the descriptor shadowing no longer leaks a parent's annotations into a submodel. So correctness is equal.
The tests are the difference.
- **Patch A** adds one unit test on a custom metaclass. It reads `Meta.__annotations__` and asserts that `safe_get_annotations(Sub) == {}`. This would fail before the fix and pass after, and it is well isolated because it never touches global metaclasses. But it only checks the helper. It does not exercise the reported symptom (dropped defaults and missing typevar substitution in generic parametrization). Its docstring is also just a placeholder.
- **Patch B** tests the actual reported behavior end to end:
- It parametrizes over both triggers named in the issue, `ModelMetaclass` and `ABCMeta`.
- It checks that `G[str]` keeps the default `a=5` and substitutes `x` to `str`, and that `G[int]` validation fails on bad input.
- It also covers non-generic and generic subclasses of a parametrized parent.
- Its fixture undoes the global pollution afterwards, which keeps the suite order-independent; order dependence was the confusing part of the original bug.
These tests would fail before the fix on Python ≤ 3.13.
On craft, Patch B adds a clear comment explaining why `getattr` is unsafe here. That matters for a subtle CPython quirk that someone might otherwise 'simplify' back. Patch A's nested if/else inside the else branch is slightly clunkier than B's `elif`.
With identical fixes, Patch B wins on more faithful and thorough regression tests and better documentation.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
11m 6s
Wall-clock time in the container
Full test suite
No new failures
12,277 passed · 501 skipped · 12,778 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
65vs 26 merged
Changed lines, ours vs merged
The issue
View on GitHubInitial Checks
- I confirm that I'm using Pydantic V2
Description
This is a regression on main (2.14.0a1); I could not reproduce it on any released version (checked 2.13.4 and 2.10.6 under the same conditions). It affects Python 3.10–3.13 (verified on 3.10, 3.12 and 3.13); Python 3.14 is unaffected.
If __annotations__ has ever been read on a metaclass appearing in a model's metatype MRO (e.g. ModelMetaclass itself, or abc.ABCMeta), then any generic model parametrized for the first time afterwards gets a broken schema:
- type variables are not substituted (
x: Tends up with ananyschema), and - field defaults are dropped (fields with defaults become required).
Parametrizations built before the read keep working, since their schemas are already cached — which makes this very order-dependent and confusing to track down in a real test suite.
Reading an attribute sounds like an odd trigger, but it happens in the wild: on CPython ≤ 3.13, reading __annotations__ on a (heap) class that has none auto-creates a plain __annotations__ = {} entry in that class's __dict__ (see the annotations HOWTO). langgraph does such a read at import time (get_cached_annotated_keys in langgraph/_internal/_fields.py has a getattr(base, "__annotations__", None) fallback — added for Python 3.14 compat, ironically — which ends up planting {} on abc.ABCMeta). So a plain import langgraph.prebuilt breaks all subsequently-parametrized generic models. We found this via the pydantic/logfire weekly CI job testing against pydantic main, where pydantic_evals.Dataset.from_dict() fails with 3 validation errors for _DatasetModel[...] — Field required for exactly its three defaulted fields — but only in test runs where a langchain test ran first (failing run).
Mechanism:
- The planted
{}lives in the metaclass's__dict__. For a model classM, attribute lookup ofM.__annotations__searchestype(M).__mro__(e.g.(ModelMetaclass, ABCMeta, type, object)) and now finds the plain dict before reaching thetype.__annotations__getset descriptor. A plain dict is not a data descriptor, so lookup falls back toM's own MRO — and for a class with no own__annotations__(like a freshly created generic submodel), this returns the parent's annotations. (Normally thetype.__annotations__data descriptor intercepts the lookup and auto-creates/returns the class's own empty dict, which is what makesgetattrlook safe.) safe_get_annotations()inpydantic/_internal/_typing_extra.pyusesgetattr(obj, '__annotations__', {})on < 3.14, so for the new submodelG[str]it returns the origin's annotations instead of{}:clean: safe_get_annotations(G[str]) == {} poisoned: safe_get_annotations(G[str]) == {'x': ~T, 'a': <class 'int'>}- In
collect_model_fields(),cls_annotationsis "only used to determine if an annotation comes from a parent class". The leak makesann_name in cls_annotationstrue for every inherited field, so instead of copying the parentFieldInfoand applyingtypevars_map(via_recreate_field_info), it takes the plainFieldInfo.from_annotation(ann_type)branch: no typevar substitution, and no default — the origin's class-attribute default is long gone because it wasdelattr'd during the origin's own field collection.
Possible fix: for classes on ≤ 3.13, read own annotations via obj.__dict__.get('__annotations__', {}) (what inspect.get_annotations() does) rather than getattr. Both call sites of safe_get_annotations() seem to want own-class annotations anyway: get_model_type_hints() iterates the MRO itself and calls it per base, and cls_annotations in collect_model_fields() is explicitly meant to be the current class's own annotations.
Example Code
from typing import Generic, TypeVar
from pydantic import BaseModel
# The trigger: any read of `__annotations__` on a metaclass. On CPython <= 3.13 this
# auto-creates a plain `__annotations__ = {}` entry in the metaclass's `__dict__`,
# which shadows the `type.__annotations__` data descriptor for every model class.
# `abc.ABCMeta.__annotations__` has the same effect, and `import langgraph.prebuilt`
# performs an equivalent read at import time.
type(BaseModel).__annotations__
T = TypeVar('T')
class G(BaseModel, Generic[T]):
x: T
a: int = 5
G[str].model_validate({'x': 'hi'})
pydantic_core._pydantic_core.ValidationError: 1 validation error for G[str]
a
Field required [type=missing, input_value={'x': 'hi'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.14/v/missing
Removing the type(BaseModel).__annotations__ line — or moving the G[str] parametrization above it — makes it pass. To run against main:
uv run --isolated --no-project --python 3.13 --with "pydantic @ git+https://github.com/pydantic/pydantic.git" repro.py
Python, Pydantic & OS Version
pydantic version: 2.14.0a1
pydantic-core version: 2.47.0
pydantic-core build: profile=release pgo=false
python version: 3.13.0 (main, Oct 12 2024, 20:24:27) [Clang 15.0.0 (clang-1500.3.9.4)]
platform: macOS-26.5.1-arm64-arm-64bit-Mach-O
related packages: mypy-2.3.0 pyright-1.1.411 email-validator-2.3.0 fastapi-0.139.2 pydantic-settings-2.14.2 typing_extensions-4.16.0
commit: a187a65467a588ccc00761d80f325b45381ca819
Selected Assignee: @Viicos
The two patches
pydantic/_internal/_typing_extra.pyindex ab6a1397..aa2ed8eb 100644@@ -293,6 +293,12 @@ def safe_get_annotations(obj: Any) -> dict[str, Any]: """ if sys.version_info >= (3, 14): return annotationlib.get_annotations(obj, format=annotationlib.Format.FORWARDREF)+ elif isinstance(obj, type):+ # Only look up the class' own annotations (similar to what `inspect.get_annotations()` does).+ # `getattr()` isn't safe here: on Python <= 3.13, reading `__annotations__` on a metaclass+ # (e.g. `ModelMetaclass` or `ABCMeta`) stores a plain `{}` in its `__dict__`, which shadows the+ # `type.__annotations__` descriptor, and the lookup would then return the annotations of a parent class.+ return obj.__dict__.get('__annotations__', {}) else: return getattr(obj, '__annotations__', {}) tests/test_generics.pyindex a0c46632..ae7ed564 100644@@ -3271,3 +3271,62 @@ def test_slots_forwarded_from_generic_class() -> None: with pytest.raises(TypeError): # As per https://docs.python.org/3/reference/datamodel.html#slots: weakref.ref(BaseInt())+++@pytest.fixture(params=['model_metaclass', 'abc_meta'])+def metaclass_with_annotations_read(request: pytest.FixtureRequest) -> Iterable[type]:+ """Read `__annotations__` on a metaclass of `BaseModel`.++ On Python <= 3.13, this stores a plain `{}` in the metaclass' `__dict__`, shadowing the `type.__annotations__`+ descriptor when looking up `__annotations__` on model classes.+ """+ from abc import ABCMeta++ from pydantic._internal._model_construction import ModelMetaclass++ metaclass = ModelMetaclass if request.param == 'model_metaclass' else ABCMeta+ keys_before = set(metaclass.__dict__)+ metaclass.__annotations__+ try:+ yield metaclass+ finally:+ for key in set(metaclass.__dict__) - keys_before:+ delattr(metaclass, key)+++@pytest.mark.usefixtures('metaclass_with_annotations_read')+def test_generic_parametrization_after_metaclass_annotations_read() -> None:+ """https://github.com/pydantic/pydantic/issues/13519"""++ T = TypeVar('T')++ class G(BaseModel, Generic[T]):+ x: T+ a: int = 5++ assert G[str].model_validate({'x': 'hi'}) == G[str](x='hi', a=5)+ assert G[str].model_fields['a'].default == 5+ assert G[str].model_fields['x'].annotation is str++ with pytest.raises(ValidationError):+ G[int].model_validate({'x': 'not an int'})+++@pytest.mark.usefixtures('metaclass_with_annotations_read')+def test_subclass_after_metaclass_annotations_read() -> None:+ T = TypeVar('T')++ class Parent(BaseModel, Generic[T]):+ x: T+ a: int = 5++ class Sub(Parent[int]):+ pass++ class GenericSub(Parent[T], Generic[T]):+ pass++ assert Sub(x=1).a == 5+ assert Sub.model_fields['x'].annotation is int+ assert GenericSub[str](x='hi').a == 5+ assert GenericSub[str].model_fields['x'].annotation is str
pydantic/_internal/_typing_extra.pyindex ab6a1397c..d68c792af 100644@@ -294,7 +294,10 @@ def safe_get_annotations(obj: Any) -> dict[str, Any]: if sys.version_info >= (3, 14): return annotationlib.get_annotations(obj, format=annotationlib.Format.FORWARDREF) else:- return getattr(obj, '__annotations__', {})+ if isinstance(obj, type):+ return obj.__dict__.get('__annotations__', {})+ else:+ return getattr(obj, '__annotations__', {}) def get_model_type_hints(tests/test_edge_cases.pyindex 3e8e38e7d..df52288cc 100644@@ -51,6 +51,7 @@ from pydantic import ( validator, ) from pydantic._internal._model_construction import ModelMetaclass+from pydantic._internal._typing_extra import safe_get_annotations from pydantic.fields import Field, computed_field from pydantic.functional_serializers import ( field_serializer,@@ -3161,3 +3162,23 @@ def test_model_fields_set_includes_extra_after_assignment(): m.extra_after_init = 3 assert m.model_fields_set == {'field', 'extra_at_init', 'extra_after_init'} assert m.model_extra == {'extra_at_init': 2, 'extra_after_init': 3}+++def test_safe_get_annotations_from_dict() -> None:+ """https://github.com/pydantic/pydantic/issues/13520"""++ class Meta(type):+ pass++ Meta.__annotations__++ class Base(metaclass=Meta):+ pass++ class Main(Base):+ f: int++ class Sub(Main):+ pass++ assert safe_get_annotations(Sub) == {}