Backtestspydanticrun-20260930-144322#13520
Generic model parametrization drops field defaults and typevar substitution after __annotations__ is read on a metaclass (Python <= 3.13)
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
The core fix is the same in both patches. On Python <= 3.13, when `obj` is a class, both read `obj.__dict__.get('__annotations__', {})` instead of using `getattr`. That addresses the root cause described in the issue: a plain dict planted on the metaclass shadows the `type.__annotations__` descriptor, so attribute lookup falls through to the parent's annotations. Both patches fix both call sites (`get_model_type_hints` and `collect_model_fields`), so they are equally correct. The differences are in tests and craft. Patch A's test checks `safe_get_annotations(Sub) == {}` using a plain metaclass. It would fail before the fix and pass after, but it only exercises the helper, not the reported symptom. Patch B's test reproduces the reported scenario end to end. It uses a `ModelMetaclass` subclass with `__annotations__` read, then a generic model with a TypeVar field and a defaulted field. It asserts that `G[str]` substitutes the TypeVar (`annotation is str`), keeps the default (`default == 5`), validates `{'x': 'hi'}` with `a == 5`, and rejects a non-str `x`. It would fail before the fix and directly guards against the regression the issue reports (dropped defaults and missing typevar substitution). On craft, Patch B adds a clear comment in the source explaining why `__dict__` is used, which matters for a subtle CPython quirk like this. Patch A adds no comment and leaves a `"""<link>"""` placeholder docstring in its test. Scope is equally minimal in both. Patch B wins on test quality and clarity.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
3m 45s13 turns
Wall-clock time in the container
Full test suite
No new failures
12,274 passed · 501 skipped · 12,775 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
34vs 26 merged
Changed lines, ours vs merged
The issue
View on GitHubInitial Checks
- I confirm that I'm using Pydantic V2
Description
This is a regression on main (2.14.0a1); I could not reproduce it on any released version (checked 2.13.4 and 2.10.6 under the same conditions). It affects Python 3.10–3.13 (verified on 3.10, 3.12 and 3.13); Python 3.14 is unaffected.
If __annotations__ has ever been read on a metaclass appearing in a model's metatype MRO (e.g. ModelMetaclass itself, or abc.ABCMeta), then any generic model parametrized for the first time afterwards gets a broken schema:
- type variables are not substituted (
x: Tends up with ananyschema), and - field defaults are dropped (fields with defaults become required).
Parametrizations built before the read keep working, since their schemas are already cached — which makes this very order-dependent and confusing to track down in a real test suite.
Reading an attribute sounds like an odd trigger, but it happens in the wild: on CPython ≤ 3.13, reading __annotations__ on a (heap) class that has none auto-creates a plain __annotations__ = {} entry in that class's __dict__ (see the annotations HOWTO). langgraph does such a read at import time (get_cached_annotated_keys in langgraph/_internal/_fields.py has a getattr(base, "__annotations__", None) fallback — added for Python 3.14 compat, ironically — which ends up planting {} on abc.ABCMeta). So a plain import langgraph.prebuilt breaks all subsequently-parametrized generic models. We found this via the pydantic/logfire weekly CI job testing against pydantic main, where pydantic_evals.Dataset.from_dict() fails with 3 validation errors for _DatasetModel[...] — Field required for exactly its three defaulted fields — but only in test runs where a langchain test ran first (failing run).
Mechanism:
- The planted
{}lives in the metaclass's__dict__. For a model classM, attribute lookup ofM.__annotations__searchestype(M).__mro__(e.g.(ModelMetaclass, ABCMeta, type, object)) and now finds the plain dict before reaching thetype.__annotations__getset descriptor. A plain dict is not a data descriptor, so lookup falls back toM's own MRO — and for a class with no own__annotations__(like a freshly created generic submodel), this returns the parent's annotations. (Normally thetype.__annotations__data descriptor intercepts the lookup and auto-creates/returns the class's own empty dict, which is what makesgetattrlook safe.) safe_get_annotations()inpydantic/_internal/_typing_extra.pyusesgetattr(obj, '__annotations__', {})on < 3.14, so for the new submodelG[str]it returns the origin's annotations instead of{}:clean: safe_get_annotations(G[str]) == {} poisoned: safe_get_annotations(G[str]) == {'x': ~T, 'a': <class 'int'>}- In
collect_model_fields(),cls_annotationsis "only used to determine if an annotation comes from a parent class". The leak makesann_name in cls_annotationstrue for every inherited field, so instead of copying the parentFieldInfoand applyingtypevars_map(via_recreate_field_info), it takes the plainFieldInfo.from_annotation(ann_type)branch: no typevar substitution, and no default — the origin's class-attribute default is long gone because it wasdelattr'd during the origin's own field collection.
Possible fix: for classes on ≤ 3.13, read own annotations via obj.__dict__.get('__annotations__', {}) (what inspect.get_annotations() does) rather than getattr. Both call sites of safe_get_annotations() seem to want own-class annotations anyway: get_model_type_hints() iterates the MRO itself and calls it per base, and cls_annotations in collect_model_fields() is explicitly meant to be the current class's own annotations.
Example Code
from typing import Generic, TypeVar
from pydantic import BaseModel
# The trigger: any read of `__annotations__` on a metaclass. On CPython <= 3.13 this
# auto-creates a plain `__annotations__ = {}` entry in the metaclass's `__dict__`,
# which shadows the `type.__annotations__` data descriptor for every model class.
# `abc.ABCMeta.__annotations__` has the same effect, and `import langgraph.prebuilt`
# performs an equivalent read at import time.
type(BaseModel).__annotations__
T = TypeVar('T')
class G(BaseModel, Generic[T]):
x: T
a: int = 5
G[str].model_validate({'x': 'hi'})
pydantic_core._pydantic_core.ValidationError: 1 validation error for G[str]
a
Field required [type=missing, input_value={'x': 'hi'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.14/v/missing
Removing the type(BaseModel).__annotations__ line — or moving the G[str] parametrization above it — makes it pass. To run against main:
uv run --isolated --no-project --python 3.13 --with "pydantic @ git+https://github.com/pydantic/pydantic.git" repro.py
Python, Pydantic & OS Version
pydantic version: 2.14.0a1
pydantic-core version: 2.47.0
pydantic-core build: profile=release pgo=false
python version: 3.13.0 (main, Oct 12 2024, 20:24:27) [Clang 15.0.0 (clang-1500.3.9.4)]
platform: macOS-26.5.1-arm64-arm-64bit-Mach-O
related packages: mypy-2.3.0 pyright-1.1.411 email-validator-2.3.0 fastapi-0.139.2 pydantic-settings-2.14.2 typing_extensions-4.16.0
commit: a187a65467a588ccc00761d80f325b45381ca819
Selected Assignee: @Viicos
The two patches
pydantic/_internal/_typing_extra.pyindex ab6a1397..b314fa96 100644@@ -294,6 +294,12 @@ def safe_get_annotations(obj: Any) -> dict[str, Any]: if sys.version_info >= (3, 14): return annotationlib.get_annotations(obj, format=annotationlib.Format.FORWARDREF) else:+ if isinstance(obj, type):+ # On Python <= 3.13, reading `__annotations__` on a metaclass plants a plain dict in the+ # metaclass `__dict__`, shadowing the `type.__annotations__` descriptor. `getattr()` would then+ # fall back to the MRO and return a parent's annotations, so read the class' own `__dict__`+ # (as `inspect.get_annotations()` does):+ return obj.__dict__.get('__annotations__', {}) return getattr(obj, '__annotations__', {}) tests/test_generics.pyindex a0c46632..8668cc3b 100644@@ -3271,3 +3271,31 @@ def test_slots_forwarded_from_generic_class() -> None: with pytest.raises(TypeError): # As per https://docs.python.org/3/reference/datamodel.html#slots: weakref.ref(BaseInt())+++def test_generic_parametrization_metaclass_annotations_leak() -> None:+ """On Python <= 3.13, reading `__annotations__` on a metaclass plants `{}` in its `__dict__`,+ shadowing the `type.__annotations__` descriptor. Parametrized generic models must not then+ pick up the parent's annotations (losing field defaults and typevar substitution).+ """+ from pydantic._internal._model_construction import ModelMetaclass++ class Meta(ModelMetaclass):+ pass++ Meta.__annotations__ # noqa: B018 -- plants `{}` in Meta.__dict__ on Python <= 3.13++ T = TypeVar('T')++ class G(BaseModel, Generic[T], metaclass=Meta):+ x: T+ a: int = 5++ assert G[str].model_fields['x'].annotation is str+ assert G[str].model_fields['a'].default == 5++ m = G[str].model_validate({'x': 'hi'})+ assert m.a == 5++ with pytest.raises(ValidationError):+ G[str](x=1)
pydantic/_internal/_typing_extra.pyindex ab6a1397c..d68c792af 100644@@ -294,7 +294,10 @@ def safe_get_annotations(obj: Any) -> dict[str, Any]: if sys.version_info >= (3, 14): return annotationlib.get_annotations(obj, format=annotationlib.Format.FORWARDREF) else:- return getattr(obj, '__annotations__', {})+ if isinstance(obj, type):+ return obj.__dict__.get('__annotations__', {})+ else:+ return getattr(obj, '__annotations__', {}) def get_model_type_hints(tests/test_edge_cases.pyindex 3e8e38e7d..df52288cc 100644@@ -51,6 +51,7 @@ from pydantic import ( validator, ) from pydantic._internal._model_construction import ModelMetaclass+from pydantic._internal._typing_extra import safe_get_annotations from pydantic.fields import Field, computed_field from pydantic.functional_serializers import ( field_serializer,@@ -3161,3 +3162,23 @@ def test_model_fields_set_includes_extra_after_assignment(): m.extra_after_init = 3 assert m.model_fields_set == {'field', 'extra_at_init', 'extra_after_init'} assert m.model_extra == {'extra_at_init': 2, 'extra_after_init': 3}+++def test_safe_get_annotations_from_dict() -> None:+ """https://github.com/pydantic/pydantic/issues/13520"""++ class Meta(type):+ pass++ Meta.__annotations__++ class Base(metaclass=Meta):+ pass++ class Main(Base):+ f: int++ class Sub(Main):+ pass++ assert safe_get_annotations(Sub) == {}
What the agent reported
I fixed the bug: G[str] now keeps its default and gets x: str even after __annotations__ has been read on a metaclass. The regression test fails without the fix and passes with it. The full suite has one failure, and it also fails without my change.
What was wrong: on Python 3.13 and older, safe_get_annotations() in pydantic/_internal/_typing_extra.py used getattr(obj, '__annotations__', {}). If __annotations__ had been read on a metaclass, a plain {} sat in that metaclass's __dict__ and hid the normal lookup. For a fresh parametrized class like G[str], which has no annotations of its own, getattr then returned the parent's annotations. collect_model_fields() took that to mean every inherited field was declared on G[str] itself, so it built bare FieldInfos. Those had no default and no type-variable substitution.
What I changed:
- The fix (
pydantic/_internal/_typing_extra.py, 6 lines added): on Python 3.13 and older, classes now read their own annotations withobj.__dict__.get('__annotations__', {}), which is whatinspect.get_annotations()does. Functions and partials still go throughgetattr, and the Python 3.14 path is unchanged. - The test (
tests/test_generics.py):test_generic_parametrization_metaclass_annotations_leakdefines a local subclass ofModelMetaclass, reads__annotations__on it, and then buildsG[str]. Using a local metaclass keeps the planted{}from leaking into other tests. It checks thatxhas typestr, thatadefaults to 5, thatmodel_validate({'x': 'hi'})works, and thatx=1is rejected.
Test evidence (Python 3.13.15):
- Your example script failed with
a: Field requiredbefore the fix and printsx='hi' a=5after it. - The new test fails on the unfixed code (
assert ~T is str) and passes with the fix. - The generics, dataclasses, main, forward-ref, edge-case, typing, validate-call, fields and internal test files: 1082 passed, 30 skipped, 12 xfailed.
- Full suite: 12273 passed, 1 failed. The failure is
tests/test_validators.py::test_v1_validator_deprecated, which checks a hardcoded file path (/work/...vs/workspace/...). It fails the same way with my changes stashed, so it comes from this environment.
Other code that uses this function: it's also called from get_model_type_hints and get_cls_type_hints (which go through each base class in turn), from dataclass field collection in _fields.py, and from pydantic/dataclasses.py. All of them want a class's own annotations, so each is now correct in the affected case as well. For normal classes the result is the same as before. The one difference is that reading annotations no longer writes an empty {} into a class's __dict__ as a side effect. The reviewer checked every caller and found nothing blocking.
Requirements:
| Item from the report | Status |
|---|---|
| Parametrizing after the metaclass read loses type-variable substitution | Fixed; the test asserts x has type str and rejects x=1 |
| Defaults are dropped (fields become required) | Fixed; the test asserts the default is 5 and model_validate({'x': 'hi'}) works |
Suggested fix: read obj.__dict__.get('__annotations__', {}) on Python 3.13 and older | Done, for classes only |
Both callers (get_model_type_hints, collect_model_fields) want the class's own annotations | Confirmed by the reviewer, along with the other callers; covered by the suite runs above |
The abc.ABCMeta / langgraph trigger | Same code path; the test uses a local metaclass rather than ABCMeta so it doesn't affect other tests |
| Python 3.14 is unaffected | That branch is untouched; I ran tests only on 3.13, where the new test passes |
The changes are uncommitted edits in the working tree.