Backtestspydanticrun-20260929-204949#13621
Memory leak: transient models using Enum methods or Field(exclude_if=...) survive GC
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Both patches make functionally identical core fixes: adding serialization_exclude_if to SerField traversal, class to EnumSerializer, and lookup to every EnumValidator<T>. On correctness they tie — both address the root cause exactly as the issue diagnoses. The difference is in tests and scope. Patch A introduces significant unrelated churn to the existing test file: it rewrites test_gc_schema_serializer/validator (renaming __schema__→__pydantic_core_schema__, converting comments to docstrings) and, critically, REMOVES the `is_free_threaded and sys.version_info < (3, 14)` xfail markers from the two pre-existing GC tests without justification. The issue itself notes GC behavior differs on free-threaded builds, so dropping these markers adds real risk of spurious failures on free-threaded Python — churn that works against the fix. Patch A's new tests also omit the free-threaded xfail marker entirely. Patch B leaves existing tests untouched and adds two focused new tests (test_gc_enum, test_gc_serialization_exclude_if) that correctly establish the reference cycle (validator/serializer stored back on the enum/model class, callable capturing cls), that faithfully mirror the issue's reproduction (enum with a method, exclude_if capturing the class), and that retain the proper free-threaded xfail markers consistent with surrounding conventions. Patch B is the tighter-scoped, lower-risk, more convention-consistent patch.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
13m 45s23 turns
Wall-clock time in the container
Full test suite
No new failures
12,324 passed · 500 skipped · 12,824 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
87vs 139 merged
Changed lines, ours vs merged
The issue
View on GitHubInitial Checks
- I confirm that I'm using Pydantic V2
Description
Transient BaseModel classes are not collected when either:
- A model field references an
Enumwhose class defines a Python method. - A model field uses
Field(exclude_if=<Python callable>).
After the defining module is removed from sys.modules and every ordinary strong reference is dropped, weak references to the model and enum classes remain alive after repeated full gc.collect() calls. Repeating this construction in disposable module namespaces causes linear memory growth.
Controls using an enum without methods or a field without exclude_if are collected normally.
I encountered this in combination with a Temporal workflow sandbox. With a large interconnected model graph and high volume Temporal workflows this can result in multi-GiB process growth relatively quickly.
The likely cause is missing PyGcTraverse edges for Python objects held by pydantic-core:
SerFieldownsserialization_exclude_if, but traverses onlyserializer: https://github.com/pydantic/pydantic/blob/c67a1397327c6abc1a2e2e921af01aebbbeedf1c/pydantic-core/src/serializers/fields.rs#L26-L38EnumSerializerownsclass, but traverses onlyserializer: https://github.com/pydantic/pydantic/blob/c67a1397327c6abc1a2e2e921af01aebbbeedf1c/pydantic-core/src/serializers/type_serializers/enum_.rs- Each
EnumValidator<T>ownslookup, but traverses onlyclassandmissing: https://github.com/pydantic/pydantic/blob/c67a1397327c6abc1a2e2e921af01aebbbeedf1c/pydantic-core/src/validators/enum_.rs
Traversing py_schema does not compensate for another unreported direct reference: CPython’s collector accounts for every actual strong edge individually.
I tested a temporary build from Pydantic main at c67a1397327c6abc1a2e2e921af01aebbbeedf1c. Adding only these traversal edges made all objects in the reproduction collectible:
- Add
serialization_exclude_iftoSerFieldtraversal. - Add
classtoEnumSerializertraversal. - Add
lookupto everyEnumValidator<T>traversal.
For the enum case, changing only the serializer or only the validator was insufficient; both missing edges were required.
This appears related to earlier missing-traversal fixes such as:
- https://github.com/pydantic/pydantic/issues/5136
- https://github.com/pydantic/pydantic-core/pull/787
- https://github.com/pydantic/pydantic-core/pull/1100
I would be happy to submit a focused patch.
(please note that this analysis was LLM assisted)
Example Code
import gc
import sys
import types
import weakref
def transient_module(name, source, class_names):
module = types.ModuleType(name)
sys.modules[name] = module
exec(compile(source, f"<{name}>", "exec"), module.__dict__)
refs = tuple(
weakref.ref(module.__dict__[class_name])
for class_name in class_names
)
del sys.modules[name]
del module
return refs
enum_ref, enum_model_ref = transient_module(
"_pydantic_enum_gc_repro",
"""
from enum import Enum
from pydantic import BaseModel
class Membership(str, Enum):
FULL = "full"
def label(self) -> str:
return self.value
class Model(BaseModel):
membership: Membership
""",
("Membership", "Model"),
)
(exclude_if_model_ref,) = transient_module(
"_pydantic_exclude_if_gc_repro",
"""
from pydantic import BaseModel, Field
class Model(BaseModel):
value: int = Field(exclude_if=lambda value: False)
""",
("Model",),
)
for _ in range(5):
gc.collect()
# All three should be False because no intentional strong references remain.
print("enum alive:", enum_ref() is not None)
print("enum model alive:", enum_model_ref() is not None)
print("exclude_if model alive:", exclude_if_model_ref() is not None)
Python, Pydantic & OS Version
pydantic version: 2.12.5
pydantic-core version: 2.41.5
pydantic-core build: profile=release pgo=false
python version: 3.11.15 (main, Aug 5 2026, 03:58:57) [GCC 12.2.0]
platform: Linux-6.5.0-1018-oem-x86_64-with-glibc2.36
related packages: typing_extensions-4.15.0 pydantic-extra-types-2.10.5 mypy-1.16.0
commit: unknownThe two patches
pydantic-core/src/serializers/fields.rsindex ba18ee4f..0d182849 100644@@ -35,7 +35,7 @@ pub(super) struct SerField { pub serialization_exclude_if: Option<Py<PyAny>>, } -impl_py_gc_traverse!(SerField { serializer });+impl_py_gc_traverse!(SerField { serializer, serialization_exclude_if }); impl SerField { pub fn new(pydantic-core/src/serializers/type_serializers/enum_.rsindex 0d2819ac..1ad123e9 100644@@ -48,7 +48,7 @@ impl BuildSerializer for EnumSerializer { } } -impl_py_gc_traverse!(EnumSerializer { serializer });+impl_py_gc_traverse!(EnumSerializer { class, serializer }); impl TypeSerializer for EnumSerializer { fn to_python<'py>(&self, value: &Bound<'py, PyAny>, state: &mut SerializationState<'py>) -> PyResult<Py<PyAny>> {pydantic-core/src/validators/enum_.rsindex 6ef685da..7717b06a 100644@@ -168,7 +168,7 @@ impl<T: EnumValidateValue> Validator for EnumValidator<T> { #[derive(Debug, Clone)] pub struct PlainEnumValidator; -impl_py_gc_traverse!(EnumValidator<PlainEnumValidator> { class, missing });+impl_py_gc_traverse!(EnumValidator<PlainEnumValidator> { class, missing, lookup }); impl EnumValidateValue for PlainEnumValidator { fn validate_value<'py, I: Input<'py> + ?Sized>(@@ -200,7 +200,7 @@ impl EnumValidateValue for PlainEnumValidator { #[derive(Debug, Clone)] pub struct IntEnumValidator; -impl_py_gc_traverse!(EnumValidator<IntEnumValidator> { class, missing });+impl_py_gc_traverse!(EnumValidator<IntEnumValidator> { class, missing, lookup }); impl EnumValidateValue for IntEnumValidator { fn validate_value<'py, I: Input<'py> + ?Sized>(@@ -216,7 +216,7 @@ impl EnumValidateValue for IntEnumValidator { #[derive(Debug, Clone)] pub struct StrEnumValidator; -impl_py_gc_traverse!(EnumValidator<StrEnumValidator> { class, missing });+impl_py_gc_traverse!(EnumValidator<StrEnumValidator> { class, missing, lookup }); impl EnumValidateValue for StrEnumValidator { fn validate_value<'py, I: Input<'py> + ?Sized>(@@ -232,7 +232,7 @@ impl EnumValidateValue for StrEnumValidator { #[derive(Debug, Clone)] pub struct FloatEnumValidator; -impl_py_gc_traverse!(EnumValidator<FloatEnumValidator> { class, missing });+impl_py_gc_traverse!(EnumValidator<FloatEnumValidator> { class, missing, lookup }); impl EnumValidateValue for FloatEnumValidator { fn validate_value<'py, I: Input<'py> + ?Sized>(pydantic-core/tests/test_garbage_collection.pyindex e53cdc56..4773f813 100644@@ -1,3 +1,4 @@+import enum import platform import sys from collections.abc import Iterable@@ -114,3 +115,77 @@ def test_gc_validator_iterator() -> None: del iterable assert_gc(lambda: len(cache) == 0)+++@pytest.mark.xfail(is_free_threaded and sys.version_info < (3, 14), reason='GC leaks on free-threaded (<3.14)')+@pytest.mark.xfail(+ condition=platform.python_implementation() == 'PyPy', reason='https://foss.heptapod.net/pypy/pypy/-/issues/3899'+)+@pytest.mark.skipif(platform.python_implementation() == 'GraalVM', reason='Cannot reliably trigger GC on GraalPy')+def test_gc_enum() -> None:+ # test that a transient enum class referenced by an enum validator/serializer does not leak,+ # i.e. that `EnumValidator.lookup` and `EnumSerializer.class` are traversed by the GC.+ # The validator/serializer are stored on the enum class, forming a reference cycle that+ # can only be broken by the GC if these traverse edges are present.+ cache: WeakValueDictionary[int, Any] = WeakValueDictionary()++ def make() -> type[enum.Enum]:+ class Color(enum.Enum):+ RED = 1+ GREEN = 2+ BLUE = 3++ def describe(self) -> str:+ return self.name++ schema = core_schema.enum_schema(Color, list(Color.__members__.values()))+ Color.__validator__ = SchemaValidator(schema) # type: ignore[attr-defined]+ Color.__serializer__ = SchemaSerializer(schema) # type: ignore[attr-defined]++ return Color++ for _ in range(10_000):+ color = make()+ cache[id(color)] = color+ del color++ assert_gc(lambda: len(cache) == 0)+++@pytest.mark.xfail(is_free_threaded and sys.version_info < (3, 14), reason='GC leaks on free-threaded (<3.14)')+@pytest.mark.xfail(+ condition=platform.python_implementation() == 'PyPy', reason='https://foss.heptapod.net/pypy/pypy/-/issues/3899'+)+@pytest.mark.skipif(platform.python_implementation() == 'GraalVM', reason='Cannot reliably trigger GC on GraalPy')+def test_gc_serialization_exclude_if() -> None:+ # test that a model with a field using `serialization_exclude_if` does not leak,+ # i.e. that `SerField.serialization_exclude_if` is traversed by the GC.+ class BaseModel:+ __serializer__: SchemaSerializer++ def __init_subclass__(cls) -> None:+ # the callable captures `cls`, creating a cycle through+ # `SerField.serialization_exclude_if` back to the model class+ def exclude_if(value: Any, _cls: type = cls) -> bool:+ return value is None++ schema = core_schema.model_schema(+ cls,+ core_schema.model_fields_schema(+ {'x': core_schema.model_field(core_schema.int_schema(), serialization_exclude_if=exclude_if)}+ ),+ )+ cls.__serializer__ = SchemaSerializer(schema)++ cache: WeakValueDictionary[int, Any] = WeakValueDictionary()++ for _ in range(10_000):++ class MyModel(BaseModel):+ pass++ cache[id(MyModel)] = MyModel++ del MyModel++ assert_gc(lambda: len(cache) == 0)
pydantic-core/src/serializers/fields.rsindex ba18ee4f3..8631d0b09 100644@@ -35,7 +35,10 @@ pub(super) struct SerField { pub serialization_exclude_if: Option<Py<PyAny>>, } -impl_py_gc_traverse!(SerField { serializer });+impl_py_gc_traverse!(SerField {+ serializer,+ serialization_exclude_if+}); impl SerField { pub fn new(pydantic-core/src/serializers/type_serializers/enum_.rsindex 0d2819ac2..b4e76133c 100644@@ -23,6 +23,8 @@ pub struct EnumSerializer { serializer: Option<Arc<CombinedSerializer>>, } +impl_py_gc_traverse!(EnumSerializer { class, serializer });+ impl BuildSerializer for EnumSerializer { const EXPECTED_TYPE: &'static str = "enum"; @@ -48,8 +50,6 @@ impl BuildSerializer for EnumSerializer { } } -impl_py_gc_traverse!(EnumSerializer { serializer });- impl TypeSerializer for EnumSerializer { fn to_python<'py>(&self, value: &Bound<'py, PyAny>, state: &mut SerializationState<'py>) -> PyResult<Py<PyAny>> { let py = value.py();pydantic-core/src/validators/enum_.rsindex 6ef685daf..28a32a874 100644@@ -168,7 +168,7 @@ impl<T: EnumValidateValue> Validator for EnumValidator<T> { #[derive(Debug, Clone)] pub struct PlainEnumValidator; -impl_py_gc_traverse!(EnumValidator<PlainEnumValidator> { class, missing });+impl_py_gc_traverse!(EnumValidator<PlainEnumValidator> { class, lookup, missing }); impl EnumValidateValue for PlainEnumValidator { fn validate_value<'py, I: Input<'py> + ?Sized>(@@ -200,7 +200,7 @@ impl EnumValidateValue for PlainEnumValidator { #[derive(Debug, Clone)] pub struct IntEnumValidator; -impl_py_gc_traverse!(EnumValidator<IntEnumValidator> { class, missing });+impl_py_gc_traverse!(EnumValidator<IntEnumValidator> { class, lookup, missing }); impl EnumValidateValue for IntEnumValidator { fn validate_value<'py, I: Input<'py> + ?Sized>(@@ -216,7 +216,7 @@ impl EnumValidateValue for IntEnumValidator { #[derive(Debug, Clone)] pub struct StrEnumValidator; -impl_py_gc_traverse!(EnumValidator<StrEnumValidator> { class, missing });+impl_py_gc_traverse!(EnumValidator<StrEnumValidator> { class, lookup, missing }); impl EnumValidateValue for StrEnumValidator { fn validate_value<'py, I: Input<'py> + ?Sized>(@@ -232,7 +232,7 @@ impl EnumValidateValue for StrEnumValidator { #[derive(Debug, Clone)] pub struct FloatEnumValidator; -impl_py_gc_traverse!(EnumValidator<FloatEnumValidator> { class, missing });+impl_py_gc_traverse!(EnumValidator<FloatEnumValidator> { class, lookup, missing }); impl EnumValidateValue for FloatEnumValidator { fn validate_value<'py, I: Input<'py> + ?Sized>(pydantic-core/tests/test_garbage_collection.pyindex e53cdc564..7ac338f77 100644@@ -1,6 +1,6 @@ import platform-import sys from collections.abc import Iterable+from enum import Enum from typing import Any from weakref import WeakValueDictionary @@ -8,7 +8,7 @@ import pytest from pydantic_core import SchemaSerializer, SchemaValidator, core_schema -from .conftest import assert_gc, is_free_threaded+from .conftest import assert_gc GC_TEST_SCHEMA_INNER = core_schema.definitions_schema( core_schema.definition_reference_schema(schema_ref='model'),@@ -21,18 +21,18 @@ GC_TEST_SCHEMA_INNER = core_schema.definitions_schema( ) -@pytest.mark.xfail(is_free_threaded and sys.version_info < (3, 14), reason='GC leaks on free-threaded (<3.14)') @pytest.mark.xfail(- condition=platform.python_implementation() == 'PyPy', reason='https://foss.heptapod.net/pypy/pypy/-/issues/3899'+ condition=platform.python_implementation() == 'PyPy', reason='https://github.com/pypy/pypy/issues/3898' ) @pytest.mark.skipif(platform.python_implementation() == 'GraalVM', reason='Cannot reliably trigger GC on GraalPy') def test_gc_schema_serializer() -> None:- # test for https://github.com/pydantic/pydantic/issues/5136+ """https://github.com/pydantic/pydantic/issues/5136"""+ class BaseModel:- __schema__: SchemaSerializer+ __pydantic_core_schema__: SchemaSerializer def __init_subclass__(cls) -> None:- cls.__schema__ = SchemaSerializer(+ cls.__pydantic_core_schema__ = SchemaSerializer( core_schema.model_schema(cls, GC_TEST_SCHEMA_INNER), config={'ser_json_timedelta': 'float'} ) @@ -50,18 +50,18 @@ def test_gc_schema_serializer() -> None: assert_gc(lambda: len(cache) == 0) -@pytest.mark.xfail(is_free_threaded and sys.version_info < (3, 14), reason='GC leaks on free-threaded (<3.14)') @pytest.mark.xfail(- condition=platform.python_implementation() == 'PyPy', reason='https://foss.heptapod.net/pypy/pypy/-/issues/3899'+ condition=platform.python_implementation() == 'PyPy', reason='https://github.com/pypy/pypy/issues/3898' ) @pytest.mark.skipif(platform.python_implementation() == 'GraalVM', reason='Cannot reliably trigger GC on GraalPy') def test_gc_schema_validator() -> None:- # test for https://github.com/pydantic/pydantic/issues/5136+ """https://github.com/pydantic/pydantic/issues/5136"""+ class BaseModel:- __validator__: SchemaValidator+ __pydantic_validator__: SchemaValidator def __init_subclass__(cls) -> None:- cls.__validator__ = SchemaValidator(+ cls.__pydantic_validator__ = SchemaValidator( schema=core_schema.model_schema(cls, GC_TEST_SCHEMA_INNER), config=core_schema.CoreConfig(extra_fields_behavior='allow'), )@@ -81,11 +81,12 @@ def test_gc_schema_validator() -> None: @pytest.mark.xfail(- condition=platform.python_implementation() == 'PyPy', reason='https://foss.heptapod.net/pypy/pypy/-/issues/3899'+ condition=platform.python_implementation() == 'PyPy', reason='https://github.com/pypy/pypy/issues/3898' ) @pytest.mark.skipif(platform.python_implementation() == 'GraalVM', reason='Cannot reliably trigger GC on GraalPy') def test_gc_validator_iterator() -> None:- # test for https://github.com/pydantic/pydantic/issues/9243+ """https://github.com/pydantic/pydantic/issues/9243"""+ class MyModel: iter: Iterable[int] @@ -114,3 +115,96 @@ def test_gc_validator_iterator() -> None: del iterable assert_gc(lambda: len(cache) == 0)+++@pytest.mark.xfail(+ condition=platform.python_implementation() == 'PyPy', reason='https://github.com/pypy/pypy/issues/3898'+)+@pytest.mark.skipif(platform.python_implementation() == 'GraalVM', reason='Cannot reliably trigger GC on GraalPy')+def test_gc_enum_serializer_class() -> None:+ """https://github.com/pydantic/pydantic/issues/13621"""++ class EnumBase:+ __pydantic_serializer__: SchemaSerializer++ cache: WeakValueDictionary[int, Any] = WeakValueDictionary()++ for _ in range(10_000):++ class MyEnum(EnumBase, str, Enum):+ FULL = 'full'++ MyEnum.__pydantic_serializer__ = SchemaSerializer(core_schema.enum_schema(MyEnum, list(MyEnum), sub_type='str'))++ cache[id(MyEnum)] = MyEnum++ del MyEnum++ assert_gc(lambda: len(cache) == 0)+++@pytest.mark.xfail(+ condition=platform.python_implementation() == 'PyPy', reason='https://github.com/pypy/pypy/issues/3898'+)+@pytest.mark.skipif(platform.python_implementation() == 'GraalVM', reason='Cannot reliably trigger GC on GraalPy')+def test_gc_enum_validator_lookup() -> None:+ """https://github.com/pydantic/pydantic/issues/13621"""++ class EnumBase:+ __pydantic_validator__: SchemaValidator++ cache: WeakValueDictionary[int, Any] = WeakValueDictionary()++ for _ in range(10_000):++ class MyEnum(EnumBase, str, Enum):+ FULL = 'full'++ MyEnum.__pydantic_validator__ = SchemaValidator(core_schema.enum_schema(MyEnum, list(MyEnum), sub_type='str'))++ cache[id(MyEnum)] = MyEnum++ del MyEnum++ assert_gc(lambda: len(cache) == 0)+++@pytest.mark.xfail(+ condition=platform.python_implementation() == 'PyPy', reason='https://github.com/pypy/pypy/issues/3898'+)+@pytest.mark.skipif(platform.python_implementation() == 'GraalVM', reason='Cannot reliably trigger GC on GraalPy')+def test_gc_field_exclude_if() -> None:+ """https://github.com/pydantic/pydantic/issues/13621"""++ class Dummy:+ pass++ class BaseModel:+ __pydantic_serializer__: SchemaSerializer++ def __init_subclass__(cls) -> None:+ cls.__pydantic_serializer__ = SchemaSerializer(+ core_schema.model_schema(+ Dummy,+ core_schema.model_fields_schema(+ {+ 'value': core_schema.model_field(+ core_schema.int_schema(), serialization_exclude_if=lambda value, _cls=cls: False+ )+ }+ ),+ )+ )++ cache: WeakValueDictionary[int, Any] = WeakValueDictionary()++ for _ in range(10_000):++ class MyModel(BaseModel):+ pass++ cache[id(MyModel)] = MyModel++ del MyModel++ assert_gc(lambda: len(cache) == 0)
What the agent reported
Done. The working tree contains exactly the fix and its regression tests, uncommitted.
What was wrong
pydantic-core omitted several PyGcTraverse edges, so CPython's cyclic collector could not see strong references that pydantic-core held to Python objects. Transient model/enum classes that formed reference cycles through those unreported edges therefore never got collected, leaking memory. Three owned-but-untraversed fields:
SerField.serialization_exclude_if(theField(exclude_if=...)callable)EnumSerializer.class(the enum class)EnumValidator<T>.lookup(holds the enum members, which reference the class)
What I changed
Rust — added the missing fields to the existing impl_py_gc_traverse! invocations (GC-only, purely additive, no behavior change):
src/serializers/fields.rs:38→SerField { serializer, serialization_exclude_if }src/serializers/type_serializers/enum_.rs:51→EnumSerializer { class, serializer }src/validators/enum_.rs→ addedlookupto all fourEnumValidator<T>monomorphizations (Plain/Int/Str/Float), now{ class, missing, lookup }
Tests — tests/test_garbage_collection.py gained two regression tests following the file's existing idiom (transient class in a 10,000-iteration loop, WeakValueDictionary, assert_gc, same xfail/skip markers):
test_gc_enum— enum with a Python method, validator+serializer stored on the class (cycle throughlookup/class)test_gc_serialization_exclude_if— model field whoseserialization_exclude_ifcallable captures the class (cycle throughSerField.serialization_exclude_if)
Test evidence
- Reproduced first: the report's exact script printed all three objects
alive: Trueon the current build; after the fix all three printFalse. - Fails before / passes after, verified by rebuild: with the Rust fix stashed and rebuilt, both new tests fail (
AssertionError: Timeout waiting for GC) for the right reason (the cycle stays uncollectable). With the fix restored and rebuilt, both pass. - Final gate (fixed build):
tests/test_garbage_collection.py+validators/test_enums.py+serializers/test_enum.py→ 32 passed, 1 skipped, including all 5 GC tests (3 pre-existing untouched + 2 new). - Reviewer confirmed the diff correctly and minimally implements the report's fix, with no existing test weakened and no unrelated changes.
Note: cargo fmt/clippy aren't installed in this environment, but the edits are single-line macro additions matching surrounding formatting.
Requirements disposition
- Add
serialization_exclude_iftoSerFieldtraversal — Implemented; covered bytest_gc_serialization_exclude_if(fails before, passes after). - Add
classtoEnumSerializertraversal — Implemented; part of the enum fix. - Add
lookupto everyEnumValidator<T>traversal — Implemented on all four monomorphizations. The report noted both the enum serializer and validator edges are required together;test_gc_enumexercises the combined cycle (fails before, passes after). - Enum case requires both serializer and validator edges — Satisfied: all three enum-related edges added; the single combined enum regression test verifies collection.
- Report's control cases (enum without methods / field without
exclude_ifcollected normally) — Confirmed still collected; the pre-existing GC tests and the diagnostic variants (plain enum, no-store) all collect normally and remain passing.