Backtestspydanticrun-20260930-183947#13621
Memory leak: transient models using Enum methods or Field(exclude_if=...) survive GC
The harness couldn't finish this ticket.
The solver produced no patch.. No judgment was produced for this ticket. It counts as an error in the run total.
Solve
14m 41s
Wall-clock time in the container
Full test suite
Not run
The harness did not reach the test stage
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
0vs 139 merged
Changed lines, ours vs merged
The issue
View on GitHubInitial Checks
- I confirm that I'm using Pydantic V2
Description
Transient BaseModel classes are not collected when either:
- A model field references an
Enumwhose class defines a Python method. - A model field uses
Field(exclude_if=<Python callable>).
After the defining module is removed from sys.modules and every ordinary strong reference is dropped, weak references to the model and enum classes remain alive after repeated full gc.collect() calls. Repeating this construction in disposable module namespaces causes linear memory growth.
Controls using an enum without methods or a field without exclude_if are collected normally.
I encountered this in combination with a Temporal workflow sandbox. With a large interconnected model graph and high volume Temporal workflows this can result in multi-GiB process growth relatively quickly.
The likely cause is missing PyGcTraverse edges for Python objects held by pydantic-core:
SerFieldownsserialization_exclude_if, but traverses onlyserializer: https://github.com/pydantic/pydantic/blob/c67a1397327c6abc1a2e2e921af01aebbbeedf1c/pydantic-core/src/serializers/fields.rs#L26-L38EnumSerializerownsclass, but traverses onlyserializer: https://github.com/pydantic/pydantic/blob/c67a1397327c6abc1a2e2e921af01aebbbeedf1c/pydantic-core/src/serializers/type_serializers/enum_.rs- Each
EnumValidator<T>ownslookup, but traverses onlyclassandmissing: https://github.com/pydantic/pydantic/blob/c67a1397327c6abc1a2e2e921af01aebbbeedf1c/pydantic-core/src/validators/enum_.rs
Traversing py_schema does not compensate for another unreported direct reference: CPython’s collector accounts for every actual strong edge individually.
I tested a temporary build from Pydantic main at c67a1397327c6abc1a2e2e921af01aebbbeedf1c. Adding only these traversal edges made all objects in the reproduction collectible:
- Add
serialization_exclude_iftoSerFieldtraversal. - Add
classtoEnumSerializertraversal. - Add
lookupto everyEnumValidator<T>traversal.
For the enum case, changing only the serializer or only the validator was insufficient; both missing edges were required.
This appears related to earlier missing-traversal fixes such as:
- https://github.com/pydantic/pydantic/issues/5136
- https://github.com/pydantic/pydantic-core/pull/787
- https://github.com/pydantic/pydantic-core/pull/1100
I would be happy to submit a focused patch.
(please note that this analysis was LLM assisted)
Example Code
import gc
import sys
import types
import weakref
def transient_module(name, source, class_names):
module = types.ModuleType(name)
sys.modules[name] = module
exec(compile(source, f"<{name}>", "exec"), module.__dict__)
refs = tuple(
weakref.ref(module.__dict__[class_name])
for class_name in class_names
)
del sys.modules[name]
del module
return refs
enum_ref, enum_model_ref = transient_module(
"_pydantic_enum_gc_repro",
"""
from enum import Enum
from pydantic import BaseModel
class Membership(str, Enum):
FULL = "full"
def label(self) -> str:
return self.value
class Model(BaseModel):
membership: Membership
""",
("Membership", "Model"),
)
(exclude_if_model_ref,) = transient_module(
"_pydantic_exclude_if_gc_repro",
"""
from pydantic import BaseModel, Field
class Model(BaseModel):
value: int = Field(exclude_if=lambda value: False)
""",
("Model",),
)
for _ in range(5):
gc.collect()
# All three should be False because no intentional strong references remain.
print("enum alive:", enum_ref() is not None)
print("enum model alive:", enum_model_ref() is not None)
print("exclude_if model alive:", exclude_if_model_ref() is not None)
Python, Pydantic & OS Version
pydantic version: 2.12.5
pydantic-core version: 2.41.5
pydantic-core build: profile=release pgo=false
python version: 3.11.15 (main, Aug 5 2026, 03:58:57) [GCC 12.2.0]
platform: Linux-6.5.0-1018-oem-x86_64-with-glibc2.36
related packages: typing_extensions-4.15.0 pydantic-extra-types-2.10.5 mypy-1.16.0
commit: unknownThe two patches
No patch was produced.pydantic-core/src/serializers/fields.rsindex ba18ee4f3..8631d0b09 100644@@ -35,7 +35,10 @@ pub(super) struct SerField { pub serialization_exclude_if: Option<Py<PyAny>>, } -impl_py_gc_traverse!(SerField { serializer });+impl_py_gc_traverse!(SerField {+ serializer,+ serialization_exclude_if+}); impl SerField { pub fn new(pydantic-core/src/serializers/type_serializers/enum_.rsindex 0d2819ac2..b4e76133c 100644@@ -23,6 +23,8 @@ pub struct EnumSerializer { serializer: Option<Arc<CombinedSerializer>>, } +impl_py_gc_traverse!(EnumSerializer { class, serializer });+ impl BuildSerializer for EnumSerializer { const EXPECTED_TYPE: &'static str = "enum"; @@ -48,8 +50,6 @@ impl BuildSerializer for EnumSerializer { } } -impl_py_gc_traverse!(EnumSerializer { serializer });- impl TypeSerializer for EnumSerializer { fn to_python<'py>(&self, value: &Bound<'py, PyAny>, state: &mut SerializationState<'py>) -> PyResult<Py<PyAny>> { let py = value.py();pydantic-core/src/validators/enum_.rsindex 6ef685daf..28a32a874 100644@@ -168,7 +168,7 @@ impl<T: EnumValidateValue> Validator for EnumValidator<T> { #[derive(Debug, Clone)] pub struct PlainEnumValidator; -impl_py_gc_traverse!(EnumValidator<PlainEnumValidator> { class, missing });+impl_py_gc_traverse!(EnumValidator<PlainEnumValidator> { class, lookup, missing }); impl EnumValidateValue for PlainEnumValidator { fn validate_value<'py, I: Input<'py> + ?Sized>(@@ -200,7 +200,7 @@ impl EnumValidateValue for PlainEnumValidator { #[derive(Debug, Clone)] pub struct IntEnumValidator; -impl_py_gc_traverse!(EnumValidator<IntEnumValidator> { class, missing });+impl_py_gc_traverse!(EnumValidator<IntEnumValidator> { class, lookup, missing }); impl EnumValidateValue for IntEnumValidator { fn validate_value<'py, I: Input<'py> + ?Sized>(@@ -216,7 +216,7 @@ impl EnumValidateValue for IntEnumValidator { #[derive(Debug, Clone)] pub struct StrEnumValidator; -impl_py_gc_traverse!(EnumValidator<StrEnumValidator> { class, missing });+impl_py_gc_traverse!(EnumValidator<StrEnumValidator> { class, lookup, missing }); impl EnumValidateValue for StrEnumValidator { fn validate_value<'py, I: Input<'py> + ?Sized>(@@ -232,7 +232,7 @@ impl EnumValidateValue for StrEnumValidator { #[derive(Debug, Clone)] pub struct FloatEnumValidator; -impl_py_gc_traverse!(EnumValidator<FloatEnumValidator> { class, missing });+impl_py_gc_traverse!(EnumValidator<FloatEnumValidator> { class, lookup, missing }); impl EnumValidateValue for FloatEnumValidator { fn validate_value<'py, I: Input<'py> + ?Sized>(pydantic-core/tests/test_garbage_collection.pyindex e53cdc564..7ac338f77 100644@@ -1,6 +1,6 @@ import platform-import sys from collections.abc import Iterable+from enum import Enum from typing import Any from weakref import WeakValueDictionary @@ -8,7 +8,7 @@ import pytest from pydantic_core import SchemaSerializer, SchemaValidator, core_schema -from .conftest import assert_gc, is_free_threaded+from .conftest import assert_gc GC_TEST_SCHEMA_INNER = core_schema.definitions_schema( core_schema.definition_reference_schema(schema_ref='model'),@@ -21,18 +21,18 @@ GC_TEST_SCHEMA_INNER = core_schema.definitions_schema( ) -@pytest.mark.xfail(is_free_threaded and sys.version_info < (3, 14), reason='GC leaks on free-threaded (<3.14)') @pytest.mark.xfail(- condition=platform.python_implementation() == 'PyPy', reason='https://foss.heptapod.net/pypy/pypy/-/issues/3899'+ condition=platform.python_implementation() == 'PyPy', reason='https://github.com/pypy/pypy/issues/3898' ) @pytest.mark.skipif(platform.python_implementation() == 'GraalVM', reason='Cannot reliably trigger GC on GraalPy') def test_gc_schema_serializer() -> None:- # test for https://github.com/pydantic/pydantic/issues/5136+ """https://github.com/pydantic/pydantic/issues/5136"""+ class BaseModel:- __schema__: SchemaSerializer+ __pydantic_core_schema__: SchemaSerializer def __init_subclass__(cls) -> None:- cls.__schema__ = SchemaSerializer(+ cls.__pydantic_core_schema__ = SchemaSerializer( core_schema.model_schema(cls, GC_TEST_SCHEMA_INNER), config={'ser_json_timedelta': 'float'} ) @@ -50,18 +50,18 @@ def test_gc_schema_serializer() -> None: assert_gc(lambda: len(cache) == 0) -@pytest.mark.xfail(is_free_threaded and sys.version_info < (3, 14), reason='GC leaks on free-threaded (<3.14)') @pytest.mark.xfail(- condition=platform.python_implementation() == 'PyPy', reason='https://foss.heptapod.net/pypy/pypy/-/issues/3899'+ condition=platform.python_implementation() == 'PyPy', reason='https://github.com/pypy/pypy/issues/3898' ) @pytest.mark.skipif(platform.python_implementation() == 'GraalVM', reason='Cannot reliably trigger GC on GraalPy') def test_gc_schema_validator() -> None:- # test for https://github.com/pydantic/pydantic/issues/5136+ """https://github.com/pydantic/pydantic/issues/5136"""+ class BaseModel:- __validator__: SchemaValidator+ __pydantic_validator__: SchemaValidator def __init_subclass__(cls) -> None:- cls.__validator__ = SchemaValidator(+ cls.__pydantic_validator__ = SchemaValidator( schema=core_schema.model_schema(cls, GC_TEST_SCHEMA_INNER), config=core_schema.CoreConfig(extra_fields_behavior='allow'), )@@ -81,11 +81,12 @@ def test_gc_schema_validator() -> None: @pytest.mark.xfail(- condition=platform.python_implementation() == 'PyPy', reason='https://foss.heptapod.net/pypy/pypy/-/issues/3899'+ condition=platform.python_implementation() == 'PyPy', reason='https://github.com/pypy/pypy/issues/3898' ) @pytest.mark.skipif(platform.python_implementation() == 'GraalVM', reason='Cannot reliably trigger GC on GraalPy') def test_gc_validator_iterator() -> None:- # test for https://github.com/pydantic/pydantic/issues/9243+ """https://github.com/pydantic/pydantic/issues/9243"""+ class MyModel: iter: Iterable[int] @@ -114,3 +115,96 @@ def test_gc_validator_iterator() -> None: del iterable assert_gc(lambda: len(cache) == 0)+++@pytest.mark.xfail(+ condition=platform.python_implementation() == 'PyPy', reason='https://github.com/pypy/pypy/issues/3898'+)+@pytest.mark.skipif(platform.python_implementation() == 'GraalVM', reason='Cannot reliably trigger GC on GraalPy')+def test_gc_enum_serializer_class() -> None:+ """https://github.com/pydantic/pydantic/issues/13621"""++ class EnumBase:+ __pydantic_serializer__: SchemaSerializer++ cache: WeakValueDictionary[int, Any] = WeakValueDictionary()++ for _ in range(10_000):++ class MyEnum(EnumBase, str, Enum):+ FULL = 'full'++ MyEnum.__pydantic_serializer__ = SchemaSerializer(core_schema.enum_schema(MyEnum, list(MyEnum), sub_type='str'))++ cache[id(MyEnum)] = MyEnum++ del MyEnum++ assert_gc(lambda: len(cache) == 0)+++@pytest.mark.xfail(+ condition=platform.python_implementation() == 'PyPy', reason='https://github.com/pypy/pypy/issues/3898'+)+@pytest.mark.skipif(platform.python_implementation() == 'GraalVM', reason='Cannot reliably trigger GC on GraalPy')+def test_gc_enum_validator_lookup() -> None:+ """https://github.com/pydantic/pydantic/issues/13621"""++ class EnumBase:+ __pydantic_validator__: SchemaValidator++ cache: WeakValueDictionary[int, Any] = WeakValueDictionary()++ for _ in range(10_000):++ class MyEnum(EnumBase, str, Enum):+ FULL = 'full'++ MyEnum.__pydantic_validator__ = SchemaValidator(core_schema.enum_schema(MyEnum, list(MyEnum), sub_type='str'))++ cache[id(MyEnum)] = MyEnum++ del MyEnum++ assert_gc(lambda: len(cache) == 0)+++@pytest.mark.xfail(+ condition=platform.python_implementation() == 'PyPy', reason='https://github.com/pypy/pypy/issues/3898'+)+@pytest.mark.skipif(platform.python_implementation() == 'GraalVM', reason='Cannot reliably trigger GC on GraalPy')+def test_gc_field_exclude_if() -> None:+ """https://github.com/pydantic/pydantic/issues/13621"""++ class Dummy:+ pass++ class BaseModel:+ __pydantic_serializer__: SchemaSerializer++ def __init_subclass__(cls) -> None:+ cls.__pydantic_serializer__ = SchemaSerializer(+ core_schema.model_schema(+ Dummy,+ core_schema.model_fields_schema(+ {+ 'value': core_schema.model_field(+ core_schema.int_schema(), serialization_exclude_if=lambda value, _cls=cls: False+ )+ }+ ),+ )+ )++ cache: WeakValueDictionary[int, Any] = WeakValueDictionary()++ for _ in range(10_000):++ class MyModel(BaseModel):+ pass++ cache[id(MyModel)] = MyModel++ del MyModel++ assert_gc(lambda: len(cache) == 0)