Backtestspydanticrun-20261027-165153#13991
Parametrized generic models no longer resolvable by __module__/__qualname__ (pickle fails) since 2.14.0
Our patch was judged better than the merged fix.
In this judgment, Patch A = our agent, Patch B = the merged fix.
Both patches make `__module__` + `__qualname__` resolvable again, so in-process and cross-process pickle work under both (independent tests confirm). They differ in how. Patch A keeps `__module__` as the origin's module (the 2.13.5 behavior the issue explicitly expects and reproduces in its output) and registers the class on both the origin module and the parametrizing module, so pickling works and the 2.14 registration is preserved. It also handles the origin module not being in `sys.modules`, and deliberately skips the origin registration when parametrizing in `__main__` so cloudpickle still pickles by value. Its tests cover the issue scenario in-process, the unimported-origin edge, and a real two-process round trip. Patch B instead rewrites `__module__` of every module-scope parametrization to the caller's module. That is a broader behavioral change: the independent fresh-process test fails on B because the qualified name is now `app.Response[Page[Item]]` instead of `models.Response[Page[Item]]`, and B's own new xfail test admits a regression it introduces: a generic whose fields hold forward refs to names in the origin module (`f: 'A'`) now gets those refs evaluated in the parametrizing module on rebuild, resolving to the wrong object. Changing the module identity of parametrized classes also affects any existing pickles, docs, and tooling that keyed on the origin module. B is cleaner conceptually but trades the reported regression for a new one and departs from the expected output; A fixes the root cause of the issue as stated with lower risk and stronger tests.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
8m 18s
Wall-clock time in the container
Full test suite
No new failures
196 passed · 11 skipped · 207 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
171vs 129 merged
Changed lines, ours vs merged
The issue
View on GitHubInitial Checks
- I confirm that I'm using Pydantic V2
Description
Since 2.14.0, instances of a parametrized generic model can no longer be pickled, or otherwise looked up by qualified name, when the model is parametrized at module scope in a module other than the one defining the generic. This includes nested parametrizations like Response[Page[Item]]. It worked in 2.13.5.
create_generic_submodel registers a module-scope parametrization in module globals "and therefore allow[s] pickling". #13243 (a __slots__ fix) also changed which module receives that registration:
- reference_module_globals = sys.modules[created_model.__module__].__dict__
+ reference_module_globals = sys.modules[model_module or created_model.__module__].__dict__
created_model.__module__ is still the origin's module, but the class is now registered on the module where the subscript was written. __module__ + __qualname__ no longer resolves, so pickle can't find the class. Neither can other tools that save a class by its qualified name and import it back later, such as Prefect's to_qualified_name / from_qualified_name, which some result serializers use. The change isn't mentioned in the 2.14 changelog, so I assume it wasn't intended.
Expected: as in 2.13.5, a module-scope parametrization is resolvable at f"{cls.__module__}.{cls.__qualname__}", so pickle round-trips the instance (including nested generic type arguments) and the restored object validates as the exact parametrized class. Registering on created_model.__module__ (as before) would restore this. Registering on both modules would also keep whatever 2.14 behavior the change was meant to add.
Example Code
# models.py
from typing import Generic, TypeVar
from pydantic import BaseModel
T = TypeVar('T')
class Page(BaseModel, Generic[T]):
items: list[T]
class Response(BaseModel, Generic[T]):
result: T
# app.py
from pydantic import BaseModel
from models import Page, Response
class Item(BaseModel):
name: str
# Parametrized at module scope, in a module other than the one defining the generics.
ItemResponse = Response[Page[Item]]
# main.py
import pickle
import models
from app import Item, ItemResponse
print(f'{ItemResponse.__module__}.{ItemResponse.__qualname__}')
print('resolvable:', getattr(models, ItemResponse.__qualname__, None) is ItemResponse)
resp = ItemResponse(result={'items': [{'name': 'a'}]})
restored = pickle.loads(pickle.dumps(resp))
assert type(restored) is ItemResponse
assert isinstance(restored.result.items[0], Item)
print('round trip OK')
Output with 2.13.5:
models.Response[Page[Item]]
resolvable: True
round trip OK
Output with 2.14.1 (also reproduces on main):
models.Response[Page[Item]]
resolvable: False
_pickle.PicklingError: Can't pickle <class 'models.Response[Page[Item]]'>: attribute lookup Response[Page[Item]] on models failed
Python, Pydantic & OS Version
pydantic version: 2.14.1
pydantic-core version: 2.50.1
pydantic-core build: profile=release pgo=false
python version: 3.13.7 (main, Sep 2 2025, 14:05:52) [Clang 20.1.4 ]
platform: macOS-15.6.1-arm64-arm-64bit-Mach-O
related packages: typing_extensions-4.16.0 pydantic-extra-types-2.11.1 email-validator-2.3.0 pydantic-settings-2.15.0 fastapi-0.143.0 pyright-1.1.414 typing_extensions-4.16.0
commit: unknownThe two patches
pydantic/_internal/_generics.pyindex 15c4c56d..4fdda348 100644@@ -151,12 +151,25 @@ def create_generic_submodel( model_module, called_globally = _get_caller_frame_info(depth=3) if called_globally: # create global reference and therefore allow pickling- object_by_reference = None- reference_name = model_name- reference_module_globals = sys.modules[model_module or created_model.__module__].__dict__- while object_by_reference is not created_model:- object_by_reference = reference_module_globals.setdefault(reference_name, created_model)- reference_name += '_'+ # The reference is created in the module where the parametrization happened, and in the module+ # of the created model (i.e. the module of the origin), so that the model can be resolved using+ # its `__module__` and `__qualname__` (which is what `pickle` does).+ # The latter is skipped if the parametrization happened in `__main__`: such a parametrization+ # can't be resolved from the origin module in another process, and creating the reference would+ # make tools such as `cloudpickle` pickle the model by reference instead of by value.+ reference_modules = [created_model.__module__] if model_module != '__main__' else []+ if model_module is not None and model_module not in reference_modules:+ reference_modules.append(model_module)+ for reference_module in reference_modules:+ if reference_module not in sys.modules:+ # E.g. the origin was created with a `__module__` that isn't imported.+ continue+ object_by_reference = None+ reference_name = model_name+ reference_module_globals = sys.modules[reference_module].__dict__+ while object_by_reference is not created_model:+ object_by_reference = reference_module_globals.setdefault(reference_name, created_model)+ reference_name += '_' return created_model tests/test_generics.pyindex 013cb3c3..d4859d97 100644@@ -2,6 +2,7 @@ import gc import itertools import json import platform+import pickle import re import subprocess import sys@@ -1039,6 +1040,77 @@ def test_generic_model_from_function_pickle_fail(create_module): pickle.dumps(original) +def test_generic_model_pickle_parametrized_in_other_module(create_module) -> None:+ @create_module+ def models_module():+ from typing import Generic, TypeVar++ from pydantic import BaseModel++ T = TypeVar('T')++ class Page(BaseModel, Generic[T]):+ items: list[T]++ class Response(BaseModel, Generic[T]):+ result: T++ app_module = create_module(+ dedent(+ f"""+ from pydantic import BaseModel++ from {models_module.__name__} import Page, Response++ class Item(BaseModel):+ name: str++ ItemPage = Page[Item]+ ItemResponse = Response[Page[Item]]+ """+ )+ )++ ItemPage = app_module.ItemPage+ ItemResponse = app_module.ItemResponse+ Item = app_module.Item++ for cls in (ItemPage, ItemResponse):+ assert cls.__module__ == models_module.__name__+ assert getattr(models_module, cls.__qualname__) is cls+ # The reference in the module where the parametrization happened is kept:+ assert getattr(app_module, cls.__qualname__) is cls++ response = ItemResponse(result={'items': [{'name': 'a'}]})+ restored = pickle.loads(pickle.dumps(response))+ assert type(restored) is ItemResponse+ assert type(restored.result) is ItemPage+ assert isinstance(restored.result.items[0], Item)+ assert restored == response+ assert ItemResponse.model_validate(restored) is restored+++def test_generic_model_parametrized_origin_module_not_imported(create_module) -> None:+ module = create_module(+ dedent(+ """+ from typing import Generic, TypeVar++ from pydantic import create_model, BaseModel++ T = TypeVar('T')++ Model = create_model('Model', __base__=(BaseModel, Generic[T]), x=(T, ...), __module__='not_imported_mod')++ IntModel = Model[int]+ """+ )+ )++ assert module.IntModel(x=1).x == 1+ assert getattr(module, module.IntModel.__qualname__) is module.IntModel++ # `cloudpickle` is in the optional `testing-extra` dependency group, so the # subprocesses below may not be able to import it. @pytest.mark.skipif(@@ -1118,6 +1190,80 @@ def test_generic_model_pickle_different_module(tmp_path) -> None: assert loader.stdout.rstrip() == 'value=5' +@pytest.mark.skipif(sys.platform == 'emscripten', reason='no subprocesses on emscripten')+def test_generic_model_pickle_parametrized_in_other_module_subprocess(tmp_path) -> None:+ """Instances of models parametrized in a module other than the one defining the generic class+ can be pickled and unpickled in a separate process importing that module."""++ tmp_path.joinpath('gen_models.py').write_text(+ dedent(+ """+ from typing import Generic, TypeVar++ from pydantic import BaseModel++ T = TypeVar('T')++ class Page(BaseModel, Generic[T]):+ items: list[T]++ class Response(BaseModel, Generic[T]):+ result: T+ """+ )+ )+ tmp_path.joinpath('gen_app.py').write_text(+ dedent(+ """+ from pydantic import BaseModel++ from gen_models import Page, Response++ class Item(BaseModel):+ name: str++ ItemResponse = Response[Page[Item]]+ """+ )+ )++ creator_code = dedent(+ f"""+ import base64+ import pickle+ import sys++ sys.path.insert(0, {str(tmp_path)!r})+ from gen_app import ItemResponse++ value = ItemResponse(result={{'items': [{{'name': 'a'}}]}})+ print(base64.b64encode(pickle.dumps(value)).decode())+ """+ )+ creator = subprocess.run([sys.executable, '-c', creator_code], capture_output=True, text=True, timeout=30)+ assert creator.returncode == 0, creator.stderr+ payload = creator.stdout.splitlines()[0]++ loader_code = dedent(+ f"""+ import base64+ import pickle+ import sys++ sys.path.insert(0, {str(tmp_path)!r})+ from gen_app import Item, ItemResponse++ restored = pickle.loads(base64.b64decode({payload!r}))+ assert type(restored) is ItemResponse+ assert isinstance(restored.result.items[0], Item)+ print(repr(restored))+ """+ )+ loader = subprocess.run([sys.executable, '-c', loader_code], capture_output=True, text=True, timeout=30)+ assert loader.returncode == 0, loader.stderr+ assert loader.stdout.rstrip() == "Response[Page[Item]](result=Page[Item](items=[Item(name='a')]))"++ def test_generic_model_redefined_without_cache_fail(create_module, monkeypatch): # match identity checker otherwise we never get to the redefinition check monkeypatch.setattr('pydantic._internal._utils.all_identical', lambda left, right: False)
pydantic/_internal/_generics.pyindex 15c4c56de..618b66ef5 100644@@ -119,7 +119,11 @@ def create_generic_submodel( Returns: The created submodel. """- namespace: dict[str, Any] = {'__module__': origin.__module__}+ model_module, called_globally = _get_caller_frame_info(depth=3)+ # If parametrized at the module level, the parametrized class is registered in this module (see below).+ # Its `__module__` must match. We fall back to the module of the origin for backwards compatibility reasons.+ module = model_module if called_globally and model_module is not None else origin.__module__+ namespace: dict[str, Any] = {'__module__': module} # As per https://docs.python.org/3/reference/datamodel.html#slots: # "The action of a __slots__ declaration is not limited to the class where it is defined. # __slots__ declared in parents are available in child classes. However, instances of a@@ -149,11 +153,10 @@ def create_generic_submodel( **kwds, ) - model_module, called_globally = _get_caller_frame_info(depth=3) if called_globally: # create global reference and therefore allow pickling object_by_reference = None reference_name = model_name- reference_module_globals = sys.modules[model_module or created_model.__module__].__dict__+ reference_module_globals = sys.modules[module].__dict__ while object_by_reference is not created_model: object_by_reference = reference_module_globals.setdefault(reference_name, created_model) reference_name += '_'tests/test_forward_ref.pyindex f0e69b73e..80af7b36b 100644@@ -1145,10 +1145,10 @@ def test_lazy_local_namespace_len() -> None: @pytest.mark.xfail(- reason='While `get_cls_type_hints` uses the correct module ns for each base, `collect_model_fields` '+ reason='While `get_cls_type_hints()` uses the correct module ns for each base, `collect_model_fields()` ' 'will still use the `FieldInfo` instances from each base (see the `parent_fields_lookup` logic). ' 'This means that `f` is still a forward ref in `Foo.model_fields`, and it gets evaluated in '- '`GenerateSchema._model_schema`, where only the module of `Foo` is considered.'+ '`GenerateSchema._model_schema()`, where only the module of `Foo` is considered.' ) def test_uses_the_correct_globals_to_resolve_model_forward_refs(create_module): @create_module@@ -1174,6 +1174,73 @@ class Foo(Bar): assert module_2.Foo.model_fields['f'].annotation is int +@pytest.mark.xfail(+ reason='Same as `test_uses_the_correct_globals_to_resolve_model_forward_refs()`. As the generic model is '+ 'parametrized in `module_2`, the `__module__` of the parametrized class is `module_2` and so `f` is '+ 'evaluated in this module when rebuilding.'+)+def test_uses_the_correct_globals_to_resolve_parametrized_generic_model_forward_refs(create_module):+ @create_module+ def module_1():+ from typing import Generic, TypeVar++ from pydantic import BaseModel++ T = TypeVar('T')++ class Bar(BaseModel, Generic[T]):+ t: T+ f: 'A'++ module_2 = create_module(+ f"""+from {module_1.__name__} import Bar++A = str++BarInt = Bar[int]+ """+ )++ # Simulates `A` being defined later in `module_1`, after `Bar` got parametrized+ # (e.g. if `module_1` imports `module_2` before defining `A`):+ exec('A = int', module_1.__dict__)++ module_2.BarInt.model_rebuild()++ assert module_2.BarInt.model_fields['f'].annotation is int+++def test_resolve_forward_ref_type_argument_in_parametrizing_module(create_module):++ @create_module+ def module_1():+ from typing import Generic, TypeVar++ from pydantic import BaseModel++ T = TypeVar('T')++ class Bar(BaseModel, Generic[T]):+ t: T++ module_2 = create_module(+ f"""+from {module_1.__name__} import Bar++BarA = Bar['A']++A = int+ """+ )++ assert not module_2.BarA.__pydantic_complete__++ # `'A'` is evaluated in `module_2`, the module where `Bar` got parametrized:+ assert module_2.BarA(t='1').t == 1+ assert module_2.BarA.model_fields['t'].annotation is int++ @pytest.mark.xfail( reason='We should keep a reference to the parent frame, not `f_locals`. ' "It's probably only reasonable to support this in Python 3.14 with PEP 649."tests/test_generics.pyindex 013cb3c30..62b6f985e 100644@@ -1,6 +1,7 @@ import gc import itertools import json+import pickle import platform import re import subprocess@@ -1039,6 +1040,54 @@ def test_generic_model_from_function_pickle_fail(create_module): pickle.dumps(original) +def test_generic_model_parametrized_in_other_module_pickle(create_module) -> None:+ """https://github.com/pydantic/pydantic/issues/13991"""++ @create_module+ def module_1():+ from typing import Generic, TypeVar++ from pydantic import BaseModel++ T = TypeVar('T')++ class Page(BaseModel, Generic[T]):+ items: list[T]++ class Response(BaseModel, Generic[T]):+ result: T++ module_2 = create_module(+ f"""+from pydantic import BaseModel++from {module_1.__name__} import Page, Response++class Item(BaseModel):+ name: str++ItemResponse = Response[Page[Item]]+ """+ )++ ItemResponse = module_2.ItemResponse+ ItemPage = ItemResponse.model_fields['result'].annotation++ assert ItemResponse.__module__ == ItemPage.__module__ == module_2.__name__+ assert getattr(module_2, ItemResponse.__qualname__) is ItemResponse+ assert getattr(module_2, ItemPage.__qualname__) is ItemPage+ # The module where the generic classes are defined shouldn't be polluted:+ assert not hasattr(module_1, ItemResponse.__qualname__)+ assert not hasattr(module_1, ItemPage.__qualname__)++ original = ItemResponse(result={'items': [{'name': 'a'}]})+ loaded = pickle.loads(pickle.dumps(original))+ assert type(loaded) is ItemResponse+ assert type(loaded.result) is ItemPage+ assert type(loaded.result.items[0]) is module_2.Item+ assert loaded == original++ # `cloudpickle` is in the optional `testing-extra` dependency group, so the # subprocesses below may not be able to import it. @pytest.mark.skipif(