Backtestspydanticrun-20261027-165153#11329
Bug: false Unexpected keyword argument in pydantic.mypy plugin (in very specific case)
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Both patches attack the same root cause: a subclass's transform runs while the plugin has started but not finished the base model (deferred because of unresolved forward refs in an import cycle), so the base's fields and extra='forbid' config are missing from the generated __init__/model_construct signatures. Both mark the base's TypeInfo metadata at the start of transform and make the subclass defer when it sees a marked-but-unfinished base, falling through on the final iteration. On the core scenario they are equivalent.
B is slightly more robust in the details. It removes the 'incomplete' marker when the transform completes (and explicitly on the final iteration when fields can't resolve), so the check is a true in-progress flag rather than A's permanent tag combined with an absence-of-METADATA_KEY test; that also covers the case where a base is re-transformed after its metadata was already stored. B calls defer() right where it returns (None, None), keeping the deferral paired with the wait, whereas A relies on the caller to defer. Neither leaves stale state in the cache in practice.
Tests: A adds a compact two-module cycle test that would fail before (missing inherited 'name') and asserts the real error ('nam') still fires, which is adequate, but it also edits pyproject's codespell ignore list purely to accommodate the test's typo, which is unrelated churn. B reproduces the issue's actual structure (forbid base, Node with Annotated tagged union, Circle/Square in sibling modules, TYPE_CHECKING imports) and checks valid inherited-field calls produce no errors, that extra='forbid' still rejects unknown kwargs, and that model_construct gets the inherited/own field types right. One oddity in B's test is that Square derives from Node while Circle derives from BaseShape, but that doesn't weaken it. Independent tests were all skipped and provide no signal.
Overall B wins on marginally better state handling and a more faithful, broader test, with A carrying a small unrelated config change.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
11m 46s
Wall-clock time in the container
Full test suite
No new failures
314 passed · 91 skipped · 405 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
87vs 66 merged
Changed lines, ours vs merged
The issue
View on GitHubInitial Checks
- I confirm that I'm using Pydantic V2
Description
Summary: In a particular scenario using a base class with ConfigDict(extra="forbid"), recursive models (in my case with tagged unions) can result in a false Unexpected keyword argument mypy error on the subclasses.
Example code
This issue is best explained demonstrated with an actual repo, so I created a mwe repo (which models a simple node graph) here: https://github.com/tlambert03/pydantic-mypy-err
git clone https://github.com/tlambert03/pydantic-mypy-err.git
cd pydantic-mypy-err
pip install mypy==1.14.1 pydantic==2.10.5
mypy example.py
Apologies that this can't be summed up in a simpler example, but hopefully that MWE should immediately demonstrate the problem and allow you to play with some variations.
This file contains the most interesting config, and is reproduced here:
from typing import TYPE_CHECKING, Annotated, ClassVar
from pydantic import BaseModel, ConfigDict, Field
if TYPE_CHECKING:
from .circle import Circle
# note, if square were defined after Node in THIS file, mypy is happy.
from .square import Square
class _MyBase(BaseModel):
"""Base class for all evented pydantic-style models."""
# Note: without this additional base with extra="forbid" mypy is happy.
model_config: ClassVar[ConfigDict] = ConfigDict(extra="forbid")
AnyNode = Annotated["Circle | Square", Field(discriminator="node_type")]
class Node(_MyBase):
name: str | None = Field(default=None, description="Name of the node.")
parent: AnyNode | None = None
Python, Pydantic & OS Version
pydantic version: 2.10.5
pydantic-core version: 2.27.2
pydantic-core build: profile=release pgo=false
install path: /Users/talley/Desktop/pydantic-mypy-err/.venv/lib/python3.12/site-packages/pydantic
python version: 3.12.7 (main, Oct 16 2024, 07:12:08) [Clang 18.1.8 ]
platform: macOS-15.2-arm64-arm-64bit
related packages: mypy-1.14.1 typing_extensions-4.12.2
commit: unknownThe two patches
pydantic/mypy.pyindex 52e13250..b084adec 100644@@ -86,6 +86,8 @@ from pydantic.version import parse_mypy_version CONFIGFILE_KEY = 'pydantic-mypy' METADATA_KEY = 'pydantic-mypy-metadata'+# Set on models whose transformation was started but not completed yet (e.g. because it was deferred):+INCOMPLETE_METADATA_KEY = 'pydantic-mypy-incomplete' BASEMODEL_FULLNAME = 'pydantic.main.BaseModel' CREATE_MODEL_FULLNAME = 'pydantic.main.create_model' BASESETTINGS_FULLNAME = 'pydantic_settings.main.BaseSettings'@@ -528,6 +530,8 @@ class PydanticModelTransformer: * stores the fields, config, and if the class is settings in the mypy metadata for access by subclasses """ info = self._cls.info+ # Let subclasses know that this model isn't fully processed yet (see `collect_fields_and_class_vars()`):+ info.metadata[INCOMPLETE_METADATA_KEY] = {} is_a_root_model = is_root_model(info) config = self.collect_config() fields, class_vars = self.collect_fields_and_class_vars(config, is_a_root_model)@@ -541,6 +545,9 @@ class PydanticModelTransformer: # Request another pass so that they can be resolved: if not self._api.final_iteration: self._api.defer()+ else:+ # No other pass will happen, so subclasses shouldn't wait for this model:+ del info.metadata[INCOMPLETE_METADATA_KEY] return False is_settings = info.has_base(BASESETTINGS_FULLNAME)@@ -555,6 +562,7 @@ class PydanticModelTransformer: 'class_vars': {class_var.name: class_var.serialize() for class_var in class_vars}, 'config': config.get_values_dict(), }+ del info.metadata[INCOMPLETE_METADATA_KEY] return True @@ -679,9 +687,12 @@ class PydanticModelTransformer: found_fields: dict[str, PydanticModelField] = {} found_class_vars: dict[str, PydanticModelClassVar] = {} for info in reversed(cls.info.mro[1:-1]): # 0 is the current class, -2 is BaseModel, -1 is object- # if BASEMODEL_METADATA_TAG_KEY in info.metadata and BASEMODEL_METADATA_KEY not in info.metadata:- # # We haven't processed the base class yet. Need another pass.- # return None, None+ if INCOMPLETE_METADATA_KEY in info.metadata and not self._api.final_iteration:+ # The base model was deferred (e.g. because of a forward reference in an import cycle),+ # so its fields and config aren't known yet. Using the (missing or outdated) metadata would+ # result in an incorrect signature, so wait until it is processed:+ self._api.defer()+ return None, None if METADATA_KEY not in info.metadata: continue tests/mypy/modules/forward_ref_unprocessed_base.pynew file mode 100644index 00000000..18f962a6@@ -0,0 +1,14 @@+"""+Fields and config of a parent model that the plugin has to defer (here because of forward references+in an import cycle) must still be taken into account when processing subclasses defined in other modules.+"""++from tests.mypy.modules.forward_ref_unprocessed_base_pkg.circle import Circle+from tests.mypy.modules.forward_ref_unprocessed_base_pkg.square import Square++circle = Circle(name='c', radius=2.0)+square = Square(name='s', side=3.0, parent=circle)+Circle.model_construct(name='c', parent=square)++Circle(name='c', unknown=1)+Square.model_construct(name='s', side='wrong')tests/mypy/modules/forward_ref_unprocessed_base_pkg/__init__.pynew file mode 100644index 00000000..e69de29btests/mypy/modules/forward_ref_unprocessed_base_pkg/circle.pynew file mode 100644index 00000000..fc31ff70@@ -0,0 +1,8 @@+from typing import Literal++from .node import BaseShape+++class Circle(BaseShape):+ node_type: Literal['circle'] = 'circle'+ radius: float = 1.0tests/mypy/modules/forward_ref_unprocessed_base_pkg/node.pynew file mode 100644index 00000000..9cf5270a@@ -0,0 +1,23 @@+from typing import TYPE_CHECKING, Annotated, ClassVar, Literal, Optional, Union++from pydantic import BaseModel, ConfigDict, Field++if TYPE_CHECKING:+ from .circle import Circle+ from .square import Square+++class _MyBase(BaseModel):+ model_config: ClassVar[ConfigDict] = ConfigDict(extra='forbid')+++AnyNode = Annotated[Union['Circle', 'Square'], Field(discriminator='node_type')]+++class Node(_MyBase):+ name: Optional[str] = None+ parent: Optional[AnyNode] = None+++class BaseShape(Node):+ node_type: Literal['circle', 'square']tests/mypy/modules/forward_ref_unprocessed_base_pkg/square.pynew file mode 100644index 00000000..48703226@@ -0,0 +1,8 @@+from typing import Literal++from .node import Node+++class Square(Node):+ node_type: Literal['square'] = 'square'+ side: float = 1.0tests/mypy/outputs/mypy-plugin_ini/forward_ref_unprocessed_base.pynew file mode 100644index 00000000..f59efad5@@ -0,0 +1,16 @@+"""+Fields and config of a parent model that the plugin has to defer (here because of forward references+in an import cycle) must still be taken into account when processing subclasses defined in other modules.+"""++from tests.mypy.modules.forward_ref_unprocessed_base_pkg.circle import Circle+from tests.mypy.modules.forward_ref_unprocessed_base_pkg.square import Square++circle = Circle(name='c', radius=2.0)+square = Square(name='s', side=3.0, parent=circle)+Circle.model_construct(name='c', parent=square)++Circle(name='c', unknown=1)+# MYPY: error: Unexpected keyword argument "unknown" for "Circle" [call-arg]+Square.model_construct(name='s', side='wrong')+# MYPY: error: Argument "side" to "model_construct" of "Square" has incompatible type "str"; expected "float" [arg-type]tests/mypy/test_mypy.pyindex c66f1964..03eadc2d 100644@@ -110,6 +110,7 @@ cases: list[ParameterSet | tuple[str, str]] = [ ('mypy-plugin.ini', 'plugin_strict_fields.py'), ('mypy-plugin.ini', 'final_with_default.py'), ('mypy-plugin.ini', 'create_model_var.py'),+ ('mypy-plugin.ini', 'forward_ref_unprocessed_base.py'), ('mypy-plugin-strict-no-any.ini', 'dataclass_no_any.py'), ('mypy-plugin-strict-no-any.ini', 'plugin_no_any_dynamic_alias.py'), ('mypy-plugin-very-strict.ini', 'metaclass_args.py'),
pydantic/mypy.pyindex 52e132507..548c2a8ef 100644@@ -86,6 +86,7 @@ from pydantic.version import parse_mypy_version CONFIGFILE_KEY = 'pydantic-mypy' METADATA_KEY = 'pydantic-mypy-metadata'+METADATA_TAG_KEY = 'pydantic-mypy-metadata-tag' BASEMODEL_FULLNAME = 'pydantic.main.BaseModel' CREATE_MODEL_FULLNAME = 'pydantic.main.create_model' BASESETTINGS_FULLNAME = 'pydantic_settings.main.BaseSettings'@@ -528,11 +529,17 @@ class PydanticModelTransformer: * stores the fields, config, and if the class is settings in the mypy metadata for access by subclasses """ info = self._cls.info+ # Tag the class as being processed by the plugin. This is used to detect base classes+ # that are Pydantic models for which the plugin hasn't completed processing yet (in which+ # case the `METADATA_KEY` is not yet present in the metadata):+ info.metadata.setdefault(METADATA_TAG_KEY, {}) is_a_root_model = is_root_model(info) config = self.collect_config() fields, class_vars = self.collect_fields_and_class_vars(config, is_a_root_model) if fields is None or class_vars is None: # Some definitions are not ready. We need another pass.+ if not self._api.final_iteration:+ self._api.defer() return False for field in fields: if field.type is None or has_placeholder(field.type):@@ -679,10 +686,13 @@ class PydanticModelTransformer: found_fields: dict[str, PydanticModelField] = {} found_class_vars: dict[str, PydanticModelClassVar] = {} for info in reversed(cls.info.mro[1:-1]): # 0 is the current class, -2 is BaseModel, -1 is object- # if BASEMODEL_METADATA_TAG_KEY in info.metadata and BASEMODEL_METADATA_KEY not in info.metadata:- # # We haven't processed the base class yet. Need another pass.- # return None, None if METADATA_KEY not in info.metadata:+ if METADATA_TAG_KEY in info.metadata and not self._api.final_iteration:+ # The base class is a Pydantic model, but the plugin hasn't finished processing it yet+ # (e.g. because its own processing got deferred, as it has fields with unresolved+ # forward references). We need another pass, otherwise the fields of the base+ # class would be missing from the generated signatures of this class.+ return None, None continue # Each class depends on the set of attributes in its dataclass ancestors.pyproject.tomlindex 27f62269f..9e9a6fc60 100644@@ -308,4 +308,4 @@ enableExperimentalFeatures = true skip = '.git,env*,pydantic/v1/*,uv.lock' # `ser` - abbreviation for "ser"ialisation # `crate` - a rust crate-ignore-words-list = 'gir,ser,crate,deques'+ignore-words-list = 'gir,ser,crate,deques,nam'tests/mypy/modules/import_cycle.pynew file mode 100644index 000000000..173c441f3@@ -0,0 +1,11 @@+"""https://github.com/pydantic/pydantic/issues/11329."""++from .import_cycle_base import Base+++class Sub(Base):+ side: float+++Sub(side=1.0, name='x')+Sub(side=1.0, nam='x')tests/mypy/modules/import_cycle_base.pynew file mode 100644index 000000000..8c7b3d92b@@ -0,0 +1,24 @@+"""Helper module for the `import_cycle.py` test module.++`Base` has a field referencing `Sub`, defined in `import_cycle.py`, thus creating an import cycle.+This results in the mypy plugin deferring the processing of `Base` (as `Sub` isn't yet resolved),+possibly after `Sub` (a subclass of `Base`) has already been processed.+"""++from typing import TYPE_CHECKING++from pydantic import BaseModel, ConfigDict++if TYPE_CHECKING:+ from .import_cycle import Sub+++# This base is required to make the issue reproducible (as without it, it adds `**kwargs`)+# to the `__init__()` signature:+class ForbidExtraBase(BaseModel):+ model_config = ConfigDict(extra='forbid')+++class Base(ForbidExtraBase):+ name: str | None = None+ parent: 'Sub | None' = Nonetests/mypy/outputs/mypy-plugin_ini/import_cycle.pynew file mode 100644index 000000000..7ca0af8f3@@ -0,0 +1,12 @@+"""https://github.com/pydantic/pydantic/issues/11329."""++from .import_cycle_base import Base+++class Sub(Base):+ side: float+++Sub(side=1.0, name='x')+Sub(side=1.0, nam='x')+# MYPY: error: Unexpected keyword argument "nam" for "Sub"; did you mean "name"? [call-arg]tests/mypy/test_mypy.pyindex c66f1964e..2ac32f6f9 100644@@ -110,6 +110,7 @@ cases: list[ParameterSet | tuple[str, str]] = [ ('mypy-plugin.ini', 'plugin_strict_fields.py'), ('mypy-plugin.ini', 'final_with_default.py'), ('mypy-plugin.ini', 'create_model_var.py'),+ ('mypy-plugin.ini', 'import_cycle.py'), ('mypy-plugin-strict-no-any.ini', 'dataclass_no_any.py'), ('mypy-plugin-strict-no-any.ini', 'plugin_no_any_dynamic_alias.py'), ('mypy-plugin-very-strict.ini', 'metaclass_args.py'),