Backtestspydanticrun-20261027-165153#13925

Issue #13925 Merged fix PR #13956 Run 27 Oct 2026 · run-20261027-165153 pydantic/pydantic

JSON model validation allocates an unused extras dict per model when extra isn't 'allow'

Better

Our patch was judged better than the merged fix.

In this judgment, Patch B = our agent, Patch A = the merged fix.

Both patches remove the unconditional `PyDict::new` in `validate_json_by_iteration` and both pass every independent allocation test, so they are equal on the core correctness of the reported JSON case. Patch A does it lazily via `get_or_insert_with` in the Allow branch and relies on the existing post-validation fix-up to materialize an empty dict for extra='allow' with no extra keys; that is correct and minimal. Patch B allocates eagerly only when extra='allow', which preserves the original semantics directly, and applies the same change to the Python-input path (`ValidateToModelExtra`), which had the identical wasted allocation; that extension is small, follows the same pattern, and adds no real risk. The decisive difference is tests: Patch A adds none, while Patch B adds parametrized tests across python/json modes and schema/runtime extra combinations asserting `model_extra` is a dict exactly when allow and `None` otherwise, plus a nested-model test checking each model gets its own dict. These guard the behavioral contract the refactor could break (e.g. allow with no extra keys must still yield an empty dict, separate dicts per nested model). Minor craft nit: B's `unreachable!` guards are slightly noisier than A's `get_or_insert_with`, but not enough to offset missing tests.

The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.

Solve

7m 14s

Wall-clock time in the container

Full test suite

No new failures

315 passed · 91 skipped · 406 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

102vs 10 merged

Changed lines, ours vs merged

Initial Checks

  • I have searched Google & GitHub for similar requests and couldn't find anything
  • I have read and followed the docs and still think this feature is missing

Description

Description

Optimization oportunity.

validate_json_by_iteration in pydantic-core/src/validators/model_fields.rs always creates the
__pydantic_extra__ dict, then throws it away unless extra='allow'. With the default extra='ignore', that's one
wasted dict per model instance, which adds up for JSON with many small nested models.

Proposed fix: only create the dict for 'allow'. In a microbenchmark it saved ~5% on model_validate_json for 100
small models. Performance improvement when extra=ignore and unchanged when 'allow'.

This overlaps with #13275, which also removes this allocation as part of a larger rewrite. If that's landing soon,
feel free to close this. Otherwise I have a small change with tests ready for just this small improvement and am happy to open a PR.

Version

python -c "import pydantic.version; print(pydantic.version.version_info())
             pydantic version: 2.13.5
        pydantic-core version: 2.46.5
          pydantic-core build: profile=release pgo=false
               python version: 3.14.7 (main, Aug 10 2026, 07:46:56) [GCC 16.1.1 20260728]
                     platform: Linux-7.2.7-arch1-1-x86_64-with-glibc2.44
             related packages: pydantic-settings-2.15.0 email-validator-2.3.0 typing_extensions-4.16.0 fastapi-0.142.2 pydantic-extra-types-2.11.1 mypy-2.4.0

Affected Components

The two patches

Our patch+87 −15
pydantic-core/src/validators/model_fields.rsindex dfa06902..c5834be1 100644@@ -435,12 +435,14 @@ impl ModelFieldsValidator {                 Key: BorrowInput<'py> + Clone + Into<LocItem>,                 Value: BorrowInput<'py>,             {-                type Output = ValResult<Bound<'py, PyDict>>;+                type Output = ValResult<Option<Bound<'py, PyDict>>>;                 fn consume_iterator(                     self,                     iterator: impl Iterator<Item = ValResult<(Key, Value)>>,-                ) -> ValResult<Bound<'py, PyDict>> {-                    let model_extra_dict = PyDict::new(self.py);+                ) -> ValResult<Option<Bound<'py, PyDict>>> {+                    // only allocate the extras dict when it will actually be used+                    let model_extra_dict_op =+                        matches!(self.extra_behavior, ExtraBehavior::Allow).then(|| PyDict::new(self.py));                     for item_result in iterator {                         let (raw_key, value) = item_result?;                         let either_str = match raw_key@@ -477,6 +479,9 @@ impl ModelFieldsValidator {                             }                             ExtraBehavior::Ignore => {}                             ExtraBehavior::Allow => {+                                let Some(model_extra_dict) = &model_extra_dict_op else {+                                    unreachable!("model_extra_dict is always created when extra='allow'");+                                };                                 let py_key = match self.extras_keys_validator {                                     Some(validator) => {                                         match validator.validate(self.py, raw_key.borrow_input(), self.state) {@@ -513,11 +518,11 @@ impl ModelFieldsValidator {                             }                         }                     }-                    Ok(model_extra_dict)+                    Ok(model_extra_dict_op)                 }             } -            let model_extra_dict = dict.iterate(ValidateToModelExtra {+            model_extra_dict_op = dict.iterate(ValidateToModelExtra {                 py,                 used_keys,                 errors: &mut errors,@@ -527,10 +532,6 @@ impl ModelFieldsValidator {                 extras_keys_validator: self.extras_keys_validator.as_deref(),                 state,             })??;--            if matches!(extra_behavior, ExtraBehavior::Allow) {-                model_extra_dict_op = Some(model_extra_dict);-            }         }          if !errors.is_empty() {@@ -565,7 +566,8 @@ impl ModelFieldsValidator {         let lookup_type = LookupType::from_bools(validate_by_alias, validate_by_name)?;          let model_dict = PyDict::new(py);-        let mut model_extra_dict_op: Option<Bound<PyDict>> = None;+        // only allocate the extras dict when it will actually be used+        let model_extra_dict_op = matches!(extra_behavior, ExtraBehavior::Allow).then(|| PyDict::new(py));         let mut field_results: Vec<Option<(LookupFieldInfo, &JsonValue)>> =             (0..self.fields.len()).map(|_| None).collect();         let mut errors: Vec<ValLineError> = Vec::new();@@ -575,7 +577,6 @@ impl ModelFieldsValidator {         let state = &mut state.scoped_set_data(Some(model_dict.clone()));         let state = &mut state.scoped_clear_field_error(); -        let model_extra_dict = PyDict::new(py);         for (key, value) in &**json_object {             let mut handled = false;             let key = key.as_ref();@@ -615,6 +616,9 @@ impl ModelFieldsValidator {                 }                 ExtraBehavior::Ignore => {}                 ExtraBehavior::Allow => {+                    let Some(model_extra_dict) = &model_extra_dict_op else {+                        unreachable!("model_extra_dict is always created when extra='allow'");+                    };                     let py_key: Bound<'_, PyString> = new_py_string(py, key, state.cache_str());                     if let Some(validator) = &self.extras_validator {                         match validator.validate(py, value, state) {@@ -701,10 +705,6 @@ impl ModelFieldsValidator {             model_dict.set_item(&field.name, field_value)?;         } -        if matches!(extra_behavior, ExtraBehavior::Allow) {-            model_extra_dict_op = Some(model_extra_dict);-        }-         if !errors.is_empty() {             return Err(ValError::LineErrors(errors));         }pydantic-core/tests/validators/test_model_fields.pyindex 933680b4..9b2cc631 100644@@ -1,3 +1,4 @@+import json import math import os import re@@ -1894,3 +1895,74 @@ def test_by_alias_and_name_config_interaction(         assert s.validate_python({'my_alias': 1}, by_alias=runtime_by_alias, by_name=runtime_by_name).my_field == 1     if name_allowed:         assert s.validate_python({'my_field': 1}, by_alias=runtime_by_alias, by_name=runtime_by_name).my_field == 1+++@pytest.mark.parametrize('mode', ['python', 'json'])+@pytest.mark.parametrize(+    'schema_extra_behavior,validate_fn_extra,input_value,expected_extra',+    [+        ('allow', None, {'f': 'x', 'extra_field': '123'}, {'extra_field': '123'}),+        ('allow', None, {'f': 'x'}, {}),+        ('ignore', 'allow', {'f': 'x', 'extra_field': '123'}, {'extra_field': '123'}),+        ('ignore', 'allow', {'f': 'x'}, {}),+        ('ignore', None, {'f': 'x', 'extra_field': '123'}, None),+        ('ignore', None, {'f': 'x'}, None),+        ('allow', 'ignore', {'f': 'x', 'extra_field': '123'}, None),+        ('forbid', None, {'f': 'x'}, None),+        ('allow', 'forbid', {'f': 'x'}, None),+    ],+)+def test_model_extra_only_dict_when_allow(+    mode: str,+    schema_extra_behavior: ExtraBehavior,+    validate_fn_extra: ExtraBehavior | None,+    input_value: dict[str, Any],+    expected_extra: dict[str, Any] | None,+) -> None:+    """`model_extra` is a dict (possibly empty) exactly when extra='allow', and `None` otherwise,+    for both python and JSON input."""+    v = SchemaValidator(+        core_schema.model_fields_schema(+            {'f': core_schema.model_field(core_schema.str_schema())}, extra_behavior=schema_extra_behavior+        )+    )++    if mode == 'python':+        m, model_extra, fields_set = v.validate_python(input_value, extra=validate_fn_extra)+    else:+        m, model_extra, fields_set = v.validate_json(json.dumps(input_value), extra=validate_fn_extra)++    assert m == {'f': 'x'}+    assert model_extra == expected_extra+    if expected_extra is None:+        assert model_extra is None+    assert fields_set == {'f', *(expected_extra or {})}+++@pytest.mark.parametrize('mode', ['python', 'json'])+@pytest.mark.parametrize('extra_behavior', ['allow', 'ignore', 'forbid'])+def test_model_extra_nested_models(mode: str, extra_behavior: ExtraBehavior) -> None:+    """Each nested model gets its own `model_extra` dict only when extra='allow'."""+    inner = core_schema.model_fields_schema(+        {'a': core_schema.model_field(core_schema.int_schema())}, extra_behavior=extra_behavior+    )+    v = SchemaValidator(+        core_schema.model_fields_schema(+            {'items': core_schema.model_field(core_schema.list_schema(inner))}, extra_behavior=extra_behavior+        )+    )+    input_value = {'items': [{'a': 1}, {'a': 2}]}++    if mode == 'python':+        m, model_extra, _ = v.validate_python(input_value)+    else:+        m, model_extra, _ = v.validate_json(json.dumps(input_value))++    if extra_behavior == 'allow':+        assert model_extra == {}+        extras = [item[1] for item in m['items']]+        assert extras == [{}, {}]+        assert extras[0] is not extras[1]+    else:+        assert model_extra is None+        assert [item[1] for item in m['items']] == [None, None] 
Merged fix · PR #13956+3 −7
pydantic-core/src/validators/model_fields.rsindex dfa069022..60c46329d 100644@@ -301,8 +301,8 @@ impl ModelFieldsValidator {             self.validate_by_get_item(py, input, dict, state)?         }; -        // if we have extra=allow, but we didn't create a dict because we were validating-        // from attributes, set it now so __pydantic_extra__ is always a dict if extra=allow+        // if we have extra=allow, but we didn't create a dict (e.g. because we were validating+        // from attributes or there were no extra keys), set it now so __pydantic_extra__ is always a dict if extra=allow         if matches!(extra_behavior, ExtraBehavior::Allow) && model_extra_dict_op.is_none() {             model_extra_dict_op = Some(PyDict::new(py));         }@@ -575,7 +575,6 @@ impl ModelFieldsValidator {         let state = &mut state.scoped_set_data(Some(model_dict.clone()));         let state = &mut state.scoped_clear_field_error(); -        let model_extra_dict = PyDict::new(py);         for (key, value) in &**json_object {             let mut handled = false;             let key = key.as_ref();@@ -615,6 +614,7 @@ impl ModelFieldsValidator {                 }                 ExtraBehavior::Ignore => {}                 ExtraBehavior::Allow => {+                    let model_extra_dict = model_extra_dict_op.get_or_insert_with(|| PyDict::new(py));                     let py_key: Bound<'_, PyString> = new_py_string(py, key, state.cache_str());                     if let Some(validator) = &self.extras_validator {                         match validator.validate(py, value, state) {@@ -701,10 +701,6 @@ impl ModelFieldsValidator {             model_dict.set_item(&field.name, field_value)?;         } -        if matches!(extra_behavior, ExtraBehavior::Allow) {-            model_extra_dict_op = Some(model_extra_dict);-        }-         if !errors.is_empty() {             return Err(ValError::LineErrors(errors));         }