Backtestspydanticrun-20261022-024030#13692
model_dump_json crashes on a SecretStr field with an unvalidated None default (internal AttributeError instead of null/warning)
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Both patches make the issue's SecretStr example stop raising, and the independent tests do not separate them. The difference is in root cause and robustness. Patch A flips `when_used` to `'unless-none'`, which only skips the custom serializer for the single value `None`; the underlying problem (the serializer dereferences `.get_secret_value()` on whatever it receives) is untouched, so any other mismatched value, e.g. `model_construct(s='hunter2')` or an unvalidated non-secret default, still surfaces an internal AttributeError wrapped in PydanticSerializationError. A also leaves the sibling `_serialize_secret` (generic `Secret[T] = None`) crashing the same way. Patch B addresses the actual cause: it type-checks the value and raises `PydanticSerializationUnexpectedValue`, which is exactly the mechanism pydantic-core uses to emit the standard mismatch UserWarning and fall back to inferred serialization, yielding `null` plus a warning, matching how `int = None` behaves and matching the issue's stated expectation of consistent degrade-gracefully behaviour. It also deliberately omits the secret value from the warning text, with a test asserting the secret does not leak, which is the correct edge handling for this type. Tests: A adds one narrow test that fails before and passes after. B adds parametrized tests across SecretStr, SecretBytes, and Secret[int], covers json, python and mode='json' paths, asserts the warning is emitted, checks Optional still produces no warning, and checks value redaction; these are meaningfully stronger. Scope: B extends the same isinstance-guard pattern to ByteSize, Base64/Encoded types and re.Pattern. That is beyond the ask and earns no credit, but the guards are conservative (warn and fall back rather than raise), so the added risk is low and does not obscure the core fix. On correctness and tests B is clearly better; A is a valid but symptomatic, None-only workaround.
Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.
Solve
9m 26s
Wall-clock time in the container
Full test suite
No new failures
1,410 passed · 40 skipped · 1,450 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
124vs 12 merged
Changed lines, ours vs merged
The issue
View on GitHubChecks
- I have checked that this issue has not already been reported.
- I have confirmed the bug exists on the latest version of pydantic (2.13.4) and on current
main.
Python Version
3.13.14
Pydantic Version
pydantic 2.13.4 / pydantic-core (latest from pip at time of report)
Description
A model field annotated SecretStr with an unvalidated None default crashes serialization to JSON. Per models docs, field defaults are only validated when validate_default=True, so constructing such a model without arguments is legal — and every other scalar annotation serializes the unvalidated default as null. For SecretStr, however, model_dump_json() raises instead.
Note this is specific to the JSON path and to the non-nullable annotation:
model_dump()(python mode) returnsNonefine;Optional[SecretStr] = Nonedumps to JSON correctly ({"s":null});- a control field
n: int = Nonealso dumps fine — so only the combination "non-nullable SecretStr + None default + JSON serializer" hits the broken path.
Example Code
import warnings
from typing import Optional
from pydantic import BaseModel, SecretStr
class M(BaseModel):
n: int = None # same pattern, control group
s: SecretStr = None # crashes
with warnings.catch_warnings(record=True) as w:
warnings.simplefilter("always")
print(M().model_dump()) # {'n': None, 's': None} -- OK
try:
print(M().model_dump_json())
except Exception as e:
print(type(e).__name__, "|", e)
print("warnings emitted:", len(w))
class O(BaseModel):
s: Optional[SecretStr] = None
print(O().model_dump_json()) # {"s":null} -- OK
Actual behavior
{'n': None, 's': None}
PydanticSerializationError | Error calling function '_serialize_secret_field': AttributeError: 'NoneType' object has no attribute 'get_secret_value'
warnings emitted: 0
{"s":null}
The exception escapes even though the value mismatch is exactly the situation the serialization layer normally degrades gracefully on (serialize as-is / warn). No UserWarning about the mismatched default is emitted either.
Looking at _serialize_secret_field, it calls .get_secret_value() unconditionally when the serializer runs with when_used='always'; there is no guard for value is None on a non-optional annotation, while core handles None defaults for other types before user serializers are reached.
Expected behavior
Either of the consistent-with-everything-else outcomes:
- serialize the unvalidated default as
null:{"n":null,"s":null}(matching howint = None,str = Noneetc. behave), or - at minimum emit the standard serialization-mismatch
UserWarninginstead of raising an internalAttributeErrorwrapped intoPydanticSerializationError.
The current traceback points users at pydantic internals rather than their own model definition.
The two patches
pydantic/_internal/_generate_schema.pyindex 3766cfee..1b7812c1 100644@@ -1825,8 +1825,16 @@ class GenerateSchema: from . import _validators metadata = {'pydantic_js_functions': [lambda _1, _2: {'type': 'string', 'format': 'regex'}]}++ def ser_pattern(pattern: Any) -> str | bytes:+ if not isinstance(pattern, re.Pattern):+ raise PydanticSerializationUnexpectedValue(+ f"Expected `{re.Pattern}` but got `{type(pattern)}` with value `'{pattern}'` - serialized value may not be as expected."+ )+ return pattern.pattern+ ser = core_schema.plain_serializer_function_ser_schema(- attrgetter('pattern'), when_used='json', return_schema=core_schema.str_schema()+ ser_pattern, when_used='json', return_schema=core_schema.str_schema() ) if pattern_type is typing.Pattern or pattern_type is re.Pattern: # bare typepydantic/types.pyindex 50f94cf4..2ad674f1 100644@@ -21,6 +21,7 @@ from typing import ( ClassVar, Generic, Literal,+ NoReturn, TypeAlias, TypeVar, cast,@@ -29,7 +30,13 @@ from uuid import UUID import annotated_types from annotated_types import BaseMetadata, MaxLen, MinLen-from pydantic_core import CoreSchema, PydanticCustomError, SchemaSerializer, core_schema+from pydantic_core import (+ CoreSchema,+ PydanticCustomError,+ PydanticSerializationUnexpectedValue,+ SchemaSerializer,+ core_schema,+) from typing_extensions import ( # noqa: UP035 (for `get_args` and `get_origin`) Protocol, TypeAliasType,@@ -1574,7 +1581,18 @@ class _SecretBase(Generic[SecretType]): raise NotImplementedError +def _raise_unexpected_serialization_value(expected: type[Any], value: Any, *, show_value: bool = True) -> NoReturn:+ # Signals pydantic-core to emit a serialization warning and fall back to inferring the type of `value`,+ # e.g. for an unvalidated default (`x: SecretStr = None`), rather than failing on a type assumption.+ value_repr = f" with value `'{value}'`" if show_value else ''+ raise PydanticSerializationUnexpectedValue(+ f'Expected `{expected}` but got `{type(value)}`{value_repr} - serialized value may not be as expected.'+ )++ def _serialize_secret(value: Secret[SecretType], info: core_schema.SerializationInfo) -> str | Secret[SecretType]:+ if not isinstance(value, Secret):+ _raise_unexpected_serialization_value(Secret, value, show_value=False) if info.mode == 'json': return str(value) else:@@ -1753,6 +1771,8 @@ def _secret_display(value: SecretType) -> str: # type: ignore def _serialize_secret_field( value: _SecretField[SecretType], info: core_schema.SerializationInfo ) -> str | _SecretField[SecretType]:+ if not isinstance(value, _SecretField):+ _raise_unexpected_serialization_value(_SecretField, value, show_value=False) if info.mode == 'json': # we want the output to always be string without the `b'` prefix for bytes, # hence we just use `secret_display`@@ -2150,10 +2170,16 @@ class ByteSize(int): custom_error_message='could not parse value and unit from byte string', ), serialization=core_schema.plain_serializer_function_ser_schema(- int, return_schema=core_schema.int_schema(ge=0)+ cls._serialize, return_schema=core_schema.int_schema(ge=0) ), ) + @staticmethod+ def _serialize(value: Any) -> int:+ if not isinstance(value, int):+ _raise_unexpected_serialization_value(ByteSize, value)+ return int(value)+ @classmethod def _validate(cls, input_value: Any, /, _: core_schema.ValidationInfo) -> ByteSize: try:@@ -2594,7 +2620,7 @@ class EncodedBytes: return core_schema.with_info_after_validator_function( function=self.decode, schema=schema,- serialization=core_schema.plain_serializer_function_ser_schema(function=self.encode),+ serialization=core_schema.plain_serializer_function_ser_schema(function=self._serialize), ) def decode(self, data: bytes, _: core_schema.ValidationInfo) -> bytes:@@ -2619,6 +2645,11 @@ class EncodedBytes: """ return self.encoder.encode(value) + def _serialize(self, value: Any) -> bytes:+ if not isinstance(value, bytes):+ _raise_unexpected_serialization_value(bytes, value)+ return self.encode(value)+ def __hash__(self) -> int: return hash(self.encoder) @@ -2693,7 +2724,7 @@ class EncodedStr: return core_schema.with_info_after_validator_function( function=self.decode_str, schema=schema,- serialization=core_schema.plain_serializer_function_ser_schema(function=self.encode_str),+ serialization=core_schema.plain_serializer_function_ser_schema(function=self._serialize), ) def decode_str(self, data: str, _: core_schema.ValidationInfo) -> str:@@ -2718,6 +2749,11 @@ class EncodedStr: """ return self.encoder.encode(value.encode()).decode() # noqa: UP008 + def _serialize(self, value: Any) -> str:+ if not isinstance(value, str):+ _raise_unexpected_serialization_value(str, value)+ return self.encode_str(value)+ def __hash__(self) -> int: return hash(self.encoder) tests/test_types.pyindex d168c449..cabbb5c0 100644@@ -1138,6 +1138,76 @@ def test_secretbytes_json(): assert Foobar(password='foo').model_dump_json() == '{"password":"**********"}' +@pytest.mark.parametrize(+ 'field_type',+ [+ pytest.param(SecretStr, id='SecretStr'),+ pytest.param(SecretBytes, id='SecretBytes'),+ pytest.param(Secret[int], id='Secret[int]'),+ pytest.param(ByteSize, id='ByteSize'),+ pytest.param(Base64Bytes, id='Base64Bytes'),+ pytest.param(Base64Str, id='Base64Str'),+ pytest.param(Base64UrlBytes, id='Base64UrlBytes'),+ pytest.param(Base64UrlStr, id='Base64UrlStr'),+ ],+)+def test_unvalidated_none_default_serialization(field_type) -> None:+ """An unvalidated `None` default on a non-nullable field should serialize as `null` with a+ serialization warning, rather than raising an internal error from the type's serializer."""++ class Model(BaseModel):+ n: int = None+ x: field_type = None++ m = Model()++ with pytest.warns(UserWarning, match='PydanticSerializationUnexpectedValue'):+ assert m.model_dump_json() == '{"n":null,"x":null}'++ with pytest.warns(UserWarning, match='PydanticSerializationUnexpectedValue'):+ assert m.model_dump() == {'n': None, 'x': None}++ with pytest.warns(UserWarning, match='PydanticSerializationUnexpectedValue'):+ assert m.model_dump(mode='json') == {'n': None, 'x': None}+++def test_secretstr_optional_none_default_serialization() -> None:+ class Model(BaseModel):+ s: SecretStr | None = None++ with warnings.catch_warnings():+ warnings.simplefilter('error')+ assert Model().model_dump_json() == '{"s":null}'+++@pytest.mark.parametrize('field_type', [SecretStr, Secret[str]], ids=['SecretStr', 'Secret[str]'])+def test_secret_unexpected_value_warning_hides_value(field_type) -> None:+ class Model(BaseModel):+ s: field_type++ m = Model.model_construct(s='hunter2')++ with pytest.warns(UserWarning, match='PydanticSerializationUnexpectedValue') as record:+ m.model_dump_json()++ assert all('hunter2' not in str(w.message) for w in record)+++@pytest.mark.parametrize(+ 'field_type', [re.Pattern, re.Pattern[str], re.Pattern[bytes]], ids=['Pattern', 'Pattern[str]', 'Pattern[bytes]']+)+def test_pattern_unvalidated_none_default_serialization(field_type) -> None:+ class Model(BaseModel):+ x: field_type = None++ m = Model()++ with pytest.warns(UserWarning, match='PydanticSerializationUnexpectedValue'):+ assert m.model_dump_json() == '{"x":null}'++ assert m.model_dump() == {'x': None}++ def test_secretbytes(): class Foobar(BaseModel): password: SecretBytes
pydantic/types.pyindex 50f94cf43..a12b19100 100644@@ -1795,7 +1795,7 @@ class _SecretField(_SecretBase[SecretType]): serialization=core_schema.plain_serializer_function_ser_schema( _serialize_secret_field, info_arg=True,- when_used='always',+ when_used='unless-none', ), ) tests/test_types.pyindex d168c4490..5d54df58f 100644@@ -2424,3 +2424,13 @@ def test_custom_serializer_override_secret_str() -> None: u = User(name='sam', password='hi') assert u.model_dump()['password'] == 'secret: **********'+++def test_secret_str_none() -> None:+ """https://github.com/pydantic/pydantic/issues/13692"""++ class Model(BaseModel):+ f: SecretStr = None++ assert Model().model_dump() == {'f': None}+ assert Model().model_dump_json() == '{"f":null}'