Backtestspydanticrun-20260930-144322#13660
None is not treated as Literal[None] when annotating the discriminator in discriminated unions.
Our patch was judged better than the merged fix.
In this judgment, Patch A = our agent, Patch B = the merged fix.
Both patches fix the root cause the same way: in `_infer_discriminator_values_for_inner_schema`, a `none` core schema is mapped to the discriminator value `[None]`, so `field: None` behaves like `Literal[None]`. Patch A's strengths: - It is a small, targeted addition that matches the existing if/elif style. - It also handles the closely related `nullable` case (e.g. `Literal['C'] | None`). That is what `X | None` compiles to, and it would otherwise still raise the same 'needs Literal' error. - Its test covers the exact issue scenario, the nullable-literal case, a duplicate-None conflict error, and the negative case that `int | None` still errors. Each of these would fail before the change. Patch B's advantage and drawbacks: - It fixes the JSON schema discriminator mapping key (`'null'` instead of `'None'`). That is a real but secondary improvement. - It rewrites the whole function as a `match` statement. This is unrelated churn that obscures the one-line fix, and it may break on Python versions below 3.10 if those are still supported. - It edits the vendored pydantic-core `TaggedUnionSchema` typing, removing `Hashable`, and drops imports from `_discriminated_union.py`. - It shrinks the existing `test_bool_discriminated_union` from a full-schema assertion to only the discriminator dict, weakening existing coverage for no reason tied to the issue. - It does not handle the `nullable` (`Literal[...] | None`) case. On correctness of the reported issue the two are equal. Patch A covers more real scenarios, has better tests, and carries far less risk and scope creep, so it is the better fix.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
1m 57s16 turns
Wall-clock time in the container
Full test suite
No new failures
12,346 passed · 500 skipped · 12,846 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
44vs 162 merged
Changed lines, ours vs merged
The issue
View on GitHubInitial Checks
- I confirm that I'm using Pydantic V2
Description
Literal[None] and None should be semantically equivalent type annotations, to the point that flake8-pyi and ruff have warnings for the former: https://docs.astral.sh/ruff/rules/redundant-none-literal/
Pydantic does however not treat them as semantically equivalent for the purposes of discriminated unions.
Example Code
from typing import Annotated, Literal
from pydantic import BaseModel, Field
class A(BaseModel):
field: Literal["A"] = "A"
class B(BaseModel):
field: None = None
class Foo(BaseModel):
a_or_b: Annotated[A | B, Field(discriminator="field")]
# Results in the error below.
# If `: None` is replaced with `: Literal[None]` then there is no error.
# ...
# pydantic.errors.PydanticUserError: Model 'B' needs field 'field' to be of type `Literal`
#
# For further information visit https://errors.pydantic.dev/2.13/u/discriminator-needs-literal
Python, Pydantic & OS Version
pydantic version: 2.13.4
pydantic-core version: 2.46.4
pydantic-core build: profile=release pgo=false
python version: 3.12.8 (main, Jan 5 2025, 05:33:15) [Clang 19.1.6 ]
platform: Linux-5.19.0-35-generic-x86_64-with-glibc2.35
related packages: typing_extensions-4.16.0 email-validator-2.3.0
commit: unknownThe two patches
pydantic/_internal/_discriminated_union.pyindex 64f23145..8a809c32 100644@@ -438,6 +438,14 @@ class _ApplyInferredDiscriminator: if schema['type'] == 'literal': return schema['expected'] + elif schema['type'] == 'none':+ # `None` is equivalent to `Literal[None]`+ return [None] # pyright: ignore[reportReturnType]++ elif schema['type'] == 'nullable':+ # `X | None` is equivalent to `X | Literal[None]`+ return [*self._infer_discriminator_values_for_inner_schema(schema['schema'], source), None] # pyright: ignore[reportReturnType]+ elif schema['type'] == 'union': # Generally when multiple values are allowed they should be placed in a single `Literal`, but # we add this case to handle the situation where a field is annotated as a `Union` of `Literal`s.tests/types/unions/test_discriminated_union.pyindex 63767910..4a349729 100644@@ -2465,6 +2465,42 @@ def test_discriminated_union_with_root_model_literal() -> None: assert exc_info.value.errors()[0]['type'] == 'union_tag_invalid' +def test_discriminated_union_none_discriminator() -> None:+ """`None` should be treated as `Literal[None]` for discriminator fields."""++ class A(BaseModel):+ field: Literal['A'] = 'A'++ class B(BaseModel):+ field: None = None++ class C(BaseModel):+ field: Literal['C'] | None++ class Foo(BaseModel):+ a_or_b: Annotated[A | B, Field(discriminator='field')]++ assert isinstance(Foo.model_validate({'a_or_b': {'field': 'A'}}).a_or_b, A)+ assert isinstance(Foo.model_validate({'a_or_b': {'field': None}}).a_or_b, B)++ with pytest.raises(ValidationError) as exc_info:+ Foo.model_validate({'a_or_b': {'field': 'B'}})+ assert exc_info.value.errors()[0]['type'] == 'union_tag_invalid'++ ta = TypeAdapter(Annotated[A | C, Field(discriminator='field')])+ assert isinstance(ta.validate_python({'field': 'C'}), C)+ assert isinstance(ta.validate_python({'field': None}), C)++ with pytest.raises(TypeError, match='Value None for discriminator'):+ TypeAdapter(Annotated[B | C, Field(discriminator='field')])++ class D(BaseModel):+ field: int | None++ with pytest.raises(PydanticUserError, match="Model 'D' needs field 'field' to be of type `Literal`"):+ TypeAdapter(Annotated[A | D, Field(discriminator='field')])++ def test_tagged_union_no_fallback_on_matched_discriminator(): """Regression test for https://github.com/pydantic/pydantic/issues/12800.
pydantic-core/python/pydantic_core/core_schema.pyindex 85f6a8c04..cb87ad5ae 100644@@ -7,7 +7,7 @@ from __future__ import annotations as _annotations import sys import warnings-from collections.abc import Callable, Generator, Hashable, Mapping+from collections.abc import Callable, Generator, Mapping from datetime import date, datetime, time, timedelta from decimal import Decimal from fractions import Fraction@@ -2761,8 +2761,8 @@ def union_schema( class TaggedUnionSchema(TypedDict, total=False): type: Required[Literal['tagged-union']]- choices: Required[dict[Hashable, CoreSchema]]- discriminator: Required[str | list[str | int] | list[list[str | int]] | Callable[[Any], Hashable]]+ choices: Required[dict[Any, CoreSchema]]+ discriminator: Required[str | list[str | int] | list[list[str | int]] | Callable[[Any], Any]] custom_error_type: str custom_error_message: str custom_error_context: dict[str, str | int | float]pydantic/_internal/_discriminated_union.pyindex 64f231459..554a4974d 100644@@ -1,6 +1,5 @@ from __future__ import annotations as _annotations -from collections.abc import Hashable, Sequence from typing import TYPE_CHECKING, Any, cast from pydantic_core import CoreSchema, core_schema@@ -132,7 +131,7 @@ class _ApplyInferredDiscriminator: # `_tagged_union_choices` is built during the call to `apply`, and will hold the choices to be included # in the output TaggedUnionSchema that will replace the union from the input schema- self._tagged_union_choices: dict[Hashable, core_schema.CoreSchema] = {}+ self._tagged_union_choices: dict[Any, core_schema.CoreSchema] = {} # `_used` is changed to True after applying the discriminator to prevent accidental reuse self._used = False@@ -303,7 +302,7 @@ class _ApplyInferredDiscriminator: def _infer_discriminator_values_for_choice( # noqa C901 self, choice: core_schema.CoreSchema, source_name: str | None- ) -> list[str | int]:+ ) -> list[Any]: """This function recurses over `choice`, extracting all discriminator values that should map to this choice. `model_name` is accepted for the purpose of producing useful error messages.@@ -323,7 +322,7 @@ class _ApplyInferredDiscriminator: ) elif choice['type'] == 'tagged-union':- values: list[str | int] = []+ values: list[Any] = [] # Ignore str/int "choices" since these are just references to other choices subchoices = [x for x in choice['choices'].values() if not isinstance(x, (str, int))] for subchoice in subchoices:@@ -374,7 +373,7 @@ class _ApplyInferredDiscriminator: def _infer_discriminator_values_for_typed_dict_choice( self, choice: core_schema.TypedDictSchema, source_name: str | None = None- ) -> list[str | int]:+ ) -> list[Any]: """This method just extracts the _infer_discriminator_values_for_choice logic specific to TypedDictSchema for the sake of readability. """@@ -388,7 +387,7 @@ class _ApplyInferredDiscriminator: def _infer_discriminator_values_for_model_choice( self, choice: core_schema.ModelFieldsSchema, source_name: str | None = None- ) -> list[str | int]:+ ) -> list[Any]: source = 'ModelFields' if source_name is None else f'Model {source_name!r}' field = choice['fields'].get(self.discriminator) if field is None:@@ -399,7 +398,7 @@ class _ApplyInferredDiscriminator: def _infer_discriminator_values_for_dataclass_choice( self, choice: core_schema.DataclassArgsSchema, source_name: str | None = None- ) -> list[str | int]:+ ) -> list[Any]: source = 'DataclassArgs' if source_name is None else f'Dataclass {source_name!r}' for field in choice['fields']: if field['name'] == self.discriminator:@@ -410,7 +409,7 @@ class _ApplyInferredDiscriminator: ) return self._infer_discriminator_values_for_field(field, source) - def _infer_discriminator_values_for_field(self, field: CoreSchemaField, source: str) -> list[str | int]:+ def _infer_discriminator_values_for_field(self, field: CoreSchemaField, source: str) -> list[Any]: if field['type'] == 'computed-field': # This should never occur as a discriminator, as it is only relevant to serialization return []@@ -429,53 +428,48 @@ class _ApplyInferredDiscriminator: ) return self._infer_discriminator_values_for_inner_schema(field['schema'], source) - def _infer_discriminator_values_for_inner_schema(- self, schema: core_schema.CoreSchema, source: str- ) -> list[str | int]:+ def _infer_discriminator_values_for_inner_schema(self, schema: core_schema.CoreSchema, source: str) -> list[Any]: """When inferring discriminator values for a field, we typically extract the expected values from a literal schema. This function does that, but also handles nested unions and defaults. """- if schema['type'] == 'literal':- return schema['expected']-- elif schema['type'] == 'union':- # Generally when multiple values are allowed they should be placed in a single `Literal`, but- # we add this case to handle the situation where a field is annotated as a `Union` of `Literal`s.- # For example, this lets us handle `Union[Literal['key'], Union[Literal['Key'], Literal['KEY']]]`- values: list[Any] = []- for choice in schema['choices']:- choice_schema = choice[0] if isinstance(choice, tuple) else choice- choice_values = self._infer_discriminator_values_for_inner_schema(choice_schema, source)- values.extend(choice_values)- return values-- elif schema['type'] == 'default':- # This will happen if the field has a default value; we ignore it while extracting the discriminator values- return self._infer_discriminator_values_for_inner_schema(schema['schema'], source)-- elif schema['type'] == 'function-after':- # After validators don't affect the discriminator values- return self._infer_discriminator_values_for_inner_schema(schema['schema'], source)-- elif schema['type'] == 'model' and schema.get('root_model'):- # Support RootModel[Literal[...]] as discriminator field type- return self._infer_discriminator_values_for_inner_schema(schema['schema'], source)-- elif schema['type'] in {'function-before', 'function-wrap', 'function-plain'}:- validator_type = repr(schema['type'].split('-')[1])- raise PydanticUserError(- f'Cannot use a mode={validator_type} validator in the'- f' discriminator field {self.discriminator!r} of {source}',- code='discriminator-validator',- )-- else:- raise PydanticUserError(- f'{source} needs field {self.discriminator!r} to be of type `Literal`',- code='discriminator-needs-literal',- )+ match schema:+ case {'type': 'literal', 'expected': expected}:+ return expected+ case {'type': 'none'}:+ return [None]+ case {'type': 'union', 'choices': choices}:+ # Generally when multiple values are allowed they should be placed in a single `Literal`, but+ # we add this case to handle the situation where a field is annotated as a `Union` of `Literal`s.+ # For example, this lets us handle `Union[Literal['key'], Union[Literal['Key'], Literal['KEY']]]`+ values: list[Any] = []+ for choice in choices:+ choice_schema = choice[0] if isinstance(choice, tuple) else choice+ choice_values = self._infer_discriminator_values_for_inner_schema(choice_schema, source)+ values.extend(choice_values)+ return values+ case {'type': 'default', 'schema': inner_schema}:+ # This will happen if the field has a default value; we ignore it while extracting the discriminator values+ return self._infer_discriminator_values_for_inner_schema(inner_schema, source)+ case {'type': 'function-after', 'schema': inner_schema}:+ # After validators don't affect the discriminator values+ return self._infer_discriminator_values_for_inner_schema(inner_schema, source)+ case {'type': 'model', 'root_model': True, 'schema': inner_schema}:+ # Support RootModel[Literal[...]] as discriminator field type+ return self._infer_discriminator_values_for_inner_schema(inner_schema, source)+ case {'type': 'function-before' | 'function-wrap' | 'function-plain' as schema_type}:+ validator_type = repr(schema_type.split('-')[1])+ raise PydanticUserError(+ f'Cannot use a mode={validator_type} validator in the'+ f' discriminator field {self.discriminator!r} of {source}',+ code='discriminator-validator',+ )+ case _:+ raise PydanticUserError(+ f'{source} needs field {self.discriminator!r} to be of type `Literal`',+ code='discriminator-needs-literal',+ ) - def _set_unique_choice_for_values(self, choice: core_schema.CoreSchema, values: Sequence[str | int]) -> None:+ def _set_unique_choice_for_values(self, choice: core_schema.CoreSchema, values: list[Any]) -> None: """This method updates `self.tagged_union_choices` so that all provided (discriminator) `values` map to the provided `choice`, validating that none of these values already map to another (different) choice. """pydantic/json_schema.pyindex c61378a5f..c2183e193 100644@@ -1384,6 +1384,8 @@ class GenerateJsonSchema: # Use the JSON representation so that the discriminator mapping # can be matched against the serialized payload value k = 'true' if k else 'false'+ elif k is None:+ k = 'null' try: # Use str(k) since keys must be strings for json; while not technically correct, # it's the closest that can be represented in valid JSONtests/test_json_schema.pyindex 2d67f01ae..9cca32928 100644@@ -3986,40 +3986,36 @@ def test_bool_discriminated_union() -> None: class Model(BaseModel): setting: Enabled | Disabled = Field(discriminator='enabled') - assert Model.model_json_schema() == {- '$defs': {- 'Disabled': {- 'properties': {'enabled': {'const': False, 'title': 'Enabled', 'type': 'boolean'}},- 'required': ['enabled'],- 'title': 'Disabled',- 'type': 'object',- },- 'Enabled': {- 'properties': {- 'config': {'title': 'Config', 'type': 'string'},- 'enabled': {'const': True, 'title': 'Enabled', 'type': 'boolean'},- },- 'required': ['enabled', 'config'],- 'title': 'Enabled',- 'type': 'object',- },- },- 'properties': {- 'setting': {- 'discriminator': {- 'mapping': {'false': '#/$defs/Disabled', 'true': '#/$defs/Enabled'},- 'propertyName': 'enabled',- },- 'oneOf': [{'$ref': '#/$defs/Enabled'}, {'$ref': '#/$defs/Disabled'}],- 'title': 'Setting',- }- },- 'required': ['setting'],- 'title': 'Model',- 'type': 'object',+ assert Model.model_json_schema()['properties']['setting']['discriminator'] == {+ 'mapping': {'false': '#/$defs/Disabled', 'true': '#/$defs/Enabled'},+ 'propertyName': 'enabled', } +def test_none_discriminated_union() -> None:+ """https://github.com/pydantic/pydantic/issues/13660"""++ class A(BaseModel):+ field: Literal['A'] = 'A'++ class B(BaseModel):+ field: None = None++ class Model(BaseModel):+ a_or_b: Annotated[A | B, Field(discriminator='field')]++ assert Model.model_json_schema()['properties']['a_or_b']['discriminator'] == {+ 'mapping': {'A': '#/$defs/A', 'null': '#/$defs/B'},+ 'propertyName': 'field',+ }++ m = Model.model_validate({'a_or_b': {'field': None}})+ assert isinstance(m.a_or_b, B)++ m2 = Model.model_validate({'a_or_b': {'field': 'A'}})+ assert isinstance(m2.a_or_b, A)++ def test_nested_discriminated_union(): class BlackCatWithHeight(BaseModel): color: Literal['black']
What the agent reported
A discriminator field annotated as plain None now works the same as Literal[None]. The new regression test fails without the fix and passes with it, and the surrounding test files still pass.
What was wrong: pydantic works out a discriminated union's tag values in _infer_discriminator_values_for_inner_schema (pydantic/_internal/_discriminated_union.py). It understood Literal[...], unions, defaults, after-validators and RootModel[Literal]. A field annotated None produces a different internal "none" schema that it didn't recognise, so it fell through to the discriminator-needs-literal error. The rest of the machinery already handles a None tag, which is why Literal[None] worked.
What I changed: two new branches in that function.
- A plain
Noneannotation now yields the tag valueNone, exactly likeLiteral[None]. X | None(i.e.Optional[X]) now yields X's tag values plusNone. By the report's own reasoningX | NonemeansX | Literal[None], and without this branchLiteral['C'] | Nonewould still fail whileLiteral['C', None]works. This goes slightly beyond the report:Optional[Literal[...]]discriminator fields used to raisediscriminator-needs-literaland are now accepted. Something likeint | Nonestill raises that error.
New test: test_discriminated_union_none_discriminator in tests/types/unions/test_discriminated_union.py covers:
- the exact example from the report:
{'field': 'A'}validates toAand{'field': None}toB, and an unknown tag givesunion_tag_invalid; Literal['C'] | None, where both'C'andNonepickC;- two union members both claiming
None, which raises the existing duplicate-tagTypeError; int | None, which still raisesdiscriminator-needs-literal.
Test evidence:
- Without the fix, the new test fails with the reported
PydanticUserError. - With the fix, it passes.
- All of
tests/types/unions/,tests/test_json_schema.pyandtests/test_types.pypass: 910 passed, 6 skipped, 6 xfailed. - The final version of the discriminated-union test file passes: 80 passed, 4 xfailed.
- A read-only review of the diff found nothing blocking and nothing unrelated changed. I added the
int | Nonecase after it pointed out that gap.
Loose end: the new branches return None in a list typed as strings and ints, so they carry # pyright: ignore comments. Pyright isn't installed here, so I didn't run a type check.
Requirements:
- Treat
NonelikeLiteral[None]for discriminators: done, with the test above. - The exact example from the report should no longer raise: done; the test reproduces it and asserts that validation succeeds.
No existing tests were changed. The changes are uncommitted in the working tree, with no scratch files.