Backtestspydanticrun-20260923-170312#13089
Decimal JSON schema pattern rejects pydantic's own scientific notation serialization output (v2.12 regression)
The merged fix was judged better than our patch.
In this judgment, Patch A = our agent, Patch B = the merged fix.
Patch A treats only the symptom and does so incompletely. It appends the exponent group `(?:[eE][+-]?\d+)?` to Case 3 (decimal_places only) and Case 4 (no constraints), but leaves Case 1 (both max_digits and decimal_places) and Case 2 (max_digits only) untouched — those patterns still terminate in `0*$` with no exponent alternative. Since scientific notation is a function of a Decimal's exponent, not its constraints, a constrained field like `Decimal` with max_digits set will still serialize e.g. `1E+20` and be rejected by its own schema. So A does not fully resolve the reported class of bug, and its added test only exercises the unconstrained path, masking that gap. Patch B addresses the root cause: it stops emitting the fragile lookahead-based pattern by default (returning None, restoring the v2.11 no-pattern behavior that cannot reject valid output), and factors the pattern construction into overridable `get_decimal_pattern`/`build_decimal_pattern` hooks that correctly model pydantic-core's `str(Decimal)` output — plain and scientific notation, positive/negative exponents, trailing zeros, and inf/nan across both validation and serialization modes and all four constraint combinations. Its test suite is substantially stronger: it round-trips actual `dump_json` output against the generated pattern across many constraint/exponent/inf-nan cases, precisely the scenarios the issue names, and would fail before the change. B is far larger in scope, but the size is justified — the old pattern was fundamentally too brittle to patch in place, as A's incomplete result demonstrates. The accompanying docs version-changed notes make the intended behavior explicit. B wins decisively on correctness and tests, which outweigh A's smaller footprint.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
10m 35s17 turns
Wall-clock time in the container
Full test suite
No new failures
6,684 passed · 373 skipped · 7,057 total
Contamination audit
Unknown
See the audit notes for this run
Patch size
65vs 573 merged
Changed lines, ours vs merged
The issue
View on GitHubInitial Checks
- I confirm that I'm using Pydantic V2
Description
model_json_schema(mode="serialization") pattern for Decimal breaks on scientific notation
The serialization schema for Decimal fields generates a regex that rejects values that model_dump_json() itself produces.
Specifically: pydantic-core's serializer calls str() on Decimal objects, which outputs scientific notation for small/large values. E.g. Decimal('0.0000001') becomes "1E-7". But the pattern added in #11987 (^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$) has no [eE] component, so it'll never match those strings.
As a result, any downstream consumer validating model_dump_json() output against model_json_schema(), such as OpenAPI validators and codegen + validation tools like Orval + Zod, will intermittently reject perfectly valid serialized values. Intermittent because only certain Decimal values trigger scientific notation.
v2.11:
{"anyOf": [{"type": "string"}, {"type": "null"}]}
v2.12:
{"anyOf": [{"pattern": "^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$", "type": "string"}, {"type": "null"}]}
Two possible fixes:
- Append
(?:[eE][+-]?\d+)?to the pattern in serialization mode - Normalize
Decimalto fixed-point during serialization before it ever hits the schema
Example Code
import json
import re
from decimal import Decimal
from pydantic import BaseModel
class MyModel(BaseModel):
value: Decimal | None
schema = MyModel.model_json_schema(mode="serialization")
pattern = next(
v for v in schema["properties"]["value"]["anyOf"] if v.get("type") == "string"
)["pattern"]
regex = re.compile(pattern)
# pydantic serializes this as "1E-7" which fails its own schema's pattern
m = MyModel(value=Decimal("0.0000001"))
serialized = json.loads(m.model_dump_json())["value"]
print(f"Pattern: {pattern}")
print(f"Serialized: {serialized!r}")
print(f"Matches: {bool(regex.match(serialized))}")
# Pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$
# Serialized: '1E-7'
# Matches: False
Python, Pydantic & OS Version
pydantic version: 2.12.5
pydantic-core version: 2.41.5
pydantic-core build: profile=release pgo=false
python version: 3.12.11 | packaged by conda-forge | (main, Jun 4 2025, 14:38:53) [Clang 18.1.8 ]
platform: macOS-26.2-arm64-arm-64bit
related packages: pyright-1.1.408 pydantic-extra-types-2.10.5 typing_extensions-4.15.0 fastapi-0.123.0 pydantic-settings-2.10.1
commit: unknownThe two patches
docs/concepts/json_schema.mdindex cd8bc01a..03f8d11b 100644@@ -278,7 +278,7 @@ print(Model.model_json_schema(mode='validation')) 'anyOf': [ {'type': 'number'}, {- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*(?:[eE][+-]?\\d+)?$', 'type': 'string', }, ],@@ -297,7 +297,7 @@ print(Model.model_json_schema(mode='serialization')) 'properties': { 'a': { 'default': '12.34',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*(?:[eE][+-]?\\d+)?$', 'title': 'A', 'type': 'string', }pydantic/json_schema.pyindex 8135c60b..4f8fc7a9 100644@@ -757,11 +757,13 @@ class GenerateJsonSchema: # Case 3: Only decimal_places is set elif max_digits is None and decimal_places is not None:- pattern += rf'\d*\.?\d{{0,{decimal_places}}}0*$'+ pattern += rf'\d*\.?\d{{0,{decimal_places}}}0*(?:[eE][+-]?\d+)?$' # Case 4: Both are None (no restrictions) else:- pattern += r'\d*\.?\d*$' # look for arbitrary integer or decimal+ # allow arbitrary integer/decimal, optionally in scientific notation+ # (pydantic-core serializes some Decimals as e.g. "1E-7")+ pattern += r'\d*\.?\d*(?:[eE][+-]?\d+)?$' return pattern tests/test_json_schema.pyindex b1907b72..ff29baae 100644@@ -548,6 +548,31 @@ def test_enum_schema_cleandoc(): } +def test_decimal_serialization_schema_matches_scientific_notation():+ import json+ import re++ class MyModel(BaseModel):+ value: Decimal | None++ schema = MyModel.model_json_schema(mode='serialization')+ string_schema = next(+ v for v in schema['properties']['value']['anyOf'] if v.get('type') == 'string'+ )+ pattern = re.compile(string_schema['pattern'])++ # pydantic-core serializes this Decimal in scientific notation ("1E-7"),+ # which must still match the schema's own pattern.+ m = MyModel(value=Decimal('0.0000001'))+ serialized = json.loads(m.model_dump_json())['value']+ assert serialized == '1E-7'+ assert pattern.match(serialized) is not None++ # A few more values that stringify to scientific notation.+ for raw in ['1E-7', '1E+20', '-2.5E-10']:+ assert pattern.match(str(Decimal(raw))) is not None++ def test_decimal_json_schema(): class Model(BaseModel): a: bytes = b'foobar'@@ -564,7 +589,7 @@ def test_decimal_json_schema(): {'type': 'number'}, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*(?:[eE][+-]?\\d+)?$', }, ], 'default': '12.34',@@ -581,7 +606,7 @@ def test_decimal_json_schema(): 'default': '12.34', 'title': 'B', 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*(?:[eE][+-]?\\d+)?$', }, }, 'title': 'Model',@@ -1172,7 +1197,7 @@ def test_special_decimal_types(field_type, expected_schema): {'type': 'number'}, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*(?:[eE][+-]?\\d+)?$', }, ], 'title': 'A',@@ -2175,7 +2200,7 @@ def test_docstring(docstring, description): {'exclusiveMinimum': 2.0, 'type': 'number'}, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*(?:[eE][+-]?\\d+)?$', }, ] },@@ -2188,7 +2213,7 @@ def test_docstring(docstring, description): {'type': 'number', 'exclusiveMaximum': 5}, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*(?:[eE][+-]?\\d+)?$', }, ] },@@ -2201,7 +2226,7 @@ def test_docstring(docstring, description): {'type': 'number', 'minimum': 2}, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*(?:[eE][+-]?\\d+)?$', }, ] },@@ -2214,7 +2239,7 @@ def test_docstring(docstring, description): {'type': 'number', 'maximum': 5}, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*(?:[eE][+-]?\\d+)?$', }, ] },@@ -2227,7 +2252,7 @@ def test_docstring(docstring, description): {'type': 'number', 'multipleOf': 5}, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*(?:[eE][+-]?\\d+)?$', }, ] },@@ -2276,7 +2301,7 @@ def test_constraints_schema_validation(kwargs, type_, expected_extra): Decimal, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*(?:[eE][+-]?\\d+)?$', }, ), (@@ -2284,7 +2309,7 @@ def test_constraints_schema_validation(kwargs, type_, expected_extra): Decimal, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*(?:[eE][+-]?\\d+)?$', }, ), (@@ -2292,7 +2317,7 @@ def test_constraints_schema_validation(kwargs, type_, expected_extra): Decimal, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*(?:[eE][+-]?\\d+)?$', }, ), (@@ -2300,7 +2325,7 @@ def test_constraints_schema_validation(kwargs, type_, expected_extra): Decimal, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*(?:[eE][+-]?\\d+)?$', }, ), (@@ -2308,7 +2333,7 @@ def test_constraints_schema_validation(kwargs, type_, expected_extra): Decimal, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*(?:[eE][+-]?\\d+)?$', }, ), ],@@ -6185,14 +6210,14 @@ def test_generate_definitions_for_no_ref_schemas(): { ('Decimal', 'serialization'): { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*(?:[eE][+-]?\\d+)?$', }, ('Decimal', 'validation'): { 'anyOf': [ {'type': 'number'}, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*(?:[eE][+-]?\\d+)?$', }, ] },
docs/api/standard_library_types.mdindex 4070fda59..5802bb2e3 100644@@ -300,11 +300,48 @@ Decimals support the following constraints (numbers must be coercible to decimal | `gt` | The value must be strictly greater than this number | [`exclusiveMinimum`](https://json-schema.org/understanding-json-schema/reference/numeric#range) keyword | | `multiple_of` | The value must be a multiple of this number | [`multipleOf`](https://json-schema.org/understanding-json-schema/reference/numeric#multiples) keyword | | `allow_inf_nan` | Whether to allow NaN (not-a-number) and infinite values | N/A |-| `max_digits` | The maximum number of decimal digits allowed. The zero before the decimal point and trailing zeros are not counted. | [`pattern`](https://json-schema.org/understanding-json-schema/reference/string#regexp) keyword, to describe the string pattern |-| `decimal_places` | The maximum number of decimal places allowed. Trailing zeros are not counted. | [`pattern`](https://json-schema.org/understanding-json-schema/reference/string#regexp) keyword, to describe the string pattern |+| `max_digits` | The maximum number of decimal digits allowed. The zero before the decimal point and trailing zeros are not counted. | N/A (see below) |+| `decimal_places` | The maximum number of decimal places allowed. Trailing zeros are not counted. | N/A (see below) | -Note that the JSON Schema [`pattern`](https://json-schema.org/understanding-json-schema/reference/string#regexp) keyword will be specified-in the JSON Schema to describe the string pattern in all cases (and can vary if `max_digits` and/or `decimal_places` is specified).+/// version-changed | v2.12+The JSON Schema includes a [`pattern`](https://json-schema.org/understanding-json-schema/reference/string#regexp) matching the `max_digits`+and `decimal_places` constraints.+///++/// version-changed | v2.14+The JSON Schema no longer includes a [`pattern`](https://json-schema.org/understanding-json-schema/reference/string#regexp) by default,+as the generated pattern can cause issues for downstream consumers due to its complexity.++The pattern can be included by subclassing [`GenerateJsonSchema`][pydantic.json_schema.GenerateJsonSchema] and overriding the+[`get_decimal_pattern()`][pydantic.json_schema.GenerateJsonSchema.get_decimal_pattern] method:++```python+from decimal import Decimal+from typing import Annotated++from pydantic import Field, TypeAdapter+from pydantic.json_schema import GenerateJsonSchema+++class MyGenerateJsonSchema(GenerateJsonSchema):+ def get_decimal_pattern(self, schema):+ return self.build_decimal_pattern(schema)+++ta = TypeAdapter(Annotated[Decimal, Field(max_digits=5, decimal_places=2)])+print(+ ta.json_schema(schema_generator=MyGenerateJsonSchema, mode='serialization')+)+"""+{+ 'pattern': '^-?(?:(?:0|[1-9][0-9]{0,2})(?:\\.[0-9]{1,2}0*)?|0E[+-][1-9][0-9]*|[1-9](?:\\.[0-9]+)?E\\+[1-2])$',+ 'type': 'string',+}+"""+```++The generated pattern also no longer uses lookahead assertions, and takes exponents and `allow_inf_nan` into account.+/// These constraints can be provided using the [`Field()`][pydantic.Field] function. The `Le`, `Ge`, `Lt`, `Gt` and `MultipleOf` metadata types from the [`annotated-types`](https://github.com/annotated-types/annotated-types)docs/concepts/json_schema.mdindex cd8bc01a8..e0a7bdf4c 100644@@ -275,13 +275,7 @@ print(Model.model_json_schema(mode='validation')) { 'properties': { 'a': {- 'anyOf': [- {'type': 'number'},- {- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- 'type': 'string',- },- ],+ 'anyOf': [{'type': 'number'}, {'type': 'string'}], 'default': '12.34', 'title': 'A', }@@ -294,14 +288,7 @@ print(Model.model_json_schema(mode='validation')) print(Model.model_json_schema(mode='serialization')) """ {- 'properties': {- 'a': {- 'default': '12.34',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- 'title': 'A',- 'type': 'string',- }- },+ 'properties': {'a': {'default': '12.34', 'title': 'A', 'type': 'string'}}, 'title': 'Model', 'type': 'object', }pydantic/json_schema.pyindex 8135c60b5..5c585426d 100644@@ -723,49 +723,11 @@ class GenerateJsonSchema: Returns: The generated JSON schema. """-- def get_decimal_pattern(schema: core_schema.DecimalSchema) -> str:- max_digits = schema.get('max_digits')- decimal_places = schema.get('decimal_places')-- pattern = (- r'^(?!^[-+.]*$)[+-]?0*' # check it is not empty string and not one or sequence of ".+-" characters.- )-- # Case 1: Both max_digits and decimal_places are set- if max_digits is not None and decimal_places is not None:- integer_places = max(0, max_digits - decimal_places)- pattern += (- rf'(?:'- rf'\d{{0,{integer_places}}}'- rf'|'- rf'(?=[\d.]{{1,{max_digits + 1}}}0*$)'- rf'\d{{0,{integer_places}}}\.\d{{0,{decimal_places}}}0*$'- rf')'- )-- # Case 2: Only max_digits is set- elif max_digits is not None and decimal_places is None:- pattern += (- rf'(?:'- rf'\d{{0,{max_digits}}}'- rf'|'- rf'(?=[\d.]{{1,{max_digits + 1}}}0*$)'- rf'\d*\.\d*0*$'- rf')'- )-- # Case 3: Only decimal_places is set- elif max_digits is None and decimal_places is not None:- pattern += rf'\d*\.?\d{{0,{decimal_places}}}0*$'-- # Case 4: Both are None (no restrictions)- else:- pattern += r'\d*\.?\d*$' # look for arbitrary integer or decimal-- return pattern-- json_schema = self.str_schema(core_schema.str_schema(pattern=get_decimal_pattern(schema)))+ str_schema = core_schema.str_schema()+ pattern = self.get_decimal_pattern(schema)+ if pattern is not None:+ str_schema['pattern'] = pattern+ json_schema = self.str_schema(str_schema) if self.mode == 'validation': multiple_of = schema.get('multiple_of') le = schema.get('le')@@ -789,6 +751,63 @@ class GenerateJsonSchema: } return json_schema + def get_decimal_pattern(self, schema: core_schema.DecimalSchema) -> str | None:+ """Get the regular expression pattern to apply to the string representation of a decimal.++ By default, no pattern is applied (`None` is returned). Subclasses can override this method+ to return a custom pattern, or [`build_decimal_pattern()`][pydantic.json_schema.GenerateJsonSchema.build_decimal_pattern]+ can be used to get a pattern reflecting the `max_digits` and `decimal_places` constraints:++ ```python+ from pydantic.json_schema import GenerateJsonSchema++ class MyGenerateJsonSchema(GenerateJsonSchema):+ def get_decimal_pattern(self, schema):+ return self.build_decimal_pattern(schema)+ ```++ Args:+ schema: The core schema.++ Returns:+ The regular expression pattern, or `None` if no pattern should be applied.+ """+ return None++ def build_decimal_pattern(self, schema: core_schema.DecimalSchema) -> str:+ """Build a regular expression pattern for the string representation of a decimal.++ The pattern takes into account the `max_digits` and `decimal_places` constraints, and the current+ mode:++ - In `'validation'` mode, the pattern accepts the string syntax understood by the [`Decimal`][decimal.Decimal]+ constructor. The `max_digits` and `decimal_places` constraints are only reflected for strings+ *without* an exponent: strings using an exponent are accepted as long as they are syntactically valid,+ as the constraints can't be expressed with a regular expression in this case.+ - In `'serialization'` mode, the pattern exactly describes the strings produced by pydantic+ (i.e. by `str(decimal)`) for decimals satisfying the constraints,+ including the ones using scientific notation (e.g. `'1E-7'`, `'1.5E+3'`).++ In both modes, trailing zeros in the decimal part are not counted (e.g. `'1.10'` is accepted with+ `decimal_places=1`), consistent with the validation behavior. If `allow_inf_nan` is enabled (on the+ core schema or in the configuration), the string representations of infinity and NaN are also accepted.++ Args:+ schema: The core schema.++ Returns:+ The regular expression pattern.+ """+ max_digits = schema.get('max_digits')+ decimal_places = schema.get('decimal_places')+ allow_inf_nan = schema.get('allow_inf_nan')+ if allow_inf_nan is None:+ allow_inf_nan = self._config.config_dict.get('allow_inf_nan', False)+ if self.mode == 'validation':+ return _decimal_validation_pattern(max_digits, decimal_places, allow_inf_nan)+ else:+ return _decimal_serialization_pattern(max_digits, decimal_places, allow_inf_nan)+ def str_schema(self, schema: core_schema.StringSchema) -> JsonSchemaValue: """Generates a JSON schema that matches a string value. @@ -2773,15 +2792,153 @@ def models_json_schema( # ##### End JSON Schema Generation Functions ##### -_HashableJsonValue: TypeAlias = (- int | float | str | bool | None | tuple['_HashableJsonValue', ...] | tuple[tuple[str, '_HashableJsonValue'], ...]-)+# Decimal JSON Schema pattern utilities.+# Note: while we could use `\d` to represent digits (`Decimal` objects allow Unicode decimal digits), we use [0-9] instead:+# - to reduce complexity (`\d` expands to a large set, and with `max_digits`/`decimal_places` constraints set, it can blow+# the compile limit on some engines).+# - [0-9] is a portable pattern (e.g. `\d` is equivalent to `[0-9]` in ECMA (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Regular_expressions/Character_class_escape#d)+# while it allows any Unicode decimal digit in Python.+++def _repeat(atom: str, min_count: int, max_count: int) -> str:+ """Build a regex repetition of `atom` between `min_count` and `max_count` times (inclusive), using the shortest quantifier."""+ if min_count > max_count or max_count == 0:+ return ''+ if min_count == max_count:+ return atom if min_count == 1 else f'{atom}{{{min_count}}}'+ if (min_count, max_count) == (0, 1):+ return f'{atom}?'+ return f'{atom}{{{min_count},{max_count}}}'+++def _int_range_pattern(max_value: int) -> str:+ """Build a regex matching the integers between 1 and `max_value` (inclusive), without leading zeros."""+ digits = str(max_value)+ length = len(digits)+ alts: list[str] = [f'[1-9]{_repeat("[0-9]", k - 1, k - 1)}' for k in range(1, length)]+ for i, char in enumerate(digits):+ d = int(char)+ low = 1 if i == 0 else 0+ high = d if i == length - 1 else d - 1+ if high >= low:+ char_class = str(low) if low == high else f'[{low}-{high}]'+ alts.append(f'{digits[:i]}{char_class}{_repeat("[0-9]", length - i - 1, length - i - 1)}')+ return alts[0] if len(alts) == 1 else f'(?:{"|".join(alts)})'+++def _decimal_validation_pattern(max_digits: int | None, decimal_places: int | None, allow_inf_nan: bool) -> str:+ """Build the decimal pattern for the `'validation'` mode."""++ number = r'(?:[0-9]+\.?[0-9]*|\.[0-9]+)' # e.g. 1, 1., .5+ exponent = r'[eE][+-]?[0-9]+' # e.g. e5, E5, e-7+ # e.g. Infinity, inf, NaN, sNaN, NaN123 (case-insensitive). Spelled out explicitly, as inline flags+ # aren't supported by all regular expression engines:+ non_finite = r'|[iI][nN][fF](?:[iI][nN][iI][tT][yY])?|[sS]?[nN][aA][nN][0-9]*' if allow_inf_nan else ''+ if max_digits is None and decimal_places is None:+ return rf'^[+-]?(?:{number}(?:{exponent})?{non_finite})$'++ alts: list[str] = []+ if max_digits is not None and decimal_places is not None:+ # If `decimal_places > max_digits`, no whole digits are allowed and `max_digits` bounds the decimal places:+ whole = max(0, max_digits - decimal_places)+ places = min(max_digits, decimal_places)+ int_part = f'0*{_repeat("[0-9]", 1, whole)}' if whole else '0+'+ opt_int_part = f'0*{_repeat("[0-9]", 0, whole)}'+ opt_frac_part = rf'\.{_repeat("[0-9]", 0, places)}0*'+ frac_part = rf'\.{_repeat("[0-9]", 1, places)}0*' if places else r'\.0+'+ alts += [f'{int_part}(?:{opt_frac_part})?', f'{opt_int_part}{frac_part}']+ elif max_digits is not None:+ # The whole part and decimal part must have at most `max_digits` digits in total,+ # so we enumerate the possible widths of the whole part:+ int_part = f'0*{_repeat("[0-9]", 1, max_digits)}' if max_digits else '0+'+ alts.append(rf'{int_part}(?:\.0*)?')+ alts += [+ rf'0*{_repeat("[0-9]", whole, whole)}\.{_repeat("[0-9]", 1, max_digits - whole)}0*'+ for whole in range(max_digits)+ ] or [r'0*\.0+']+ else:+ assert decimal_places is not None+ opt_frac_part = rf'\.{_repeat("[0-9]", 0, decimal_places)}0*'+ frac_part = rf'\.{_repeat("[0-9]", 1, decimal_places)}0*' if decimal_places else r'\.0+'+ alts += [f'[0-9]+(?:{opt_frac_part})?', f'[0-9]*{frac_part}']++ alts.append(f'{number}{exponent}')+ return rf'^[+-]?(?:{"|".join(alts)}{non_finite})$'+++def _decimal_serialization_pattern(max_digits: int | None, decimal_places: int | None, allow_inf_nan: bool) -> str:+ """Build the decimal pattern for the `'serialization'` mode.++ Decimals are serialized using `str()`, producing either:++ - The plain notation (`'123.45'`), when the exponent is lower than or equal to 0 and the adjusted+ exponent is greater than or equal to -6.+ - The scientific notation otherwise (`'1E+3'`, `'1.5E-7'`, `'0E-10'`), with a single digit before+ the optional decimal point and an always signed exponent.++ Trailing zeros of the decimal part are preserved by `str()`, but aren't counted by the validation logic.+ """++ def frac_part(places: int) -> str:+ return rf'\.{_repeat("[0-9]", 1, places)}0*' if places else r'\.0+'++ alts: list[str] = []++ # Plain notation:+ if max_digits is not None and decimal_places is not None:+ whole = max(0, max_digits - decimal_places)+ places = min(max_digits, decimal_places)+ int_part = f'(?:0|[1-9]{_repeat("[0-9]", 0, whole - 1)})' if whole else '0'+ alts.append(f'{int_part}(?:{frac_part(places)})?')+ elif max_digits is not None:+ # The whole part and decimal part must have at most `max_digits` digits in total,+ # so we enumerate the possible widths of the whole part:+ alts.append(rf'0(?:\.{_repeat("[0-9]", 1, max_digits)}0*)?')+ alts += [+ f'[1-9]{_repeat("[0-9]", whole - 1, whole - 1)}(?:{frac_part(max_digits - whole)})?'+ for whole in range(1, max_digits + 1)+ ]+ elif decimal_places is not None:+ alts.append(f'(?:0|[1-9][0-9]*)(?:{frac_part(decimal_places)})?')+ else:+ alts.append(r'(?:0|[1-9][0-9]*)(?:\.[0-9]+)?')++ # Zero with an exponent (e.g. `'0E-10'`), which normalizes to zero and thus satisfies any constraint:+ alts.append(r'0E[+-][1-9][0-9]*')++ # Scientific notation with a positive exponent `k` (e.g. `'1.5E+3'`): the value has `k + 1` whole digits+ # and no decimal places:+ if max_digits is None:+ alts.append(r'[1-9](?:\.[0-9]+)?E\+[1-9][0-9]*')+ else:+ max_exponent = max_digits - (decimal_places or 0) - 1+ if max_exponent >= 1:+ alts.append(rf'[1-9](?:\.[0-9]+)?E\+{_int_range_pattern(max_exponent)}')++ # Scientific notation with a negative exponent `k` (`k >= 7`, e.g. `'1.5E-7'`): the value has no whole digits,+ # and `k` + (the number of significant decimal digits of the coefficient) decimal places:+ if max_digits is None and decimal_places is None:+ alts.append(r'[1-9](?:\.[0-9]+)?E-[1-9][0-9]*')+ else:+ max_places = min(c for c in (max_digits, decimal_places) if c is not None)+ alts += [f'[1-9](?:{frac_part(max_places - k)})?E-{k}' for k in range(7, max_places + 1)]++ if allow_inf_nan:+ # `str()` produces `'Infinity'`, `'NaN'` and `'sNaN'` (with an optional payload):+ alts += ['Infinity', 's?NaN[0-9]*']++ return rf'^-?(?:{"|".join(alts)})$' def _deduplicate_schemas(schemas: Iterable[JsonDict]) -> list[JsonDict]: return list({_make_json_hashable(schema): schema for schema in schemas}.values()) +_HashableJsonValue: TypeAlias = (+ int | float | str | bool | None | tuple['_HashableJsonValue', ...] | tuple[tuple[str, '_HashableJsonValue'], ...]+)++ def _make_json_hashable(value: JsonValue) -> _HashableJsonValue: if isinstance(value, dict): return tuple(sorted((k, _make_json_hashable(v)) for k, v in value.items()))tests/test_json_schema.pyindex b1907b720..bb38003f8 100644@@ -36,6 +36,7 @@ from typing_extensions import TypeAliasType, TypedDict, deprecated import pydantic from pydantic import ( AfterValidator,+ AllowInfNan, BaseModel, BeforeValidator, Field,@@ -64,6 +65,7 @@ from pydantic.json_schema import ( DEFAULT_REF_TEMPLATE, Examples, GenerateJsonSchema,+ JsonSchemaMode, JsonSchemaValue, NoDefault, PydanticJsonSchemaWarning,@@ -562,10 +564,7 @@ def test_decimal_json_schema(): 'b': { 'anyOf': [ {'type': 'number'},- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ], 'default': '12.34', 'title': 'B',@@ -581,7 +580,6 @@ def test_decimal_json_schema(): 'default': '12.34', 'title': 'B', 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$', }, }, 'title': 'Model',@@ -1170,10 +1168,7 @@ def test_special_decimal_types(field_type, expected_schema): 'a': { 'anyOf': [ {'type': 'number'},- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ], 'title': 'A', }@@ -2173,10 +2168,7 @@ def test_docstring(docstring, description): { 'anyOf': [ {'exclusiveMinimum': 2.0, 'type': 'number'},- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ] }, ),@@ -2186,10 +2178,7 @@ def test_docstring(docstring, description): { 'anyOf': [ {'type': 'number', 'exclusiveMaximum': 5},- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ] }, ),@@ -2199,10 +2188,7 @@ def test_docstring(docstring, description): { 'anyOf': [ {'type': 'number', 'minimum': 2},- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ] }, ),@@ -2212,10 +2198,7 @@ def test_docstring(docstring, description): { 'anyOf': [ {'type': 'number', 'maximum': 5},- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ] }, ),@@ -2225,10 +2208,7 @@ def test_docstring(docstring, description): { 'anyOf': [ {'type': 'number', 'multipleOf': 5},- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ] }, ),@@ -2274,42 +2254,27 @@ def test_constraints_schema_validation(kwargs, type_, expected_extra): ( {'gt': 2}, Decimal,- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ), ( {'lt': 5}, Decimal,- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ), ( {'ge': 2}, Decimal,- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ), ( {'le': 5}, Decimal,- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ), ( {'multiple_of': 5}, Decimal,- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ), ], )@@ -6183,17 +6148,11 @@ def test_generate_definitions_for_no_ref_schemas(): ) assert result == ( {- ('Decimal', 'serialization'): {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ ('Decimal', 'serialization'): {'type': 'string'}, ('Decimal', 'validation'): { 'anyOf': [ {'type': 'number'},- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ] }, ('Model', 'validation'): {'$ref': '#/$defs/Model'},@@ -7281,15 +7240,42 @@ def test_json_schema_arguments_v3_aliases() -> None: } +class DecimalPatternGenerateJsonSchema(GenerateJsonSchema):+ def get_decimal_pattern(self, schema: core_schema.DecimalSchema) -> str | None:+ return self.build_decimal_pattern(schema)++ @pytest.fixture def get_decimal_pattern():- def pattern(max_digits=None, decimal_places=None) -> str:- filed = TypeAdapter(Annotated[Decimal, Field(max_digits=max_digits, decimal_places=decimal_places)])- return filed.json_schema()['anyOf'][1]['pattern']+ def pattern(max_digits=None, decimal_places=None, mode: JsonSchemaMode = 'validation') -> str:+ ta = TypeAdapter(Annotated[Decimal, Field(max_digits=max_digits, decimal_places=decimal_places)])+ json_schema = ta.json_schema(schema_generator=DecimalPatternGenerateJsonSchema, mode=mode)+ if mode == 'validation':+ json_schema = json_schema['anyOf'][1]+ return json_schema['pattern'] return pattern +def test_decimal_pattern_not_set_by_default() -> None:+ ta = TypeAdapter(Annotated[Decimal, Field(max_digits=4, decimal_places=2)])++ assert ta.json_schema() == {'anyOf': [{'type': 'number'}, {'type': 'string'}]}+ assert ta.json_schema(mode='serialization') == {'type': 'string'}+++def test_decimal_pattern_custom() -> None:+ class CustomGenerateJsonSchema(GenerateJsonSchema):+ def get_decimal_pattern(self, schema: core_schema.DecimalSchema) -> str | None:+ return rf'^\d{{1,{schema["max_digits"]}}}$'++ ta = TypeAdapter(Annotated[Decimal, Field(max_digits=4)])++ assert ta.json_schema(schema_generator=CustomGenerateJsonSchema) == {+ 'anyOf': [{'type': 'number'}, {'type': 'string', 'pattern': r'^\d{1,4}$'}]+ }++ @pytest.mark.parametrize('valid_decimal', ['0.1', '0000.1', '11.1', '001.1', '11111111.1', '0.100000', '0.01', '0.11']) def test_decimal_pattern_with_only_decimal_places_set(valid_decimal, get_decimal_pattern) -> None: decimal_places = 2@@ -7391,6 +7377,162 @@ def test_decimal_pattern_reject_invalid_not_numerical_values_with_decimal_places assert re.fullmatch(pattern, invalid_decimal) is None +@pytest.mark.parametrize(['max_digits', 'decimal_places'], [(None, None), (3, None), (None, 2), (4, 2)])+@pytest.mark.parametrize(+ 'valid_decimal', ['1e5', '1E5', '1E+5', '1e-5', '-1.5E1', '+.5e1', '1.e5', '01.10E01', '1.234E+1', '12345E-3']+)+def test_decimal_pattern_with_exponent(max_digits, decimal_places, valid_decimal, get_decimal_pattern) -> None:+ pattern = get_decimal_pattern(max_digits, decimal_places)++ assert re.fullmatch(pattern, valid_decimal) is not None+++@pytest.mark.parametrize(['max_digits', 'decimal_places'], [(None, None), (3, None), (None, 2), (4, 2)])+@pytest.mark.parametrize('invalid_decimal', ['e5', '.e5', '1e', '1e+', '1E5.0', '1.5.E5', '1e5e5', '1e1.5', '1f5'])+def test_decimal_pattern_reject_invalid_with_exponent(+ max_digits, decimal_places, invalid_decimal, get_decimal_pattern+) -> None:+ pattern = get_decimal_pattern(max_digits, decimal_places)++ assert re.fullmatch(pattern, invalid_decimal) is None+++@pytest.mark.parametrize(+ ['max_digits', 'decimal_places', 'valid_decimals'],+ [+ (+ None,+ None,+ [+ '0',+ '-0',+ '-0.0',+ '0.00',+ '12.34',+ '100',+ '0.000001',+ '1E-7',+ '1.5E-7',+ '0E-10',+ '0E+5',+ '1E+2',+ '1.0E+3',+ '-1.5E-100',+ '1E+100',+ ],+ ),+ (3, None, ['0', '0.100', '100', '10.0', '1.23', '0.001', '0.001000', '1E+2', '0E-10', '0E+15']),+ (None, 2, ['0', '1.23', '1.230000', '123456789.12', '0.10', '1E+2', '1.5E+3', '0E-10', '1E+100']),+ (5, 2, ['0', '-0.10', '123.45', '123.450', '999', '99.9', '1E+2', '1.5E+2', '0E-10']),+ (2, 3, ['0.0', '0.01', '0.010', '-0.01', '0E-10']),+ (10, 8, ['0', '99.12345678', '99.1234567800', '0.00000001', '1E-7', '1.5E-7', '1E-8', '1E+1', '0E-20']),+ (30, None, ['1E-7', '1.2345678901234567890123E-7', '1E-30', '1E+29', '123456789012345678901234567890']),+ (4, 0, ['0', '9999', '9999.0', '1E+3', '1.5E+3', '0E-10']),+ ],+)+def test_decimal_pattern_serialization(max_digits, decimal_places, valid_decimals, get_decimal_pattern) -> None:+ pattern = get_decimal_pattern(max_digits, decimal_places, mode='serialization')+ ta = TypeAdapter(Annotated[Decimal, Field(max_digits=max_digits, decimal_places=decimal_places)])++ for valid_decimal in valid_decimals:+ serialized = json.loads(ta.dump_json(ta.validate_python(valid_decimal)))+ assert serialized == str(Decimal(valid_decimal))+ assert re.fullmatch(pattern, serialized) is not None, valid_decimal+++@pytest.mark.parametrize(+ ['max_digits', 'decimal_places', 'invalid_decimals'],+ [+ # Not produced by `str()` (or not a decimal):+ (+ None,+ None,+ [+ '',+ '.',+ '-',+ '+1',+ '01',+ '1.',+ '.1',+ '-01.5',+ '1e5',+ '1E5',+ '1E-05',+ '1E+0',+ '0E-0',+ '00E-10',+ '10E+2',+ '0.0E+2',+ 'E5',+ 'Infinity',+ 'NaN',+ ],+ ),+ # Doesn't satisfy the constraints:+ (3, None, ['1000', '1.234', '0.0001', '1E+3', '1E-4', '1.5E+3', '1.5E-7']),+ (None, 2, ['1.234', '0.001', '1E-7', '1.5E-7', '0.001E+2']),+ (5, 2, ['1234', '123.456', '1000.5', '0.001', '1E+3', '1E-7']),+ (2, 3, ['1', '0.001', '1.0', '1E+1', '1E-7']),+ (10, 8, ['100', '99.123456789', '0.000000001', '1E-9', '1.5E-8', '1.23E-7', '1E+2']),+ (4, 0, ['0.1', '10000', '1E+4', '1E-7']),+ ],+)+def test_decimal_pattern_serialization_reject_invalid(+ max_digits, decimal_places, invalid_decimals, get_decimal_pattern+) -> None:+ pattern = get_decimal_pattern(max_digits, decimal_places, mode='serialization')++ for invalid_decimal in invalid_decimals:+ assert re.fullmatch(pattern, invalid_decimal) is None, invalid_decimal+++def test_decimal_pattern_allow_inf_nan() -> None:+ class Model(BaseModel):+ model_config = ConfigDict(allow_inf_nan=True)++ a: Decimal+ b: Annotated[Decimal, AllowInfNan(False)] = Decimal(0)++ class ModelNoConfig(BaseModel):+ a: Annotated[Decimal, AllowInfNan()]++ non_finite = ['Infinity', '-Infinity', 'inf', 'INF', '+Inf', 'NaN', 'nan', '-NaN', 'sNaN', 'snan', 'NaN123']++ for model, field in [(Model, 'a'), (ModelNoConfig, 'a')]:+ validation_pattern = model.model_json_schema(schema_generator=DecimalPatternGenerateJsonSchema)['properties'][+ field+ ]['anyOf'][1]['pattern']+ serialization_pattern = model.model_json_schema(+ schema_generator=DecimalPatternGenerateJsonSchema, mode='serialization'+ )['properties'][field]['pattern']++ for value in ['1.5', *non_finite]:+ assert re.fullmatch(validation_pattern, value) is not None, value+ serialized = model.model_validate({field: value}).model_dump(mode='json')[field]+ assert re.fullmatch(serialization_pattern, serialized) is not None, serialized++ for value in ['Infinity1', 'inf.0', 'in', 'NaN1.5', 'sInf', 'i n f']:+ assert re.fullmatch(validation_pattern, value) is None, value+ for value in ['inf', 'nan', 'INFINITY', '+Infinity', 'sNaN1.5', 'sInf']:+ assert re.fullmatch(serialization_pattern, value) is None, value++ # The schema takes precedence over the config:+ for mode in ('validation', 'serialization'):+ json_schema = Model.model_json_schema(schema_generator=DecimalPatternGenerateJsonSchema, mode=mode)+ pattern = json_schema['properties']['b']+ pattern = (pattern['anyOf'][1] if mode == 'validation' else pattern)['pattern']+ for value in non_finite:+ assert re.fullmatch(pattern, value) is None, value+++def test_decimal_pattern_allow_inf_nan_disabled_by_default(get_decimal_pattern) -> None:+ for mode in ('validation', 'serialization'):+ pattern = get_decimal_pattern(mode=mode)+ for value in ['Infinity', '-Infinity', 'inf', 'NaN', 'sNaN']:+ assert re.fullmatch(pattern, value) is None, value++ def test_union_format_primitive_type_array() -> None: class Sub(BaseModel): pass
What the agent reported
The change is complete and verified. Final report below.
What was wrong
GenerateJsonSchema.decimal_schema in pydantic/json_schema.py generates a string pattern for Decimal fields. For unconstrained decimals (and decimal-places-only decimals) the pattern was ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ — with no exponent component. But pydantic-core's serializer calls str() on Decimal, which emits scientific notation for small/large values (Decimal('0.0000001') → "1E-7"). So model_dump_json() output was rejected by the model's own model_json_schema(), breaking OpenAPI/codegen validators intermittently (v2.12 regression from PR #11987).
What I changed
pydantic/json_schema.py— inget_decimal_pattern, appended an optional scientific-notation group(?:[eE][+-]?\d+)?before the terminal$in the two$-anchored cases that actually rejected such strings:- Case 3 (decimal_places only):
…0*(?:[eE][+-]?\d+)?$ - Case 4 (no constraints):
\d*\.?\d*(?:[eE][+-]?\d+)?$ - Cases 1 & 2 (max_digits) were left unchanged — their unanchored first alternative already matches scientific notation, so they never rejected it.
tests/test_json_schema.py— added regression testtest_decimal_serialization_schema_matches_scientific_notation(reproduces the exact report:Decimal('0.0000001')→"1E-7"must match the schema's own pattern), and updated all existing hardcoded pattern assertions to the corrected pattern.docs/concepts/json_schema.md— updated the two printed-output examples (validation + serialization) to the new pattern (these are executed as doc tests).
I chose the regex approach over pydantic-core normalization because the latter lives in a separate crate not editable here; correcting the schema pattern fully resolves the reported mismatch and is also correct for validation mode, since Decimal accepts scientific-notation string input.
Test evidence
- Regression test fails before the fix:
assert pattern.match('1E-7') is not None→AssertionError: assert None is not Noneagainst pattern…\d*\.?\d*$. - After the fix:
./bt-teston the regression test +test_decimal_json_schema+test_special_decimal_types→ 5 passed. - Full
tests/test_json_schema.py→ 559 passed, 3 skipped, 1 xfailed. - Docs examples
./bt-test tests/test_docs.py -k json_schema→ 25 passed (includes the changedjson_schema.md:263-309block). - Reviewer confirmed the regex is well-formed, accepts
"1E-7"/"1E+20"/"-2.5E-10", still rejects empty/"++"/"1.2.3", and nothing unrelated changed.
Requirements disposition
- Fix option A — append
(?:[eE][+-]?\d+)?to the pattern → Implemented (Cases 3 & 4). Test evidence above. - Fix option B — normalize Decimal to fixed-point during serialization → Skipped (mutually exclusive alternative). Evidence: this would require changes in pydantic-core (
str(Decimal)behavior lives in thepydantic-coreRust crate, not in this repo'spydantic/Python source), which is out of scope for apydantic/json_schema.pyfix; option A resolves the report entirely. - Reported scenario: serialized output must match its own schema → Implemented + covered by the regression test asserting the exact
model_dump_json→ pattern round-trip.
All changes are left uncommitted in the working tree. One non-blocking cosmetic nit noted by the reviewer (regression test locally re-imports json/re, already available at module level) — left as-is since it's harmless.