Backtestspydanticrun-20261003-150343#13089
Decimal JSON schema pattern rejects pydantic's own scientific notation serialization output (v2.12 regression)
The merged fix was judged better than our patch.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Both patches fix the reported failure: `model_dump_json()` output such as '1E-7' will no longer be rejected by the serialization schema. Patch B is the minimal fix the issue suggests. It adds a scientific-notation alternative to the existing pattern in both modes. That alternative ignores `max_digits`/`decimal_places` entirely, so the constrained patterns become partly meaningless. It also keeps the lookahead-based, `\d`-based pattern, whose portability to downstream engines (ECMA/Zod, OpenAPI validators) is exactly what the issue complains about. NaN/Infinity outputs under `allow_inf_nan` are still rejected. Its tests do check real round-trips of serialized scientific values against the pattern, which is good, but the coverage is narrow. Patch A goes to the root cause. The default schema no longer emits a pattern, which restores v2.11 behavior, so no downstream consumer can reject valid output. It also exposes an overridable `get_decimal_pattern` hook plus a `build_decimal_pattern` helper that produces mode-aware patterns: - In serialization mode the pattern exactly describes `str(Decimal)` output, including exponents, `0E-n`, and inf/nan, while still respecting `max_digits`/`decimal_places`. - In validation mode it accepts Decimal-constructor syntax. - It uses `[0-9]` and no lookaheads for portability. Patch A's tests are far more thorough. They round-trip many values through `dump_json` and assert pattern matches. They also cover rejection of strings that violate constraints, `allow_inf_nan` config versus field precedence, and the new default of no pattern. Patch A is a larger change and alters the default schema, but it reverts to the prior stable behavior and removes the class of bugs rather than patching one symptom. On correctness and test strength, A is the better fix.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
13m 46s
Wall-clock time in the container
Full test suite
No new failures
12,648 passed · 509 skipped · 13,157 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
88vs 573 merged
Changed lines, ours vs merged
The issue
View on GitHubInitial Checks
- I confirm that I'm using Pydantic V2
Description
model_json_schema(mode="serialization") pattern for Decimal breaks on scientific notation
The serialization schema for Decimal fields generates a regex that rejects values that model_dump_json() itself produces.
Specifically: pydantic-core's serializer calls str() on Decimal objects, which outputs scientific notation for small/large values. E.g. Decimal('0.0000001') becomes "1E-7". But the pattern added in #11987 (^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$) has no [eE] component, so it'll never match those strings.
As a result, any downstream consumer validating model_dump_json() output against model_json_schema(), such as OpenAPI validators and codegen + validation tools like Orval + Zod, will intermittently reject perfectly valid serialized values. Intermittent because only certain Decimal values trigger scientific notation.
v2.11:
{"anyOf": [{"type": "string"}, {"type": "null"}]}
v2.12:
{"anyOf": [{"pattern": "^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$", "type": "string"}, {"type": "null"}]}
Two possible fixes:
- Append
(?:[eE][+-]?\d+)?to the pattern in serialization mode - Normalize
Decimalto fixed-point during serialization before it ever hits the schema
Example Code
import json
import re
from decimal import Decimal
from pydantic import BaseModel
class MyModel(BaseModel):
value: Decimal | None
schema = MyModel.model_json_schema(mode="serialization")
pattern = next(
v for v in schema["properties"]["value"]["anyOf"] if v.get("type") == "string"
)["pattern"]
regex = re.compile(pattern)
# pydantic serializes this as "1E-7" which fails its own schema's pattern
m = MyModel(value=Decimal("0.0000001"))
serialized = json.loads(m.model_dump_json())["value"]
print(f"Pattern: {pattern}")
print(f"Serialized: {serialized!r}")
print(f"Matches: {bool(regex.match(serialized))}")
# Pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$
# Serialized: '1E-7'
# Matches: False
Python, Pydantic & OS Version
pydantic version: 2.12.5
pydantic-core version: 2.41.5
pydantic-core build: profile=release pgo=false
python version: 3.12.11 | packaged by conda-forge | (main, Jun 4 2025, 14:38:53) [Clang 18.1.8 ]
platform: macOS-26.2-arm64-arm-64bit
related packages: pyright-1.1.408 pydantic-extra-types-2.10.5 typing_extensions-4.15.0 fastapi-0.123.0 pydantic-settings-2.10.1
commit: unknownThe two patches
docs/concepts/json_schema.mdindex cd8bc01a..9cd151d9 100644@@ -278,7 +278,7 @@ print(Model.model_json_schema(mode='validation')) 'anyOf': [ {'type': 'number'}, {- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*(?:\\d*\\.?\\d*$|(?:\\d+\\.?\\d*|\\.\\d+)[eE][+-]?\\d+$)', 'type': 'string', }, ],@@ -297,7 +297,7 @@ print(Model.model_json_schema(mode='serialization')) 'properties': { 'a': { 'default': '12.34',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*(?:\\d*\\.?\\d*$|(?:\\d+\\.?\\d*|\\.\\d+)[eE][+-]?\\d+$)', 'title': 'A', 'type': 'string', }pydantic/json_schema.pyindex 8135c60b..33a7890e 100644@@ -735,7 +735,7 @@ class GenerateJsonSchema: # Case 1: Both max_digits and decimal_places are set if max_digits is not None and decimal_places is not None: integer_places = max(0, max_digits - decimal_places)- pattern += (+ fixed_point = ( rf'(?:' rf'\d{{0,{integer_places}}}' rf'|'@@ -746,7 +746,7 @@ class GenerateJsonSchema: # Case 2: Only max_digits is set elif max_digits is not None and decimal_places is None:- pattern += (+ fixed_point = ( rf'(?:' rf'\d{{0,{max_digits}}}' rf'|'@@ -757,13 +757,18 @@ class GenerateJsonSchema: # Case 3: Only decimal_places is set elif max_digits is None and decimal_places is not None:- pattern += rf'\d*\.?\d{{0,{decimal_places}}}0*$'+ fixed_point = rf'\d*\.?\d{{0,{decimal_places}}}0*$' # Case 4: Both are None (no restrictions) else:- pattern += r'\d*\.?\d*$' # look for arbitrary integer or decimal+ fixed_point = r'\d*\.?\d*$' # look for arbitrary integer or decimal - return pattern+ # Decimals are serialized using `str()`, which uses scientific notation for small/large values+ # (e.g. `Decimal('0.0000001')` -> `'1E-7'`), and such strings are also accepted during validation.+ # The digits constraints are not checked for this form, as they depend on the exponent value:+ scientific = r'(?:\d+\.?\d*|\.\d+)[eE][+-]?\d+$'++ return rf'{pattern}(?:{fixed_point}|{scientific})' json_schema = self.str_schema(core_schema.str_schema(pattern=get_decimal_pattern(schema))) if self.mode == 'validation':tests/test_json_schema.pyindex b1907b72..50ed0492 100644@@ -564,7 +564,7 @@ def test_decimal_json_schema(): {'type': 'number'}, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*(?:\\d*\\.?\\d*$|(?:\\d+\\.?\\d*|\\.\\d+)[eE][+-]?\\d+$)', }, ], 'default': '12.34',@@ -581,7 +581,7 @@ def test_decimal_json_schema(): 'default': '12.34', 'title': 'B', 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*(?:\\d*\\.?\\d*$|(?:\\d+\\.?\\d*|\\.\\d+)[eE][+-]?\\d+$)', }, }, 'title': 'Model',@@ -1172,7 +1172,7 @@ def test_special_decimal_types(field_type, expected_schema): {'type': 'number'}, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*(?:\\d*\\.?\\d*$|(?:\\d+\\.?\\d*|\\.\\d+)[eE][+-]?\\d+$)', }, ], 'title': 'A',@@ -2175,7 +2175,7 @@ def test_docstring(docstring, description): {'exclusiveMinimum': 2.0, 'type': 'number'}, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*(?:\\d*\\.?\\d*$|(?:\\d+\\.?\\d*|\\.\\d+)[eE][+-]?\\d+$)', }, ] },@@ -2188,7 +2188,7 @@ def test_docstring(docstring, description): {'type': 'number', 'exclusiveMaximum': 5}, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*(?:\\d*\\.?\\d*$|(?:\\d+\\.?\\d*|\\.\\d+)[eE][+-]?\\d+$)', }, ] },@@ -2201,7 +2201,7 @@ def test_docstring(docstring, description): {'type': 'number', 'minimum': 2}, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*(?:\\d*\\.?\\d*$|(?:\\d+\\.?\\d*|\\.\\d+)[eE][+-]?\\d+$)', }, ] },@@ -2214,7 +2214,7 @@ def test_docstring(docstring, description): {'type': 'number', 'maximum': 5}, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*(?:\\d*\\.?\\d*$|(?:\\d+\\.?\\d*|\\.\\d+)[eE][+-]?\\d+$)', }, ] },@@ -2227,7 +2227,7 @@ def test_docstring(docstring, description): {'type': 'number', 'multipleOf': 5}, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*(?:\\d*\\.?\\d*$|(?:\\d+\\.?\\d*|\\.\\d+)[eE][+-]?\\d+$)', }, ] },@@ -2276,7 +2276,7 @@ def test_constraints_schema_validation(kwargs, type_, expected_extra): Decimal, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*(?:\\d*\\.?\\d*$|(?:\\d+\\.?\\d*|\\.\\d+)[eE][+-]?\\d+$)', }, ), (@@ -2284,7 +2284,7 @@ def test_constraints_schema_validation(kwargs, type_, expected_extra): Decimal, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*(?:\\d*\\.?\\d*$|(?:\\d+\\.?\\d*|\\.\\d+)[eE][+-]?\\d+$)', }, ), (@@ -2292,7 +2292,7 @@ def test_constraints_schema_validation(kwargs, type_, expected_extra): Decimal, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*(?:\\d*\\.?\\d*$|(?:\\d+\\.?\\d*|\\.\\d+)[eE][+-]?\\d+$)', }, ), (@@ -2300,7 +2300,7 @@ def test_constraints_schema_validation(kwargs, type_, expected_extra): Decimal, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*(?:\\d*\\.?\\d*$|(?:\\d+\\.?\\d*|\\.\\d+)[eE][+-]?\\d+$)', }, ), (@@ -2308,7 +2308,7 @@ def test_constraints_schema_validation(kwargs, type_, expected_extra): Decimal, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*(?:\\d*\\.?\\d*$|(?:\\d+\\.?\\d*|\\.\\d+)[eE][+-]?\\d+$)', }, ), ],@@ -6185,14 +6185,14 @@ def test_generate_definitions_for_no_ref_schemas(): { ('Decimal', 'serialization'): { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*(?:\\d*\\.?\\d*$|(?:\\d+\\.?\\d*|\\.\\d+)[eE][+-]?\\d+$)', }, ('Decimal', 'validation'): { 'anyOf': [ {'type': 'number'}, { 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',+ 'pattern': '^(?!^[-+.]*$)[+-]?0*(?:\\d*\\.?\\d*$|(?:\\d+\\.?\\d*|\\.\\d+)[eE][+-]?\\d+$)', }, ] },@@ -7391,6 +7391,45 @@ def test_decimal_pattern_reject_invalid_not_numerical_values_with_decimal_places assert re.fullmatch(pattern, invalid_decimal) is None +@pytest.mark.parametrize('mode', ['validation', 'serialization'])+@pytest.mark.parametrize(+ 'constraints,value',+ [+ ({}, Decimal('0.0000001')),+ ({}, Decimal('-1E+3')),+ ({}, Decimal('1.5E+10')),+ ({}, Decimal('0E-8')),+ ({'decimal_places': 8}, Decimal('0.0000001')),+ ({'decimal_places': 0}, Decimal('1.5E+3')),+ ({'max_digits': 8}, Decimal('0.0000001')),+ ({'max_digits': 4, 'decimal_places': 0}, Decimal('1.5E+3')),+ ({'max_digits': 8, 'decimal_places': 7}, Decimal('-0.0000001')),+ ],+)+def test_decimal_pattern_matches_scientific_notation_serialization(mode, constraints, value) -> None:+ """The pattern should match the scientific notation produced when serializing decimals."""+ ta = TypeAdapter(Annotated[Decimal, Field(**constraints)])+ json_schema = ta.json_schema(mode=mode)+ pattern = json_schema['pattern'] if mode == 'serialization' else json_schema['anyOf'][1]['pattern']++ serialized = json.loads(ta.dump_json(ta.validate_python(value)))+ assert 'E' in serialized+ assert re.fullmatch(pattern, serialized) is not None+ assert ta.validate_json(json.dumps(serialized)) == value+++@pytest.mark.parametrize('valid_decimal', ['1E-7', '1e-7', '-1.5E+3', '+1.E3', '.5e2', '0E-8', '001E10'])+def test_decimal_pattern_with_scientific_notation(valid_decimal, get_decimal_pattern) -> None:+ pattern = get_decimal_pattern()+ assert re.fullmatch(pattern, valid_decimal) is not None+++@pytest.mark.parametrize('invalid_decimal', ['E5', '-E5', '.E5', '1E', '1E+', '1E-7.5', '1EE7', '1E-7x', 'e'])+def test_decimal_pattern_reject_invalid_scientific_notation(invalid_decimal, get_decimal_pattern) -> None:+ pattern = get_decimal_pattern()+ assert re.fullmatch(pattern, invalid_decimal) is None++ def test_union_format_primitive_type_array() -> None: class Sub(BaseModel): pass
docs/api/standard_library_types.mdindex 4070fda59..5802bb2e3 100644@@ -300,11 +300,48 @@ Decimals support the following constraints (numbers must be coercible to decimal | `gt` | The value must be strictly greater than this number | [`exclusiveMinimum`](https://json-schema.org/understanding-json-schema/reference/numeric#range) keyword | | `multiple_of` | The value must be a multiple of this number | [`multipleOf`](https://json-schema.org/understanding-json-schema/reference/numeric#multiples) keyword | | `allow_inf_nan` | Whether to allow NaN (not-a-number) and infinite values | N/A |-| `max_digits` | The maximum number of decimal digits allowed. The zero before the decimal point and trailing zeros are not counted. | [`pattern`](https://json-schema.org/understanding-json-schema/reference/string#regexp) keyword, to describe the string pattern |-| `decimal_places` | The maximum number of decimal places allowed. Trailing zeros are not counted. | [`pattern`](https://json-schema.org/understanding-json-schema/reference/string#regexp) keyword, to describe the string pattern |+| `max_digits` | The maximum number of decimal digits allowed. The zero before the decimal point and trailing zeros are not counted. | N/A (see below) |+| `decimal_places` | The maximum number of decimal places allowed. Trailing zeros are not counted. | N/A (see below) | -Note that the JSON Schema [`pattern`](https://json-schema.org/understanding-json-schema/reference/string#regexp) keyword will be specified-in the JSON Schema to describe the string pattern in all cases (and can vary if `max_digits` and/or `decimal_places` is specified).+/// version-changed | v2.12+The JSON Schema includes a [`pattern`](https://json-schema.org/understanding-json-schema/reference/string#regexp) matching the `max_digits`+and `decimal_places` constraints.+///++/// version-changed | v2.14+The JSON Schema no longer includes a [`pattern`](https://json-schema.org/understanding-json-schema/reference/string#regexp) by default,+as the generated pattern can cause issues for downstream consumers due to its complexity.++The pattern can be included by subclassing [`GenerateJsonSchema`][pydantic.json_schema.GenerateJsonSchema] and overriding the+[`get_decimal_pattern()`][pydantic.json_schema.GenerateJsonSchema.get_decimal_pattern] method:++```python+from decimal import Decimal+from typing import Annotated++from pydantic import Field, TypeAdapter+from pydantic.json_schema import GenerateJsonSchema+++class MyGenerateJsonSchema(GenerateJsonSchema):+ def get_decimal_pattern(self, schema):+ return self.build_decimal_pattern(schema)+++ta = TypeAdapter(Annotated[Decimal, Field(max_digits=5, decimal_places=2)])+print(+ ta.json_schema(schema_generator=MyGenerateJsonSchema, mode='serialization')+)+"""+{+ 'pattern': '^-?(?:(?:0|[1-9][0-9]{0,2})(?:\\.[0-9]{1,2}0*)?|0E[+-][1-9][0-9]*|[1-9](?:\\.[0-9]+)?E\\+[1-2])$',+ 'type': 'string',+}+"""+```++The generated pattern also no longer uses lookahead assertions, and takes exponents and `allow_inf_nan` into account.+/// These constraints can be provided using the [`Field()`][pydantic.Field] function. The `Le`, `Ge`, `Lt`, `Gt` and `MultipleOf` metadata types from the [`annotated-types`](https://github.com/annotated-types/annotated-types)docs/concepts/json_schema.mdindex cd8bc01a8..e0a7bdf4c 100644@@ -275,13 +275,7 @@ print(Model.model_json_schema(mode='validation')) { 'properties': { 'a': {- 'anyOf': [- {'type': 'number'},- {- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- 'type': 'string',- },- ],+ 'anyOf': [{'type': 'number'}, {'type': 'string'}], 'default': '12.34', 'title': 'A', }@@ -294,14 +288,7 @@ print(Model.model_json_schema(mode='validation')) print(Model.model_json_schema(mode='serialization')) """ {- 'properties': {- 'a': {- 'default': '12.34',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- 'title': 'A',- 'type': 'string',- }- },+ 'properties': {'a': {'default': '12.34', 'title': 'A', 'type': 'string'}}, 'title': 'Model', 'type': 'object', }pydantic/json_schema.pyindex 8135c60b5..5c585426d 100644@@ -723,49 +723,11 @@ class GenerateJsonSchema: Returns: The generated JSON schema. """-- def get_decimal_pattern(schema: core_schema.DecimalSchema) -> str:- max_digits = schema.get('max_digits')- decimal_places = schema.get('decimal_places')-- pattern = (- r'^(?!^[-+.]*$)[+-]?0*' # check it is not empty string and not one or sequence of ".+-" characters.- )-- # Case 1: Both max_digits and decimal_places are set- if max_digits is not None and decimal_places is not None:- integer_places = max(0, max_digits - decimal_places)- pattern += (- rf'(?:'- rf'\d{{0,{integer_places}}}'- rf'|'- rf'(?=[\d.]{{1,{max_digits + 1}}}0*$)'- rf'\d{{0,{integer_places}}}\.\d{{0,{decimal_places}}}0*$'- rf')'- )-- # Case 2: Only max_digits is set- elif max_digits is not None and decimal_places is None:- pattern += (- rf'(?:'- rf'\d{{0,{max_digits}}}'- rf'|'- rf'(?=[\d.]{{1,{max_digits + 1}}}0*$)'- rf'\d*\.\d*0*$'- rf')'- )-- # Case 3: Only decimal_places is set- elif max_digits is None and decimal_places is not None:- pattern += rf'\d*\.?\d{{0,{decimal_places}}}0*$'-- # Case 4: Both are None (no restrictions)- else:- pattern += r'\d*\.?\d*$' # look for arbitrary integer or decimal-- return pattern-- json_schema = self.str_schema(core_schema.str_schema(pattern=get_decimal_pattern(schema)))+ str_schema = core_schema.str_schema()+ pattern = self.get_decimal_pattern(schema)+ if pattern is not None:+ str_schema['pattern'] = pattern+ json_schema = self.str_schema(str_schema) if self.mode == 'validation': multiple_of = schema.get('multiple_of') le = schema.get('le')@@ -789,6 +751,63 @@ class GenerateJsonSchema: } return json_schema + def get_decimal_pattern(self, schema: core_schema.DecimalSchema) -> str | None:+ """Get the regular expression pattern to apply to the string representation of a decimal.++ By default, no pattern is applied (`None` is returned). Subclasses can override this method+ to return a custom pattern, or [`build_decimal_pattern()`][pydantic.json_schema.GenerateJsonSchema.build_decimal_pattern]+ can be used to get a pattern reflecting the `max_digits` and `decimal_places` constraints:++ ```python+ from pydantic.json_schema import GenerateJsonSchema++ class MyGenerateJsonSchema(GenerateJsonSchema):+ def get_decimal_pattern(self, schema):+ return self.build_decimal_pattern(schema)+ ```++ Args:+ schema: The core schema.++ Returns:+ The regular expression pattern, or `None` if no pattern should be applied.+ """+ return None++ def build_decimal_pattern(self, schema: core_schema.DecimalSchema) -> str:+ """Build a regular expression pattern for the string representation of a decimal.++ The pattern takes into account the `max_digits` and `decimal_places` constraints, and the current+ mode:++ - In `'validation'` mode, the pattern accepts the string syntax understood by the [`Decimal`][decimal.Decimal]+ constructor. The `max_digits` and `decimal_places` constraints are only reflected for strings+ *without* an exponent: strings using an exponent are accepted as long as they are syntactically valid,+ as the constraints can't be expressed with a regular expression in this case.+ - In `'serialization'` mode, the pattern exactly describes the strings produced by pydantic+ (i.e. by `str(decimal)`) for decimals satisfying the constraints,+ including the ones using scientific notation (e.g. `'1E-7'`, `'1.5E+3'`).++ In both modes, trailing zeros in the decimal part are not counted (e.g. `'1.10'` is accepted with+ `decimal_places=1`), consistent with the validation behavior. If `allow_inf_nan` is enabled (on the+ core schema or in the configuration), the string representations of infinity and NaN are also accepted.++ Args:+ schema: The core schema.++ Returns:+ The regular expression pattern.+ """+ max_digits = schema.get('max_digits')+ decimal_places = schema.get('decimal_places')+ allow_inf_nan = schema.get('allow_inf_nan')+ if allow_inf_nan is None:+ allow_inf_nan = self._config.config_dict.get('allow_inf_nan', False)+ if self.mode == 'validation':+ return _decimal_validation_pattern(max_digits, decimal_places, allow_inf_nan)+ else:+ return _decimal_serialization_pattern(max_digits, decimal_places, allow_inf_nan)+ def str_schema(self, schema: core_schema.StringSchema) -> JsonSchemaValue: """Generates a JSON schema that matches a string value. @@ -2773,15 +2792,153 @@ def models_json_schema( # ##### End JSON Schema Generation Functions ##### -_HashableJsonValue: TypeAlias = (- int | float | str | bool | None | tuple['_HashableJsonValue', ...] | tuple[tuple[str, '_HashableJsonValue'], ...]-)+# Decimal JSON Schema pattern utilities.+# Note: while we could use `\d` to represent digits (`Decimal` objects allow Unicode decimal digits), we use [0-9] instead:+# - to reduce complexity (`\d` expands to a large set, and with `max_digits`/`decimal_places` constraints set, it can blow+# the compile limit on some engines).+# - [0-9] is a portable pattern (e.g. `\d` is equivalent to `[0-9]` in ECMA (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Regular_expressions/Character_class_escape#d)+# while it allows any Unicode decimal digit in Python.+++def _repeat(atom: str, min_count: int, max_count: int) -> str:+ """Build a regex repetition of `atom` between `min_count` and `max_count` times (inclusive), using the shortest quantifier."""+ if min_count > max_count or max_count == 0:+ return ''+ if min_count == max_count:+ return atom if min_count == 1 else f'{atom}{{{min_count}}}'+ if (min_count, max_count) == (0, 1):+ return f'{atom}?'+ return f'{atom}{{{min_count},{max_count}}}'+++def _int_range_pattern(max_value: int) -> str:+ """Build a regex matching the integers between 1 and `max_value` (inclusive), without leading zeros."""+ digits = str(max_value)+ length = len(digits)+ alts: list[str] = [f'[1-9]{_repeat("[0-9]", k - 1, k - 1)}' for k in range(1, length)]+ for i, char in enumerate(digits):+ d = int(char)+ low = 1 if i == 0 else 0+ high = d if i == length - 1 else d - 1+ if high >= low:+ char_class = str(low) if low == high else f'[{low}-{high}]'+ alts.append(f'{digits[:i]}{char_class}{_repeat("[0-9]", length - i - 1, length - i - 1)}')+ return alts[0] if len(alts) == 1 else f'(?:{"|".join(alts)})'+++def _decimal_validation_pattern(max_digits: int | None, decimal_places: int | None, allow_inf_nan: bool) -> str:+ """Build the decimal pattern for the `'validation'` mode."""++ number = r'(?:[0-9]+\.?[0-9]*|\.[0-9]+)' # e.g. 1, 1., .5+ exponent = r'[eE][+-]?[0-9]+' # e.g. e5, E5, e-7+ # e.g. Infinity, inf, NaN, sNaN, NaN123 (case-insensitive). Spelled out explicitly, as inline flags+ # aren't supported by all regular expression engines:+ non_finite = r'|[iI][nN][fF](?:[iI][nN][iI][tT][yY])?|[sS]?[nN][aA][nN][0-9]*' if allow_inf_nan else ''+ if max_digits is None and decimal_places is None:+ return rf'^[+-]?(?:{number}(?:{exponent})?{non_finite})$'++ alts: list[str] = []+ if max_digits is not None and decimal_places is not None:+ # If `decimal_places > max_digits`, no whole digits are allowed and `max_digits` bounds the decimal places:+ whole = max(0, max_digits - decimal_places)+ places = min(max_digits, decimal_places)+ int_part = f'0*{_repeat("[0-9]", 1, whole)}' if whole else '0+'+ opt_int_part = f'0*{_repeat("[0-9]", 0, whole)}'+ opt_frac_part = rf'\.{_repeat("[0-9]", 0, places)}0*'+ frac_part = rf'\.{_repeat("[0-9]", 1, places)}0*' if places else r'\.0+'+ alts += [f'{int_part}(?:{opt_frac_part})?', f'{opt_int_part}{frac_part}']+ elif max_digits is not None:+ # The whole part and decimal part must have at most `max_digits` digits in total,+ # so we enumerate the possible widths of the whole part:+ int_part = f'0*{_repeat("[0-9]", 1, max_digits)}' if max_digits else '0+'+ alts.append(rf'{int_part}(?:\.0*)?')+ alts += [+ rf'0*{_repeat("[0-9]", whole, whole)}\.{_repeat("[0-9]", 1, max_digits - whole)}0*'+ for whole in range(max_digits)+ ] or [r'0*\.0+']+ else:+ assert decimal_places is not None+ opt_frac_part = rf'\.{_repeat("[0-9]", 0, decimal_places)}0*'+ frac_part = rf'\.{_repeat("[0-9]", 1, decimal_places)}0*' if decimal_places else r'\.0+'+ alts += [f'[0-9]+(?:{opt_frac_part})?', f'[0-9]*{frac_part}']++ alts.append(f'{number}{exponent}')+ return rf'^[+-]?(?:{"|".join(alts)}{non_finite})$'+++def _decimal_serialization_pattern(max_digits: int | None, decimal_places: int | None, allow_inf_nan: bool) -> str:+ """Build the decimal pattern for the `'serialization'` mode.++ Decimals are serialized using `str()`, producing either:++ - The plain notation (`'123.45'`), when the exponent is lower than or equal to 0 and the adjusted+ exponent is greater than or equal to -6.+ - The scientific notation otherwise (`'1E+3'`, `'1.5E-7'`, `'0E-10'`), with a single digit before+ the optional decimal point and an always signed exponent.++ Trailing zeros of the decimal part are preserved by `str()`, but aren't counted by the validation logic.+ """++ def frac_part(places: int) -> str:+ return rf'\.{_repeat("[0-9]", 1, places)}0*' if places else r'\.0+'++ alts: list[str] = []++ # Plain notation:+ if max_digits is not None and decimal_places is not None:+ whole = max(0, max_digits - decimal_places)+ places = min(max_digits, decimal_places)+ int_part = f'(?:0|[1-9]{_repeat("[0-9]", 0, whole - 1)})' if whole else '0'+ alts.append(f'{int_part}(?:{frac_part(places)})?')+ elif max_digits is not None:+ # The whole part and decimal part must have at most `max_digits` digits in total,+ # so we enumerate the possible widths of the whole part:+ alts.append(rf'0(?:\.{_repeat("[0-9]", 1, max_digits)}0*)?')+ alts += [+ f'[1-9]{_repeat("[0-9]", whole - 1, whole - 1)}(?:{frac_part(max_digits - whole)})?'+ for whole in range(1, max_digits + 1)+ ]+ elif decimal_places is not None:+ alts.append(f'(?:0|[1-9][0-9]*)(?:{frac_part(decimal_places)})?')+ else:+ alts.append(r'(?:0|[1-9][0-9]*)(?:\.[0-9]+)?')++ # Zero with an exponent (e.g. `'0E-10'`), which normalizes to zero and thus satisfies any constraint:+ alts.append(r'0E[+-][1-9][0-9]*')++ # Scientific notation with a positive exponent `k` (e.g. `'1.5E+3'`): the value has `k + 1` whole digits+ # and no decimal places:+ if max_digits is None:+ alts.append(r'[1-9](?:\.[0-9]+)?E\+[1-9][0-9]*')+ else:+ max_exponent = max_digits - (decimal_places or 0) - 1+ if max_exponent >= 1:+ alts.append(rf'[1-9](?:\.[0-9]+)?E\+{_int_range_pattern(max_exponent)}')++ # Scientific notation with a negative exponent `k` (`k >= 7`, e.g. `'1.5E-7'`): the value has no whole digits,+ # and `k` + (the number of significant decimal digits of the coefficient) decimal places:+ if max_digits is None and decimal_places is None:+ alts.append(r'[1-9](?:\.[0-9]+)?E-[1-9][0-9]*')+ else:+ max_places = min(c for c in (max_digits, decimal_places) if c is not None)+ alts += [f'[1-9](?:{frac_part(max_places - k)})?E-{k}' for k in range(7, max_places + 1)]++ if allow_inf_nan:+ # `str()` produces `'Infinity'`, `'NaN'` and `'sNaN'` (with an optional payload):+ alts += ['Infinity', 's?NaN[0-9]*']++ return rf'^-?(?:{"|".join(alts)})$' def _deduplicate_schemas(schemas: Iterable[JsonDict]) -> list[JsonDict]: return list({_make_json_hashable(schema): schema for schema in schemas}.values()) +_HashableJsonValue: TypeAlias = (+ int | float | str | bool | None | tuple['_HashableJsonValue', ...] | tuple[tuple[str, '_HashableJsonValue'], ...]+)++ def _make_json_hashable(value: JsonValue) -> _HashableJsonValue: if isinstance(value, dict): return tuple(sorted((k, _make_json_hashable(v)) for k, v in value.items()))tests/test_json_schema.pyindex b1907b720..bb38003f8 100644@@ -36,6 +36,7 @@ from typing_extensions import TypeAliasType, TypedDict, deprecated import pydantic from pydantic import ( AfterValidator,+ AllowInfNan, BaseModel, BeforeValidator, Field,@@ -64,6 +65,7 @@ from pydantic.json_schema import ( DEFAULT_REF_TEMPLATE, Examples, GenerateJsonSchema,+ JsonSchemaMode, JsonSchemaValue, NoDefault, PydanticJsonSchemaWarning,@@ -562,10 +564,7 @@ def test_decimal_json_schema(): 'b': { 'anyOf': [ {'type': 'number'},- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ], 'default': '12.34', 'title': 'B',@@ -581,7 +580,6 @@ def test_decimal_json_schema(): 'default': '12.34', 'title': 'B', 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$', }, }, 'title': 'Model',@@ -1170,10 +1168,7 @@ def test_special_decimal_types(field_type, expected_schema): 'a': { 'anyOf': [ {'type': 'number'},- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ], 'title': 'A', }@@ -2173,10 +2168,7 @@ def test_docstring(docstring, description): { 'anyOf': [ {'exclusiveMinimum': 2.0, 'type': 'number'},- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ] }, ),@@ -2186,10 +2178,7 @@ def test_docstring(docstring, description): { 'anyOf': [ {'type': 'number', 'exclusiveMaximum': 5},- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ] }, ),@@ -2199,10 +2188,7 @@ def test_docstring(docstring, description): { 'anyOf': [ {'type': 'number', 'minimum': 2},- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ] }, ),@@ -2212,10 +2198,7 @@ def test_docstring(docstring, description): { 'anyOf': [ {'type': 'number', 'maximum': 5},- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ] }, ),@@ -2225,10 +2208,7 @@ def test_docstring(docstring, description): { 'anyOf': [ {'type': 'number', 'multipleOf': 5},- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ] }, ),@@ -2274,42 +2254,27 @@ def test_constraints_schema_validation(kwargs, type_, expected_extra): ( {'gt': 2}, Decimal,- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ), ( {'lt': 5}, Decimal,- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ), ( {'ge': 2}, Decimal,- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ), ( {'le': 5}, Decimal,- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ), ( {'multiple_of': 5}, Decimal,- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ), ], )@@ -6183,17 +6148,11 @@ def test_generate_definitions_for_no_ref_schemas(): ) assert result == ( {- ('Decimal', 'serialization'): {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ ('Decimal', 'serialization'): {'type': 'string'}, ('Decimal', 'validation'): { 'anyOf': [ {'type': 'number'},- {- 'type': 'string',- 'pattern': '^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$',- },+ {'type': 'string'}, ] }, ('Model', 'validation'): {'$ref': '#/$defs/Model'},@@ -7281,15 +7240,42 @@ def test_json_schema_arguments_v3_aliases() -> None: } +class DecimalPatternGenerateJsonSchema(GenerateJsonSchema):+ def get_decimal_pattern(self, schema: core_schema.DecimalSchema) -> str | None:+ return self.build_decimal_pattern(schema)++ @pytest.fixture def get_decimal_pattern():- def pattern(max_digits=None, decimal_places=None) -> str:- filed = TypeAdapter(Annotated[Decimal, Field(max_digits=max_digits, decimal_places=decimal_places)])- return filed.json_schema()['anyOf'][1]['pattern']+ def pattern(max_digits=None, decimal_places=None, mode: JsonSchemaMode = 'validation') -> str:+ ta = TypeAdapter(Annotated[Decimal, Field(max_digits=max_digits, decimal_places=decimal_places)])+ json_schema = ta.json_schema(schema_generator=DecimalPatternGenerateJsonSchema, mode=mode)+ if mode == 'validation':+ json_schema = json_schema['anyOf'][1]+ return json_schema['pattern'] return pattern +def test_decimal_pattern_not_set_by_default() -> None:+ ta = TypeAdapter(Annotated[Decimal, Field(max_digits=4, decimal_places=2)])++ assert ta.json_schema() == {'anyOf': [{'type': 'number'}, {'type': 'string'}]}+ assert ta.json_schema(mode='serialization') == {'type': 'string'}+++def test_decimal_pattern_custom() -> None:+ class CustomGenerateJsonSchema(GenerateJsonSchema):+ def get_decimal_pattern(self, schema: core_schema.DecimalSchema) -> str | None:+ return rf'^\d{{1,{schema["max_digits"]}}}$'++ ta = TypeAdapter(Annotated[Decimal, Field(max_digits=4)])++ assert ta.json_schema(schema_generator=CustomGenerateJsonSchema) == {+ 'anyOf': [{'type': 'number'}, {'type': 'string', 'pattern': r'^\d{1,4}$'}]+ }++ @pytest.mark.parametrize('valid_decimal', ['0.1', '0000.1', '11.1', '001.1', '11111111.1', '0.100000', '0.01', '0.11']) def test_decimal_pattern_with_only_decimal_places_set(valid_decimal, get_decimal_pattern) -> None: decimal_places = 2@@ -7391,6 +7377,162 @@ def test_decimal_pattern_reject_invalid_not_numerical_values_with_decimal_places assert re.fullmatch(pattern, invalid_decimal) is None +@pytest.mark.parametrize(['max_digits', 'decimal_places'], [(None, None), (3, None), (None, 2), (4, 2)])+@pytest.mark.parametrize(+ 'valid_decimal', ['1e5', '1E5', '1E+5', '1e-5', '-1.5E1', '+.5e1', '1.e5', '01.10E01', '1.234E+1', '12345E-3']+)+def test_decimal_pattern_with_exponent(max_digits, decimal_places, valid_decimal, get_decimal_pattern) -> None:+ pattern = get_decimal_pattern(max_digits, decimal_places)++ assert re.fullmatch(pattern, valid_decimal) is not None+++@pytest.mark.parametrize(['max_digits', 'decimal_places'], [(None, None), (3, None), (None, 2), (4, 2)])+@pytest.mark.parametrize('invalid_decimal', ['e5', '.e5', '1e', '1e+', '1E5.0', '1.5.E5', '1e5e5', '1e1.5', '1f5'])+def test_decimal_pattern_reject_invalid_with_exponent(+ max_digits, decimal_places, invalid_decimal, get_decimal_pattern+) -> None:+ pattern = get_decimal_pattern(max_digits, decimal_places)++ assert re.fullmatch(pattern, invalid_decimal) is None+++@pytest.mark.parametrize(+ ['max_digits', 'decimal_places', 'valid_decimals'],+ [+ (+ None,+ None,+ [+ '0',+ '-0',+ '-0.0',+ '0.00',+ '12.34',+ '100',+ '0.000001',+ '1E-7',+ '1.5E-7',+ '0E-10',+ '0E+5',+ '1E+2',+ '1.0E+3',+ '-1.5E-100',+ '1E+100',+ ],+ ),+ (3, None, ['0', '0.100', '100', '10.0', '1.23', '0.001', '0.001000', '1E+2', '0E-10', '0E+15']),+ (None, 2, ['0', '1.23', '1.230000', '123456789.12', '0.10', '1E+2', '1.5E+3', '0E-10', '1E+100']),+ (5, 2, ['0', '-0.10', '123.45', '123.450', '999', '99.9', '1E+2', '1.5E+2', '0E-10']),+ (2, 3, ['0.0', '0.01', '0.010', '-0.01', '0E-10']),+ (10, 8, ['0', '99.12345678', '99.1234567800', '0.00000001', '1E-7', '1.5E-7', '1E-8', '1E+1', '0E-20']),+ (30, None, ['1E-7', '1.2345678901234567890123E-7', '1E-30', '1E+29', '123456789012345678901234567890']),+ (4, 0, ['0', '9999', '9999.0', '1E+3', '1.5E+3', '0E-10']),+ ],+)+def test_decimal_pattern_serialization(max_digits, decimal_places, valid_decimals, get_decimal_pattern) -> None:+ pattern = get_decimal_pattern(max_digits, decimal_places, mode='serialization')+ ta = TypeAdapter(Annotated[Decimal, Field(max_digits=max_digits, decimal_places=decimal_places)])++ for valid_decimal in valid_decimals:+ serialized = json.loads(ta.dump_json(ta.validate_python(valid_decimal)))+ assert serialized == str(Decimal(valid_decimal))+ assert re.fullmatch(pattern, serialized) is not None, valid_decimal+++@pytest.mark.parametrize(+ ['max_digits', 'decimal_places', 'invalid_decimals'],+ [+ # Not produced by `str()` (or not a decimal):+ (+ None,+ None,+ [+ '',+ '.',+ '-',+ '+1',+ '01',+ '1.',+ '.1',+ '-01.5',+ '1e5',+ '1E5',+ '1E-05',+ '1E+0',+ '0E-0',+ '00E-10',+ '10E+2',+ '0.0E+2',+ 'E5',+ 'Infinity',+ 'NaN',+ ],+ ),+ # Doesn't satisfy the constraints:+ (3, None, ['1000', '1.234', '0.0001', '1E+3', '1E-4', '1.5E+3', '1.5E-7']),+ (None, 2, ['1.234', '0.001', '1E-7', '1.5E-7', '0.001E+2']),+ (5, 2, ['1234', '123.456', '1000.5', '0.001', '1E+3', '1E-7']),+ (2, 3, ['1', '0.001', '1.0', '1E+1', '1E-7']),+ (10, 8, ['100', '99.123456789', '0.000000001', '1E-9', '1.5E-8', '1.23E-7', '1E+2']),+ (4, 0, ['0.1', '10000', '1E+4', '1E-7']),+ ],+)+def test_decimal_pattern_serialization_reject_invalid(+ max_digits, decimal_places, invalid_decimals, get_decimal_pattern+) -> None:+ pattern = get_decimal_pattern(max_digits, decimal_places, mode='serialization')++ for invalid_decimal in invalid_decimals:+ assert re.fullmatch(pattern, invalid_decimal) is None, invalid_decimal+++def test_decimal_pattern_allow_inf_nan() -> None:+ class Model(BaseModel):+ model_config = ConfigDict(allow_inf_nan=True)++ a: Decimal+ b: Annotated[Decimal, AllowInfNan(False)] = Decimal(0)++ class ModelNoConfig(BaseModel):+ a: Annotated[Decimal, AllowInfNan()]++ non_finite = ['Infinity', '-Infinity', 'inf', 'INF', '+Inf', 'NaN', 'nan', '-NaN', 'sNaN', 'snan', 'NaN123']++ for model, field in [(Model, 'a'), (ModelNoConfig, 'a')]:+ validation_pattern = model.model_json_schema(schema_generator=DecimalPatternGenerateJsonSchema)['properties'][+ field+ ]['anyOf'][1]['pattern']+ serialization_pattern = model.model_json_schema(+ schema_generator=DecimalPatternGenerateJsonSchema, mode='serialization'+ )['properties'][field]['pattern']++ for value in ['1.5', *non_finite]:+ assert re.fullmatch(validation_pattern, value) is not None, value+ serialized = model.model_validate({field: value}).model_dump(mode='json')[field]+ assert re.fullmatch(serialization_pattern, serialized) is not None, serialized++ for value in ['Infinity1', 'inf.0', 'in', 'NaN1.5', 'sInf', 'i n f']:+ assert re.fullmatch(validation_pattern, value) is None, value+ for value in ['inf', 'nan', 'INFINITY', '+Infinity', 'sNaN1.5', 'sInf']:+ assert re.fullmatch(serialization_pattern, value) is None, value++ # The schema takes precedence over the config:+ for mode in ('validation', 'serialization'):+ json_schema = Model.model_json_schema(schema_generator=DecimalPatternGenerateJsonSchema, mode=mode)+ pattern = json_schema['properties']['b']+ pattern = (pattern['anyOf'][1] if mode == 'validation' else pattern)['pattern']+ for value in non_finite:+ assert re.fullmatch(pattern, value) is None, value+++def test_decimal_pattern_allow_inf_nan_disabled_by_default(get_decimal_pattern) -> None:+ for mode in ('validation', 'serialization'):+ pattern = get_decimal_pattern(mode=mode)+ for value in ['Infinity', '-Infinity', 'inf', 'NaN', 'sNaN']:+ assert re.fullmatch(pattern, value) is None, value++ def test_union_format_primitive_type_array() -> None: class Sub(BaseModel): pass