Backtestspytestrun-20261106-234554#14953

Issue #14953 Merged fix PR #14954 Run 6 Nov 2026 · run-20261106-234554 pytest-dev/pytest

max_warnings rejects the integer shown in the native TOML documentation

Better

Our patch was judged better than the merged fix.

In this judgment, Patch A = our agent, Patch B = the merged fix.

Both patches fix the root cause by registering `max_warnings` as a str/int union and rewriting `_get_max_warnings` so a native TOML `0` is not treated as unset; the independent tests confirm both pass every scenario, including zero, pyproject, unset, INI, `-o`, and CLI precedence.

Correctness: Patch A keeps the empty-string default and checks `ini_value == ""`, which preserves the prior behaviour for an empty INI value. Patch B changes the default to `None` and does `int()` on any string, so an empty `max_warnings =` line in pytest.ini (previously treated as unset via the truthiness check) would now raise ValueError. Minor, but A is slightly more careful at the edge the issue explicitly calls out (distinguishing 0 from unset).

Tests: A adds a strong suite: zero and nonzero, quoted and unquoted, in both pytest.toml and pyproject.toml, a not-exceeded case, CLI-overrides-TOML, `-o max_warnings=0`, and a getini type check. B adds a single parametrized test with value 1 only, never exercising the zero regression case the issue highlights as essential.

Scope: A goes beyond the ask by also changing `faulthandler_timeout` to `str | float` and editing an unrelated `minversion` doc example. That is extra risk (e.g. whether a native TOML int satisfies the float member cannot be verified here) and counts against A, but it is isolated and does not obscure the max_warnings fix. B stays tightly scoped and updates the confval type and help text in the reference docs, which A omits; B also adds an AUTHORS entry.

On balance, A's more rigorous handling of the unset/zero distinction and much stronger regression coverage outweigh its scope creep and missing doc type update.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

4m 45s

Wall-clock time in the container

Full test suite

No new failures

542 passed · 10 skipped · 552 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

132vs 33 merged

Changed lines, ours vs merged

Both the 0 configuration reference and the warnings how-to show an unquoted native TOML integer. The configuration reference declares the option as int:

[pytest]
max_warnings = 10

However, max_warnings is registered as a string configuration option, so pytest rejects this documented representation.

Minimal reproduction

pytest.toml:

[pytest]
max_warnings = 0

test_warning.py:

import warnings


def test_warning():
    warnings.warn(UserWarning("example warning"))

Run:

pytest -qq --disable-warnings

Actual behavior

Pytest 9.1.1 runs the test, then emits a traceback ending with:

TypeError: .../pytest.toml: config option 'max_warnings' expects a string, got int: 0

The process exits with code 1.

At the pinned revision, following #14791, values that fail Config.getini()’s registered-type validation are reported as configuration errors:

ERROR: .../pytest.toml: config option 'max_warnings' expects a string, got int: 0

The pinned revision exits with code 4. For this case, #14791 changes how the invalid value is reported, but max_warnings remains string-typed.

The same rejection occurs with an unquoted integer under [tool.pytest] in pyproject.toml.

Expected behavior

Based on the documented type and examples, the integer should be accepted and used as the warning threshold. Here, one warning exceeds the explicit threshold of zero, so pytest would exit with MAX_WARNINGS_ERROR (code 6).

Quoting the value is a working workaround:

[pytest]
max_warnings = "0"

This produces:

Tests pass, but maximum allowed warnings exceeded: 1 > 0

and exits with code 6.

The issue is not specific to zero: all unquoted native TOML integers are rejected. Zero provides a useful regression case because a fix must also distinguish the valid integer 0 from the empty value representing an unset option.

<details>
<summary><strong>Verified implementation and compatibility notes</strong></summary>

max_warnings is currently registered without an explicit type or default, giving it the implicit string type and empty-string default.

If native TOML integers should be supported, one tested direction is:

parser.addini(
    "max_warnings",
    help=...,
    type=str | int,
    default="",
)

Pytest tries union members in order. With str | int:

  • native TOML integers remain integers;
  • quoted TOML values remain strings;
  • values from pytest.ini and -o remain strings as they are today;
  • the unset value remains "".

_get_max_warnings() would also need to distinguish integer zero from the empty default while preserving --max-warnings precedence:

 def _get_max_warnings(self) -> int | None:
     """Return the max_warnings threshold, from CLI or INI, or None if unset."""
     value = self.config.option.max_warnings
     if value is not None:
         return int(value)
     ini_value = self.config.getini("max_warnings")
-    if ini_value:
-        return int(ini_value)
-    return None
+    if ini_value == "":
+        return None
+    return int(ini_value)

A temporary prototype was tested with:

  • unquoted and quoted values in pytest.toml;
  • unquoted and quoted values under [tool.pytest];
  • pytest.ini;
  • -o;
  • an unset option;
  • an explicit zero;
  • --max-warnings precedence.

The prototype passed the focused coverage listed above, and the existing testing/test_warnings.py::TestMaxWarnings tests passed unchanged (11 passed).

This is one possible implementation direction, not a requirement. A documentation change requiring quoted strings remains another possible resolution.

</details>

<details>
<summary><strong>History and related precedent</strong></summary>

#14371 requested max_warnings = N, where N is an integer. #14372 added the option, described the work as INI configuration support, and tested configuration through pytest.ini; its documentation also declared int and showed an unquoted native TOML integer.

The closest implementation precedent is #14692, which fixed #14675. The numeric truncation_limit_* options had the same native-TOML mismatch, and the accepted fix registered them as int | str so native integers and existing string values both remained supported.

There are also documentation-only precedents. #13906 quoted native TOML examples for three string-registered verbosity settings. #14255 quoted numeric logging levels, which also accept named levels.

</details>

Environment

  • pytest pinned revision 5d85267e3539968dad33937295b0a77b6c7eb7b3
  • pytest stable 9.1.1
  • Python 3.12.13
  • OS Red Hat Enterprise Linux 9.8 (Plow)

AI assistance

AI assistance was used collaboratively during the investigation and drafting; I directed the work, reproduced the behavior, reviewed the source and history, and personally verified the claims and test results.

Should max_warnings accept both native TOML integers and the existing string representation, or should the documentation require quoted strings instead?

The two patches

Our patch+127 −5
changelog/14953.bugfix.rstnew file mode 100644index 000000000..05984a52d@@ -0,0 +1 @@+The :confval:`max_warnings` and :confval:`faulthandler_timeout` configuration options now accept native numbers in TOML configuration files, as shown in their documentation, while still accepting strings for backward compatibility.doc/en/reference/reference.rstindex 362d5c917..890dacef2 100644@@ -2217,7 +2217,7 @@ passed multiple times. The expected format is ``name=value``. For example::        .. code-block:: toml              [pytest]-            minversion = 3.0  # will fail if we run with pytest-2.8+            minversion = "3.0"  # will fail if we run with pytest-2.8     .. tab:: ini src/_pytest/faulthandler.pyindex 080cf5838..f6fe973ff 100644@@ -24,7 +24,9 @@ def pytest_addoption(parser: Parser) -> None:         "Exit the test process if a test takes more than "         "faulthandler_timeout seconds to finish"     )-    parser.addini("faulthandler_timeout", help_timeout, default=0.0)+    # ``str | float`` so native TOML numbers are accepted, while INI files and+    # ``-o`` overrides keep providing the value as a string.+    parser.addini("faulthandler_timeout", help_timeout, type=str | float, default=0.0)     parser.addini(         "faulthandler_exit_on_timeout", help_exit_on_timeout, type="bool", default=False     )src/_pytest/main.pyindex 1b337e20c..b475d370a 100644@@ -141,9 +141,13 @@ def pytest_addoption(parser: Parser) -> None:         "warnings.filterwarnings. "         "Processed after -W/--pythonwarnings.",     )+    # ``str | int`` so native TOML integers are accepted, while INI files and+    # ``-o`` overrides keep providing the value as a string.     parser.addini(         "max_warnings",         help="Exit with error if all tests pass but the number of warnings exceeds this threshold",+        type=str | int,+        default="",     )      group = parser.getgroup("collect", "collection")src/_pytest/terminal.pyindex 825435225..db84e825a 100644@@ -1093,9 +1093,11 @@ class TerminalReporter:         if value is not None:             return int(value)         ini_value = self.config.getini("max_warnings")-        if ini_value:-            return int(ini_value)-        return None+        # Check against the empty default explicitly: a native TOML ``0`` is+        # falsy but is a valid threshold.+        if ini_value == "":+            return None+        return int(ini_value)      #     # Summaries for sessionfinish.testing/test_faulthandler.pyindex 67ca221f3..d649d83da 100644@@ -155,6 +155,57 @@ def test_timeout_and_exit(pytester: Pytester, exit_on_timeout: bool) -> None:     assert result.ret == 1  +@pytest.mark.parametrize(+    ("value", "expected"),+    [("5", 5.0), ("0.5", 0.5), ('"5"', 5.0), ('"0.5"', 0.5), ("0", 0.0)],+)+@pytest.mark.parametrize("config_file", ["pytest.toml", "pyproject.toml"])+def test_timeout_toml_value(+    pytester: Pytester, config_file: str, value: str, expected: float+) -> None:+    """faulthandler_timeout accepts native TOML numbers and quoted strings (#14953)."""+    from _pytest.faulthandler import get_timeout_config_value++    section = "[pytest]" if config_file == "pytest.toml" else "[tool.pytest]"+    pytester.makefile(+        ".toml", **{config_file[:-5]: f"{section}\nfaulthandler_timeout = {value}\n"}+    )+    config = pytester.parseconfig()+    assert get_timeout_config_value(config) == expected+++def test_timeout_toml_native_number_dumps_traceback(pytester: Pytester) -> None:+    """A native TOML faulthandler_timeout triggers the traceback dump (#14953)."""+    pytester.makepyfile(+        """+    import os, time+    def test_timeout():+        time.sleep(1 if "CI" in os.environ else 0.1)+    """+    )+    pytester.maketoml(+        """+        [pytest]+        faulthandler_timeout = 0.01+        """+    )+    result = pytester.runpytest_subprocess()+    result.stderr.fnmatch_lines(["*most recent call first*"])+    result.stdout.fnmatch_lines(["*1 passed*"])+    assert result.ret == 0+++def test_timeout_config_unset_and_ini(pytester: Pytester) -> None:+    """Unset and INI-file faulthandler_timeout values keep working."""+    from _pytest.faulthandler import get_timeout_config_value++    assert get_timeout_config_value(pytester.parseconfig()) == 0.0+    pytester.makeini("[pytest]\nfaulthandler_timeout = 2.5\n")+    config = pytester.parseconfig()+    assert config.getini("faulthandler_timeout") == "2.5"+    assert get_timeout_config_value(config) == 2.5++ @pytest.mark.parametrize("hook_name", ["pytest_enter_pdb", "pytest_exception_interact"]) def test_cancel_timeout_on_hook(monkeypatch, hook_name) -> None:     """Make sure that we are cancelling any scheduled traceback dumping duetesting/test_warnings.pyindex 017781c23..bde981c54 100644@@ -1145,6 +1145,68 @@ class TestMaxWarnings:         result.assert_outcomes(passed=1, warnings=1)         assert result.ret == ExitCode.OK +    @pytest.mark.parametrize("value", ["0", '"0"', "1", '"1"'])+    @pytest.mark.parametrize("config_file", ["pytest.toml", "pyproject.toml"])+    @pytest.mark.filterwarnings("default::UserWarning")+    def test_max_warnings_toml_exceeded(+        self, pytester: Pytester, config_file: str, value: str+    ) -> None:+        """max_warnings accepts native TOML integers and quoted strings (#14953).++        An explicit ``0`` must be honored as a threshold, not treated as unset.+        """+        section = "[pytest]" if config_file == "pytest.toml" else "[tool.pytest]"+        pytester.makefile(+            ".toml", **{config_file[:-5]: f"{section}\nmax_warnings = {value}\n"}+        )+        pytester.makepyfile(self.PYFILE)+        result = pytester.runpytest()+        assert result.ret == ExitCode.MAX_WARNINGS_ERROR+        expected = value.strip('"')+        result.stdout.fnmatch_lines(+            [f"*Tests pass, but maximum allowed warnings exceeded: 2 > {expected}*"]+        )++    @pytest.mark.parametrize("value", ["2", '"2"'])+    @pytest.mark.filterwarnings("default::UserWarning")+    def test_max_warnings_toml_not_exceeded(+        self, pytester: Pytester, value: str+    ) -> None:+        """A TOML max_warnings at or above the warning count passes (#14953)."""+        pytester.maketoml(f"[pytest]\nmax_warnings = {value}\n")+        pytester.makepyfile(self.PYFILE)+        result = pytester.runpytest()+        result.assert_outcomes(passed=2, warnings=2)+        assert result.ret == ExitCode.OK++    @pytest.mark.filterwarnings("default::UserWarning")+    def test_max_warnings_cli_overrides_toml(self, pytester: Pytester) -> None:+        """--max-warnings takes precedence over a native TOML max_warnings."""+        pytester.maketoml("[pytest]\nmax_warnings = 0\n")+        pytester.makepyfile(self.PYFILE)+        result = pytester.runpytest("--max-warnings", "2")+        result.assert_outcomes(passed=2, warnings=2)+        assert result.ret == ExitCode.OK++    @pytest.mark.filterwarnings("default::UserWarning")+    def test_max_warnings_override_ini(self, pytester: Pytester) -> None:+        """max_warnings can be set via -o, including an explicit zero."""+        pytester.makepyfile(self.PYFILE)+        result = pytester.runpytest("-o", "max_warnings=0")+        assert result.ret == ExitCode.MAX_WARNINGS_ERROR++    def test_max_warnings_getini_types(self, pytester: Pytester) -> None:+        """getini keeps string values as strings and native TOML ints as ints."""+        config = pytester.parseconfig()+        assert config.getini("max_warnings") == ""+        pytester.maketoml("[pytest]\nmax_warnings = 0\n")+        assert pytester.parseconfig().getini("max_warnings") == 0+        pytester.maketoml('[pytest]\nmax_warnings = "3"\n')+        assert pytester.parseconfig().getini("max_warnings") == "3"+        pytester.maketoml("[pytest]\nmax_warnings = true\n")+        with pytest.raises(pytest.UsageError, match="max_warnings"):+            pytester.parseconfig().getini("max_warnings")+  def test_pythonwarnings_not_duplicated(pytester: Pytester) -> None:     """Regression test for #13484: -W values should not be duplicated in 
Merged fix · PR #14954+28 −5
AUTHORSindex ba5672d4c..093d5fd47 100644@@ -87,6 +87,7 @@ croc100 Cal Leeming Carl Friedrich Bolz Carlos Jenkins+Cen Fangyu (Dmao233) Ceridwen Charles Cloud Charles Machalowchangelog/14953.bugfix.rstnew file mode 100644index 000000000..bacb10587@@ -0,0 +1 @@+The :confval:`max_warnings` configuration option now accepts integer values in TOML configuration files, while still accepting strings for backward compatibility.doc/en/reference/reference.rstindex 362d5c917..cb9974433 100644@@ -1705,9 +1705,11 @@ passed multiple times. The expected format is ``name=value``. For example::   .. confval:: max_warnings-   :type: ``int``+   :type: ``int | str``     .. versionadded:: 9.1+   .. versionchanged:: 9.2+        Added support for specifying the value as an integer in TOML configuration.     Maximum number of warnings allowed before the test run is considered a failure.    When all tests pass, but the total number of warnings exceeds this value, pytest exits with@@ -3671,7 +3673,7 @@ All the command-line flags can also be obtained by running ``pytest --help``::                             Each line specifies a pattern for                             warnings.filterwarnings. Processed after                             -W/--pythonwarnings.-      max_warnings (string):+      max_warnings (int | string):                             Exit with error if all tests pass but the number of                             warnings exceeds this threshold       norecursedirs (args): Directory patterns to avoid for recursionsrc/_pytest/main.pyindex 1b337e20c..adfdb37db 100644@@ -144,6 +144,8 @@ def pytest_addoption(parser: Parser) -> None:     parser.addini(         "max_warnings",         help="Exit with error if all tests pass but the number of warnings exceeds this threshold",+        type=int | str,+        default=None,     )      group = parser.getgroup("collect", "collection")src/_pytest/terminal.pyindex 825435225..481cbc969 100644@@ -1092,10 +1092,10 @@ class TerminalReporter:         value = self.config.option.max_warnings         if value is not None:             return int(value)-        ini_value = self.config.getini("max_warnings")-        if ini_value:+        ini_value: int | str | None = self.config.getini("max_warnings")+        if isinstance(ini_value, str):             return int(ini_value)-        return None+        return ini_value      #     # Summaries for sessionfinish.testing/test_warnings.pyindex 017781c23..0344cb453 100644@@ -1065,6 +1065,23 @@ class TestMaxWarnings:         result = pytester.runpytest()         assert result.ret == ExitCode.MAX_WARNINGS_ERROR +    @pytest.mark.filterwarnings("default::UserWarning")+    @pytest.mark.parametrize("value", ["1", '"1"'])+    def test_max_warnings_toml_option(self, pytester: Pytester, value: str) -> None:+        """max_warnings can be set via TOML configuration.++        Supports both int and str (for backward compat).+        """+        pytester.maketoml(+            f"""+            [pytest]+            max_warnings = {value}+            """+        )+        pytester.makepyfile(self.PYFILE)+        result = pytester.runpytest()+        assert result.ret == ExitCode.MAX_WARNINGS_ERROR+     @pytest.mark.filterwarnings("default::UserWarning")     def test_max_warnings_with_test_failure(self, pytester: Pytester) -> None:         """When tests fail AND warnings exceed max, TESTS_FAILED takes priority."""