Backtestspytestrun-20261106-234554#15132

Issue #15132 Merged fix PR #15133 Run 6 Nov 2026 · run-20261106-234554 pytest-dev/pytest

Native TOML numeric faulthandler_timeout is rejected

Better

Our patch was judged better than the merged fix.

In this judgment, Patch B = our agent, Patch A = the merged fix.

Both patches make the identical source change: `parser.addini("faulthandler_timeout", ..., type=float | str, default=0.0)`, so correctness is equal and the independent tests confirm both fix the reported rejection of native TOML ints/floats while keeping quoted strings working. The difference is in tests, docs, and scope. Patch A adds one small parametrized test that only checks `getini` returns the raw value (5, 0.01, or the string), which verifies acceptance but not that the value is actually honoured by the timeout path. Patch B adds broader coverage: it goes through `get_timeout_config_value` (the real consumer, which coerces to float), checks the empty-string disable case, checks that INI values still coerce, checks that non-numeric non-string TOML values (bool, array) are still rejected, and adds an end-to-end subprocess test showing a native TOML float actually triggers the traceback dump. One risk in B is that the invalid-value test asserts an exact error-message fragment (`expects one of float | string`) whose wording I cannot verify from the diff; if it differs the test would fail, but the rest of B's suite stands independently. B also updates the reference docs (`:type:` and the `--help` listing) to reflect the new accepted type, which A leaves stale. B's incidental doc change to `minversion = "3.0"` is unrelated churn, though harmless and arguably a correction. A's AUTHORS edit is neutral. Overall B delivers the same fix with meaningfully stronger regression coverage and kept documentation consistent, so B is the better patch.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

4m 19s

Wall-clock time in the container

Full test suite

No new failures

22 passed · 0 skipped · 22 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

84vs 22 merged

Changed lines, ours vs merged

Summary

faulthandler_timeout rejects native numeric values in TOML configuration, even though the option is documented as a float and the TOML examples use a number.

Reproduction

On pytest main at f5240bef4, with Python 3.12 on Windows:

[pytest]
faulthandler_timeout = 5

Running a test produces:

ERROR: pytest.toml: config option 'faulthandler_timeout' expects a string, got int: 5

The same option works when configured through INI-style configuration or as a quoted TOML string.

Expected behavior

Native TOML integers and floats should be accepted for this float-valued option, consistent with the documentation and the other typed TOML options.

I have a small local fix and regression test ready for review.

The two patches

Our patch+80 −4
changelog/15132.bugfix.rstnew file mode 100644index 000000000..a9db0380c@@ -0,0 +1 @@+The :confval:`faulthandler_timeout` configuration option now accepts integer and float values in TOML configuration files, while still accepting strings for backward compatibility.doc/en/reference/reference.rstindex 540742624..e55f48605 100644@@ -1650,9 +1650,12 @@ passed multiple times. The expected format is ``name=value``. For example::   .. confval:: faulthandler_timeout-   :type: ``float``+   :type: ``float | str``    :default: ``0`` (disabled) +   .. versionchanged:: 9.2+        Added support for specifying the value as a number in TOML configuration.+    Dumps the tracebacks of all threads if a test takes longer than ``X`` seconds to run (including    fixture setup and teardown). Implemented using the :func:`faulthandler.dump_traceback_later` function,    so all caveats there apply.@@ -2222,7 +2225,7 @@ passed multiple times. The expected format is ``name=value``. For example::        .. code-block:: toml              [pytest]-            minversion = 3.0  # will fail if we run with pytest-2.8+            minversion = "3.0"  # will fail if we run with pytest-2.8     .. tab:: ini @@ -3787,7 +3790,7 @@ All the command-line flags can also be obtained by running ``pytest --help``::                             Default value for --log-file-date-format       log_auto_indent (string):                             Default value for --log-auto-indent-      faulthandler_timeout (string):+      faulthandler_timeout (float | string):                             Dump the traceback of all threads if a test takes                             more than TIMEOUT seconds to finish       faulthandler_exit_on_timeout (bool):src/_pytest/faulthandler.pyindex 080cf5838..e92b98936 100644@@ -24,7 +24,7 @@ def pytest_addoption(parser: Parser) -> None:         "Exit the test process if a test takes more than "         "faulthandler_timeout seconds to finish"     )-    parser.addini("faulthandler_timeout", help_timeout, default=0.0)+    parser.addini("faulthandler_timeout", help_timeout, type=float | str, default=0.0)     parser.addini(         "faulthandler_exit_on_timeout", help_exit_on_timeout, type="bool", default=False     )testing/test_faulthandler.pyindex 67ca221f3..3c7053c43 100644@@ -118,6 +118,78 @@ def test_timeout(pytester: Pytester, enabled: bool) -> None:     assert result.ret == 0  +@pytest.mark.parametrize(+    ("value", "expected"),+    [("5", 5.0), ("0.5", 0.5), ('"5"', 5.0), ('"0.5"', 0.5), ('""', 0.0)],+)+def test_timeout_toml_value(pytester: Pytester, value: str, expected: float) -> None:+    """Native TOML numbers are accepted for faulthandler_timeout, as are+    strings for backward compatibility (#15132)."""+    from _pytest.faulthandler import get_timeout_config_value++    pytester.maketoml(+        f"""+        [pytest]+        faulthandler_timeout = {value}+        """+    )+    config = pytester.parseconfig()+    assert get_timeout_config_value(config) == expected+++@pytest.mark.parametrize("value", ["true", "[1]"])+def test_timeout_toml_invalid_value(pytester: Pytester, value: str) -> None:+    """Non-numeric, non-string TOML values are still rejected (#15132)."""+    pytester.maketoml(+        f"""+        [pytest]+        faulthandler_timeout = {value}+        """+    )+    pytester.makepyfile("def test_ok(): pass")+    result = pytester.runpytest()+    result.stderr.fnmatch_lines(+        ["*config option 'faulthandler_timeout' expects one of float | string, got *"]+    )+    assert result.ret == pytest.ExitCode.USAGE_ERROR+++@pytest.mark.parametrize(("value", "expected"), [("5", 5.0), ("", 0.0)])+def test_timeout_ini_value(pytester: Pytester, value: str, expected: float) -> None:+    """INI values keep being coerced to a float, an empty value disables it."""+    from _pytest.faulthandler import get_timeout_config_value++    pytester.makeini(+        f"""+        [pytest]+        faulthandler_timeout = {value}+        """+    )+    config = pytester.parseconfig()+    assert get_timeout_config_value(config) == expected+++def test_timeout_toml_number_dumps_traceback(pytester: Pytester) -> None:+    """End to end: a native TOML number enables the timeout (#15132)."""+    pytester.makepyfile(+        """+    import os, time+    def test_timeout():+        time.sleep(1 if "CI" in os.environ else 0.1)+    """+    )+    pytester.maketoml(+        """+        [pytest]+        faulthandler_timeout = 0.01+        """+    )+    result = pytester.runpytest_subprocess()+    result.stderr.fnmatch_lines(["*most recent call first*"])+    result.stdout.fnmatch_lines(["*1 passed*"])+    assert result.ret == 0++ @pytest.mark.keep_ci_var @pytest.mark.skipif(     "CI" in os.environ and sys.platform == "linux" and sys.version_info >= (3, 14), 
Merged fix · PR #15133+21 −1
AUTHORSindex 3d74b238d..f621dd6dd 100644@@ -204,6 +204,7 @@ Grig Gheorghiu Grigorii Eremeev (budulianin) Guido Wesdorp Guoqiang Zhang+hfycium Hamza Mobeen Harald Armin Massa Harshnachangelog/15132.bugfix.rstnew file mode 100644index 000000000..1bcb0b05a@@ -0,0 +1,2 @@+Native TOML numeric values are now accepted for the ``faulthandler_timeout``+configuration option.src/_pytest/faulthandler.pyindex 080cf5838..e92b98936 100644@@ -24,7 +24,7 @@ def pytest_addoption(parser: Parser) -> None:         "Exit the test process if a test takes more than "         "faulthandler_timeout seconds to finish"     )-    parser.addini("faulthandler_timeout", help_timeout, default=0.0)+    parser.addini("faulthandler_timeout", help_timeout, type=float | str, default=0.0)     parser.addini(         "faulthandler_exit_on_timeout", help_exit_on_timeout, type="bool", default=False     )testing/test_faulthandler.pyindex 67ca221f3..78681d572 100644@@ -118,6 +118,23 @@ def test_timeout(pytester: Pytester, enabled: bool) -> None:     assert result.ret == 0  +@pytest.mark.parametrize(+    ("value", "expected"), [(5, 5), (0.01, 0.01), ('"0.01"', "0.01")]+)+def test_timeout_toml(+    pytester: Pytester, value: float | str, expected: float | str+) -> None:+    """The timeout accepts native TOML numbers and strings."""+    pytester.maketoml(+        f"""+        [pytest]+        faulthandler_timeout = {value}+        """+    )+    config = pytester.parseconfig()+    assert config.getini("faulthandler_timeout") == expected++ @pytest.mark.keep_ci_var @pytest.mark.skipif(     "CI" in os.environ and sys.platform == "linux" and sys.version_info >= (3, 14),