Backtestspytestrun-20261106-234554#15132
Native TOML numeric faulthandler_timeout is rejected
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Both patches make the identical source change: `parser.addini("faulthandler_timeout", ..., type=float | str, default=0.0)`, so correctness is equal and the independent tests confirm both fix the reported rejection of native TOML ints/floats while keeping quoted strings working. The difference is in tests, docs, and scope. Patch A adds one small parametrized test that only checks `getini` returns the raw value (5, 0.01, or the string), which verifies acceptance but not that the value is actually honoured by the timeout path. Patch B adds broader coverage: it goes through `get_timeout_config_value` (the real consumer, which coerces to float), checks the empty-string disable case, checks that INI values still coerce, checks that non-numeric non-string TOML values (bool, array) are still rejected, and adds an end-to-end subprocess test showing a native TOML float actually triggers the traceback dump. One risk in B is that the invalid-value test asserts an exact error-message fragment (`expects one of float | string`) whose wording I cannot verify from the diff; if it differs the test would fail, but the rest of B's suite stands independently. B also updates the reference docs (`:type:` and the `--help` listing) to reflect the new accepted type, which A leaves stale. B's incidental doc change to `minversion = "3.0"` is unrelated churn, though harmless and arguably a correction. A's AUTHORS edit is neutral. Overall B delivers the same fix with meaningfully stronger regression coverage and kept documentation consistent, so B is the better patch.
Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.
Solve
4m 19s
Wall-clock time in the container
Full test suite
No new failures
22 passed · 0 skipped · 22 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
84vs 22 merged
Changed lines, ours vs merged
The issue
View on GitHubSummary
faulthandler_timeout rejects native numeric values in TOML configuration, even though the option is documented as a float and the TOML examples use a number.
Reproduction
On pytest main at f5240bef4, with Python 3.12 on Windows:
[pytest]
faulthandler_timeout = 5
Running a test produces:
ERROR: pytest.toml: config option 'faulthandler_timeout' expects a string, got int: 5
The same option works when configured through INI-style configuration or as a quoted TOML string.
Expected behavior
Native TOML integers and floats should be accepted for this float-valued option, consistent with the documentation and the other typed TOML options.
I have a small local fix and regression test ready for review.
The two patches
changelog/15132.bugfix.rstnew file mode 100644index 000000000..a9db0380c@@ -0,0 +1 @@+The :confval:`faulthandler_timeout` configuration option now accepts integer and float values in TOML configuration files, while still accepting strings for backward compatibility.doc/en/reference/reference.rstindex 540742624..e55f48605 100644@@ -1650,9 +1650,12 @@ passed multiple times. The expected format is ``name=value``. For example:: .. confval:: faulthandler_timeout- :type: ``float``+ :type: ``float | str`` :default: ``0`` (disabled) + .. versionchanged:: 9.2+ Added support for specifying the value as a number in TOML configuration.+ Dumps the tracebacks of all threads if a test takes longer than ``X`` seconds to run (including fixture setup and teardown). Implemented using the :func:`faulthandler.dump_traceback_later` function, so all caveats there apply.@@ -2222,7 +2225,7 @@ passed multiple times. The expected format is ``name=value``. For example:: .. code-block:: toml [pytest]- minversion = 3.0 # will fail if we run with pytest-2.8+ minversion = "3.0" # will fail if we run with pytest-2.8 .. tab:: ini @@ -3787,7 +3790,7 @@ All the command-line flags can also be obtained by running ``pytest --help``:: Default value for --log-file-date-format log_auto_indent (string): Default value for --log-auto-indent- faulthandler_timeout (string):+ faulthandler_timeout (float | string): Dump the traceback of all threads if a test takes more than TIMEOUT seconds to finish faulthandler_exit_on_timeout (bool):src/_pytest/faulthandler.pyindex 080cf5838..e92b98936 100644@@ -24,7 +24,7 @@ def pytest_addoption(parser: Parser) -> None: "Exit the test process if a test takes more than " "faulthandler_timeout seconds to finish" )- parser.addini("faulthandler_timeout", help_timeout, default=0.0)+ parser.addini("faulthandler_timeout", help_timeout, type=float | str, default=0.0) parser.addini( "faulthandler_exit_on_timeout", help_exit_on_timeout, type="bool", default=False )testing/test_faulthandler.pyindex 67ca221f3..3c7053c43 100644@@ -118,6 +118,78 @@ def test_timeout(pytester: Pytester, enabled: bool) -> None: assert result.ret == 0 +@pytest.mark.parametrize(+ ("value", "expected"),+ [("5", 5.0), ("0.5", 0.5), ('"5"', 5.0), ('"0.5"', 0.5), ('""', 0.0)],+)+def test_timeout_toml_value(pytester: Pytester, value: str, expected: float) -> None:+ """Native TOML numbers are accepted for faulthandler_timeout, as are+ strings for backward compatibility (#15132)."""+ from _pytest.faulthandler import get_timeout_config_value++ pytester.maketoml(+ f"""+ [pytest]+ faulthandler_timeout = {value}+ """+ )+ config = pytester.parseconfig()+ assert get_timeout_config_value(config) == expected+++@pytest.mark.parametrize("value", ["true", "[1]"])+def test_timeout_toml_invalid_value(pytester: Pytester, value: str) -> None:+ """Non-numeric, non-string TOML values are still rejected (#15132)."""+ pytester.maketoml(+ f"""+ [pytest]+ faulthandler_timeout = {value}+ """+ )+ pytester.makepyfile("def test_ok(): pass")+ result = pytester.runpytest()+ result.stderr.fnmatch_lines(+ ["*config option 'faulthandler_timeout' expects one of float | string, got *"]+ )+ assert result.ret == pytest.ExitCode.USAGE_ERROR+++@pytest.mark.parametrize(("value", "expected"), [("5", 5.0), ("", 0.0)])+def test_timeout_ini_value(pytester: Pytester, value: str, expected: float) -> None:+ """INI values keep being coerced to a float, an empty value disables it."""+ from _pytest.faulthandler import get_timeout_config_value++ pytester.makeini(+ f"""+ [pytest]+ faulthandler_timeout = {value}+ """+ )+ config = pytester.parseconfig()+ assert get_timeout_config_value(config) == expected+++def test_timeout_toml_number_dumps_traceback(pytester: Pytester) -> None:+ """End to end: a native TOML number enables the timeout (#15132)."""+ pytester.makepyfile(+ """+ import os, time+ def test_timeout():+ time.sleep(1 if "CI" in os.environ else 0.1)+ """+ )+ pytester.maketoml(+ """+ [pytest]+ faulthandler_timeout = 0.01+ """+ )+ result = pytester.runpytest_subprocess()+ result.stderr.fnmatch_lines(["*most recent call first*"])+ result.stdout.fnmatch_lines(["*1 passed*"])+ assert result.ret == 0++ @pytest.mark.keep_ci_var @pytest.mark.skipif( "CI" in os.environ and sys.platform == "linux" and sys.version_info >= (3, 14),
AUTHORSindex 3d74b238d..f621dd6dd 100644@@ -204,6 +204,7 @@ Grig Gheorghiu Grigorii Eremeev (budulianin) Guido Wesdorp Guoqiang Zhang+hfycium Hamza Mobeen Harald Armin Massa Harshnachangelog/15132.bugfix.rstnew file mode 100644index 000000000..1bcb0b05a@@ -0,0 +1,2 @@+Native TOML numeric values are now accepted for the ``faulthandler_timeout``+configuration option.src/_pytest/faulthandler.pyindex 080cf5838..e92b98936 100644@@ -24,7 +24,7 @@ def pytest_addoption(parser: Parser) -> None: "Exit the test process if a test takes more than " "faulthandler_timeout seconds to finish" )- parser.addini("faulthandler_timeout", help_timeout, default=0.0)+ parser.addini("faulthandler_timeout", help_timeout, type=float | str, default=0.0) parser.addini( "faulthandler_exit_on_timeout", help_exit_on_timeout, type="bool", default=False )testing/test_faulthandler.pyindex 67ca221f3..78681d572 100644@@ -118,6 +118,23 @@ def test_timeout(pytester: Pytester, enabled: bool) -> None: assert result.ret == 0 +@pytest.mark.parametrize(+ ("value", "expected"), [(5, 5), (0.01, 0.01), ('"0.01"', "0.01")]+)+def test_timeout_toml(+ pytester: Pytester, value: float | str, expected: float | str+) -> None:+ """The timeout accepts native TOML numbers and strings."""+ pytester.maketoml(+ f"""+ [pytest]+ faulthandler_timeout = {value}+ """+ )+ config = pytester.parseconfig()+ assert config.getini("faulthandler_timeout") == expected++ @pytest.mark.keep_ci_var @pytest.mark.skipif( "CI" in os.environ and sys.platform == "linux" and sys.version_info >= (3, 14),