Backtestspytestrun-20261106-234554#15290

Issue #15290 Merged fix PR #15296 Run 6 Nov 2026 · run-20261106-234554 pytest-dev/pytest

TOML config opts verbosity_assertions, verbosity_subtests, & verbosity_test_cases no longer accept numeric string values in 9.2.0

Better

Our patch was judged better than the merged fix.

In this judgment, Patch A = our agent, Patch B = the merged fix.

Both patches make the independent tests pass, so the decision rests on how they fix it and what else they change. Patch A fixes the problem at the config type-coercion layer: in TOML mode, a string value for an option registered as needing legacy string coercion is run through the existing INI int/float/bool conversion before the type error is raised. The declared option type stays `int | Literal["auto"]`, so help output, docs and the error message remain accurate, and invalid strings like "loud" or "2.5" still produce the same clear error. Its tests cover all three options, positive, zero and negative strings, invalid strings, and an end-to-end run. Patch B instead loosens the declared type to `int | str`, which makes the config layer accept any string and pushes validation into get_verbosity, weakening the type information that TOML native typing was meant to provide. It also introduces a new deprecation warning (including a docs section and a 'deprecated in 9.2.1' marker) and extends that deprecation to tmp_path_retention_count, an option the issue never mentions. The issue points out that the changelog promised numeric strings keep working for backward compatibility; adding a deprecation and a scheduled removal in a bugfix is a policy change beyond the ask, and it adds risk (warning emission mid-session via a config-time warning helper, per-option warning dedup state on Config, an extra option touched). B also removes an existing invalid-value test case. A resolves the issue narrowly and correctly with thorough tests; B over-reaches and degrades type precision.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

6m 40s

Wall-clock time in the container

Full test suite

No new failures

2,376 passed · 23 skipped · 2,399 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

60vs 124 merged

Changed lines, ours vs merged

As of version 9.2.0, the options verbosity_assertions, verbosity_subtests, & verbosity_test_cases no longer accept numeric string values in pyproject.toml configuration files.

Despite the changelog entry stating that they still accept numeric string values for backwards compatibility, they only accept integer values or 'auto'.

The two patches

Our patch+60 −0
changelog/15290.bugfix.rstnew file mode 100644index 000000000..ea4148ae0@@ -0,0 +1 @@+The :confval:`verbosity_assertions`, :confval:`verbosity_test_cases` and :confval:`verbosity_subtests` configuration options once again accept numeric string values (e.g. ``"2"``) in TOML configuration files, as documented for backward compatibility; this regressed in pytest 9.2.0.src/_pytest/config/__init__.pyindex 60ba5f54a..f0c9fae2c 100644@@ -1964,6 +1964,23 @@ class Config:                     )                 except (TypeError, ValueError):                     pass+            # For backward compatibility, some options also accept in TOML+            # files the string form of a scalar member (e.g. "2" for+            # `int | Literal["auto"]`), coerced like in INI mode.+            if (+                mode == "toml"+                and isinstance(value, str)+                and canonical_name in self._parser._ini_toml_str_coercion+            ):+                for member in type:+                    if member not in ("int", "float", "bool"):+                        continue+                    try:+                        return self._getini_value(+                            "ini", name, canonical_name, member, value, default+                        )+                    except (TypeError, ValueError):+                        pass             raise TypeError(                 f"{self.inipath}: config option '{name}' expects one of "                 f"{_ini_type_repr(type)}, got {builtins.type(value).__name__}: {value!r}"@@ -2292,6 +2309,9 @@ class Config:             type=int | Literal["auto"],             default=Config._VERBOSITY_INI_DEFAULT,         )+        # Verbosity levels used to be strings, so numeric strings like "2"+        # are still accepted in TOML files.+        parser._ini_toml_str_coercion.add(Config._verbosity_ini_name(verbosity_type))      def _warn_about_missing_assertion(self, mode: str) -> None:         if not _assertion_supported():src/_pytest/config/argparsing.pyindex 7180cde2c..3de9483f1 100644@@ -144,6 +144,10 @@ class Parser:         self._inidict: dict[str, tuple[str, IniType, Any]] = {}         # Maps alias -> canonical name.         self._ini_aliases: dict[str, str] = {}+        # Canonical names of union-typed ini options which, for backward+        # compatibility, also accept the string form of their int/float/bool+        # members in TOML files (e.g. "2" for `int | Literal["auto"]`).+        self._ini_toml_str_coercion: set[str] = set()      @property     def prog(self) -> str:testing/test_config.pyindex 282e66409..6c0f4c1d6 100644@@ -3645,3 +3645,38 @@ class TestNativeTomlOptionTypes:         pytester.maketoml("[pytest]\nverbosity_assertions = 2\n")         config = pytester.parseconfig()         assert config.get_verbosity(Config.VERBOSITY_ASSERTIONS) == 2++    @pytest.mark.parametrize(+        "name",+        ["verbosity_assertions", "verbosity_test_cases", "verbosity_subtests"],+    )+    @pytest.mark.parametrize(+        ("toml_value", "expected"), [('"2"', 2), ('"-1"', -1), ('"0"', 0)]+    )+    def test_verbosity_numeric_string(+        self, pytester: Pytester, name: str, toml_value: str, expected: int+    ) -> None:+        """Numeric strings are still accepted for backward compatibility (#15290)."""+        pytester.makepyprojecttoml(f"[tool.pytest]\n{name} = {toml_value}\n")+        config = pytester.parseconfig()+        assert config.getini(name) == expected+        verbosity_type = name.removeprefix("verbosity_")+        assert config.get_verbosity(verbosity_type) == expected++    @pytest.mark.parametrize("value", ['"loud"', '"2.5"', '""'])+    def test_verbosity_invalid_string(self, pytester: Pytester, value: str) -> None:+        pytester.makepyprojecttoml(f"[tool.pytest]\nverbosity_test_cases = {value}\n")+        config = pytester.parseconfig()+        with pytest.raises(+            UsageError,+            match=r"config option 'verbosity_test_cases' expects one of int \| 'auto'",+        ):+            config.get_verbosity(Config.VERBOSITY_TEST_CASES)++    def test_verbosity_numeric_string_end_to_end(self, pytester: Pytester) -> None:+        """A numeric string verbosity in pyproject.toml takes effect (#15290)."""+        pytester.makepyprojecttoml('[tool.pytest]\nverbosity_test_cases = "1"\n')+        pytester.makepyfile("def test_one(): pass")+        result = pytester.runpytest()+        result.stdout.fnmatch_lines(["*test_one PASSED*"])+        assert result.ret == 0 
Merged fix · PR #15296+119 −5
changelog/15290.bugfix.rstnew file mode 100644index 000000000..533bf64d2@@ -0,0 +1 @@+Native TOML configuration again accepts numeric strings such as ``"2"`` for :confval:`verbosity_assertions`, :confval:`verbosity_test_cases` and :confval:`verbosity_subtests`, now with a deprecation warning -- regressed by :pr:`15153`.changelog/15290.deprecation.rstnew file mode 100644index 000000000..496f0e5e1@@ -0,0 +1 @@+Numeric strings such as ``"2"`` for :confval:`verbosity_assertions`, :confval:`verbosity_test_cases`, :confval:`verbosity_subtests` and :confval:`tmp_path_retention_count` in native TOML configuration are deprecated, use integers instead -- see :ref:`toml-numeric-strings`.doc/en/deprecations.rstindex 328411a9f..874ff25d2 100644@@ -14,6 +14,30 @@ Deprecated Features Below is a complete list of all pytest features which are considered deprecated. Using those features will issue :class:`~pytest.PytestWarning` or subclasses, which can be filtered using :ref:`standard warning filters <warnings>`. +.. _toml-numeric-strings:++Numeric strings for integer options in TOML+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~++.. deprecated:: 9.2.1++In native TOML configuration, :confval:`verbosity_assertions`,+:confval:`verbosity_test_cases` and :confval:`verbosity_subtests` take an+integer or ``"auto"``, and :confval:`tmp_path_retention_count` takes an+integer. Numeric strings are still accepted for backward+compatibility, but issue a deprecation warning and will be rejected in pytest 10.++.. code-block:: toml++    # Deprecated+    verbosity_assertions = "2"++    # Use this instead+    verbosity_assertions = 2++INI files are not affected, since all their values are strings.++ .. _terminalreporter-write-line-bytes:  Passing ``bytes`` to ``TerminalReporter.write_line()``src/_pytest/config/__init__.pyindex 60ba5f54a..d19722287 100644@@ -1261,6 +1261,7 @@ class Config:         self.trace = self.pluginmanager.trace.root.get("config")         self.hook = self.pluginmanager.hook         self._inicache: dict[str, Any] = {}+        self._warned_verbosity_strings: set[str] = set()         self._inicfg: ConfigDict = {}         self._cleanup_stack = contextlib.ExitStack()         self.pluginmanager.register(self, "pytestconfig")@@ -2268,8 +2269,26 @@ class Config:         level = self.getini(ini_name)         if level == Config._VERBOSITY_INI_DEFAULT:             return global_level--        return int(level)+        if isinstance(level, int):+            return level+        try:+            int_level = int(level)+        except ValueError:+            raise UsageError(+                f"{self.inipath}: config option '{ini_name}' expects an int "+                f"or 'auto', got {level!r}"+            ) from None+        # A numeric str only gets here from TOML: in ini mode the int member+        # of the option type already converted it.+        if ini_name not in self._warned_verbosity_strings:+            self._warned_verbosity_strings.add(ini_name)+            self.issue_config_time_warning(+                _pytest.deprecated.TOML_NUMERIC_STRING.format(+                    name=ini_name, value=level+                ),+                stacklevel=2,+            )+        return int_level      @staticmethod     def _verbosity_ini_name(verbosity_type: str) -> str:@@ -2289,7 +2308,9 @@ class Config:         parser.addini(             Config._verbosity_ini_name(verbosity_type),             help=help,-            type=int | Literal["auto"],+            # str rather than Literal["auto"]: numeric strings in TOML are+            # deprecated but still accepted, see get_verbosity.+            type=int | str,             default=Config._VERBOSITY_INI_DEFAULT,         ) src/_pytest/deprecated.pyindex 98b7e112b..c6999dd54 100644@@ -114,6 +114,13 @@ INI_STRING_TYPE_NON_STR_VALUE = PytestRemovedIn10Warning(     '(e.g. type="args") instead of relying on the implicit string default.' ) +TOML_NUMERIC_STRING = UnformattedWarning(+    PytestRemovedIn10Warning,+    "{name} = {value!r} in TOML configuration is deprecated, "+    "use an integer instead: {name} = {value}\n"+    "See https://docs.pytest.org/en/stable/deprecations.html#toml-numeric-strings",+)+ # You want to make some `__init__` or function "private". # #   def my_private_function(some, args):src/_pytest/tmpdir.pyindex ca94fdde1..3e68606f8 100644@@ -30,6 +30,7 @@ from _pytest.config import ExitCode from _pytest.config import hookimpl from _pytest.config.argparsing import Parser from _pytest.deprecated import check_ispytest+from _pytest.deprecated import TOML_NUMERIC_STRING from _pytest.fixtures import fixture from _pytest.fixtures import FixtureRequest from _pytest.monkeypatch import MonkeyPatch@@ -97,7 +98,18 @@ class TempPathFactory:         :meta private:         """         check_ispytest(_ispytest)-        count = int(config.getini("tmp_path_retention_count"))+        count = config.getini("tmp_path_retention_count")+        if isinstance(count, str):+            # A str only gets here from TOML: in ini mode the int member of+            # the option type already converted it.+            value = count+            count = int(value)+            config.issue_config_time_warning(+                TOML_NUMERIC_STRING.format(+                    name="tmp_path_retention_count", value=value+                ),+                stacklevel=2,+            )         if count < 0:             raise ValueError(                 f"tmp_path_retention_count must be >= 0. Current input: {count}."@@ -299,6 +311,7 @@ def pytest_addoption(parser: Parser) -> None:         "tmp_path_retention_count",         help="How many sessions should we keep the `tmp_path` directories, according to `tmp_path_retention_policy`.",         default=3,+        # str: numeric strings in TOML are deprecated but still accepted.         type=int | str,     ) testing/test_config.pyindex 282e66409..71cf1779f 100644@@ -13,6 +13,7 @@ import textwrap from typing import Any from typing import cast from typing import Literal+import warnings  import _pytest._code from _pytest.assertion.rewrite import AssertionRewritingHook@@ -3630,7 +3631,6 @@ class TestNativeTomlOptionTypes:     @pytest.mark.parametrize(         ("name", "value"),         [-            ("verbosity_assertions", '"loud"'),             ("log_file_mode", '"x"'),             ("parametrize_long_str_id_strategy", '"bogus"'),         ],@@ -3645,3 +3645,36 @@ class TestNativeTomlOptionTypes:         pytester.maketoml("[pytest]\nverbosity_assertions = 2\n")         config = pytester.parseconfig()         assert config.get_verbosity(Config.VERBOSITY_ASSERTIONS) == 2++    def test_get_verbosity_numeric_string_deprecated(self, pytester: Pytester) -> None:+        pytester.maketoml('[pytest]\nverbosity_assertions = "2"\n')+        pytester.makepyfile(+            """+            from pytest import Config++            def test_a(pytestconfig):+                assert pytestconfig.get_verbosity(Config.VERBOSITY_ASSERTIONS) == 2++            def test_b(pytestconfig):+                assert pytestconfig.get_verbosity(Config.VERBOSITY_ASSERTIONS) == 2+            """+        )+        result = pytester.runpytest()+        result.assert_outcomes(passed=2, warnings=1)+        result.stdout.fnmatch_lines(+            ["*PytestRemovedIn10Warning: verbosity_assertions = '2' in TOML*"]+        )++    def test_get_verbosity_ini_numeric_string(self, pytester: Pytester) -> None:+        pytester.makeini("[pytest]\nverbosity_assertions = 2\n")+        config = pytester.parseconfig()+        with warnings.catch_warnings():+            warnings.simplefilter("error")+            assert config.get_verbosity(Config.VERBOSITY_ASSERTIONS) == 2++    @pytest.mark.parametrize("toml_value", ['"loud"', "true", "2.5"])+    def test_get_verbosity_invalid(self, pytester: Pytester, toml_value: str) -> None:+        pytester.maketoml(f"[pytest]\nverbosity_assertions = {toml_value}\n")+        config = pytester.parseconfig()+        with pytest.raises(UsageError, match="verbosity_assertions"):+            config.get_verbosity(Config.VERBOSITY_ASSERTIONS)testing/test_tmpdir.pyindex 93177e122..5c53e5923 100644@@ -232,6 +232,20 @@ class TestConfigTmpPath:             # Check the base dir itself is gone without depending on test results             assert base_dir == [] +    def test_retention_count_toml_string_deprecated(self, pytester: Pytester) -> None:+        pytester.maketoml('[pytest]\ntmp_path_retention_count = "2"\n')+        pytester.makepyfile(+            """+            def test_1(tmp_path_factory):+                assert tmp_path_factory._retention_count == 2+            """+        )+        result = pytester.runpytest()+        result.assert_outcomes(passed=1, warnings=1)+        result.stdout.fnmatch_lines(+            ["*PytestRemovedIn10Warning: tmp_path_retention_count = '2' in TOML*"]+        )+     @pytest.mark.parametrize("policy", ['"failed"', '"all"'])     @pytest.mark.parametrize("count", [0, 1, 3])     def test_retention_count(self, pytester: Pytester, policy, count) -> None: