Backtestspytestrun-20261106-234554#14619

Issue #14619 Merged fix PR #14978 Run 6 Nov 2026 · run-20261106-234554 pytest-dev/pytest

Spurious comma in parametrize causes obfuscated crash

Better

Our patch was judged better than the merged fix.

In this judgment, Patch B = our agent, Patch A = the merged fix.

Both patches fix the root crash in ParameterSet._for_parametrize by detecting a non-sized value before calling len() and reporting a clean collection error that names the offending test; A does it via try/except TypeError, B via an isinstance(Sized) check, and either works for the None/int cases in the issue (independent tests 1 and 2 pass on both). The issue has a second explicit ask: when the value is a string, show it as the unpacked character list (['f','o','o']) so the user can see it was treated as a sequence. Patch B implements this and tests it; Patch A ignores it entirely, and the independent test for it fails on A. B's tests are also stronger: a parametrized case covering None/int with both the '"x,"' and tuple-argnames forms, a no_fnmatch_line('*TypeError*') guard, and assert_outcomes(errors=1), versus A's single None case. B's error message keeps the existing names/count framing, which is consistent with the adjacent mismatch message, and it updates the stale TODO comment accurately. Scope and craft are comparable otherwise (A touches AUTHORS, neutral). B is the more complete fix of the reported issue.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

3m 34s

Wall-clock time in the container

Full test suite

No new failures

123 passed · 1 skipped · 124 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

98vs 43 merged

Changed lines, ours vs merged

This is a regression in pytest 9.1.

@pytest.mark.parametrize("x,", [None, "foo", "bar"])  # Spurious comma after parameter name
def test1(x):
    pass

output:

________________________________________________________________ ERROR collecting test.py ________________________________________________________________
.pixi/envs/py314/lib/python3.14/site-packages/pluggy/_hooks.py:512: in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.pixi/envs/py314/lib/python3.14/site-packages/pluggy/_manager.py:120: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.pixi/envs/py314/lib/python3.14/site-packages/_pytest/python.py:250: in pytest_pycollect_makeitem
    return list(collector._genfunctions(name, obj))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.pixi/envs/py314/lib/python3.14/site-packages/_pytest/python.py:476: in _genfunctions
    self.ihook.pytest_generate_tests.call_extra(methods, dict(metafunc=metafunc))
.pixi/envs/py314/lib/python3.14/site-packages/pluggy/_hooks.py:573: in call_extra
    return self._hookexec(self.name, hookimpls, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.pixi/envs/py314/lib/python3.14/site-packages/pluggy/_manager.py:120: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.pixi/envs/py314/lib/python3.14/site-packages/_pytest/python.py:124: in pytest_generate_tests
    metafunc.parametrize(*marker.args, **marker.kwargs, _param_mark=marker)
.pixi/envs/py314/lib/python3.14/site-packages/_pytest/python.py:1316: in parametrize
    argnames, parametersets = ParameterSet._for_parametrize(
.pixi/envs/py314/lib/python3.14/site-packages/_pytest/mark/structures.py:218: in _for_parametrize
    if len(param.values) != len(argnames):
       ^^^^^^^^^^^^^^^^^
E   TypeError: object of type 'NoneType' has no len()

As there is no hint as to where the offending test is, the only way to find it is to bisect the whole module.

If the first value is a string, pytest cleanly points at the offending test, but IMHO the error message is improvable as it could better clarify that it's treating the first string as a sequence:

@pytest.mark.parametrize("x,", ["foo", "bar"])
def test1(x):
    pass

output:

test.py::test1: in "parametrize" the number of names (1):
  ['x']
must be equal to the number of values (3):
  foo

suggested output:

test.py::test1: in "parametrize" the number of names (1):
  ['x']
must be equal to the number of values (3):
  ['f', 'o', 'o']

The two patches

Our patch+94 −4
changelog/14619.bugfix.rstnew file mode 100644index 000000000..c46fb0365@@ -0,0 +1 @@+Fixed an obscure ``TypeError: object of type 'NoneType' has no len()`` crash during collection when a ``parametrize`` value is not a sequence although one is expected, for example due to a spurious trailing comma in ``@pytest.mark.parametrize("x,", [None])``. A clear error pointing at the offending test is now reported instead, and string values unpacked into their characters are now shown as such in the error message.src/_pytest/mark/structures.pyindex 635cf9d47..2afd71c35 100644@@ -159,7 +159,8 @@ class ParameterSet(NamedTuple):             return cls.param(parameterset)         else:             # TODO: Refactor to fix this type-ignore. Currently the following-            # passes type-checking but crashes:+            # passes type-checking but fails at collection time (see the+            # non-sequence check in _for_parametrize):             #             #   @pytest.mark.parametrize(('x', 'y'), [1, 2])             #   def test_foo(x, y): pass@@ -217,7 +218,30 @@ class ParameterSet(NamedTuple):         if parameters:             # Check all parameter sets have the correct number of values.             for param in parameters:-                if len(param.values) != len(argnames):+                # Typed as object: values is not guaranteed to be a sequence,+                # see extract_from.+                values: object = param.values+                if not isinstance(values, collections.abc.Sized):+                    # A non-sequence value was not wrapped into a tuple, e.g.+                    # ``parametrize("x,", [None])`` or ``parametrize(("x",), [1])``.+                    msg = (+                        '{nodeid}: in "parametrize" the number of names ({names_len}):\n'+                        "  {names}\n"+                        "must be equal to the number of values, but got a value "+                        "of type {type_name} which is not a sequence:\n"+                        "  {values!r}"+                    )+                    fail(+                        msg.format(+                            nodeid=nodeid,+                            values=values,+                            type_name=type(values).__name__,+                            names=argnames,+                            names_len=len(argnames),+                        ),+                        pytrace=False,+                    )+                if len(values) != len(argnames):                     msg = (                         '{nodeid}: in "parametrize" the number of names ({names_len}):\n'                         "  {names}\n"@@ -227,10 +251,12 @@ class ParameterSet(NamedTuple):                     fail(                         msg.format(                             nodeid=nodeid,-                            values=param.values,+                            # Show strings as the sequence of characters they+                            # were unpacked into, to make that unpacking visible.+                            values=list(values) if isinstance(values, str) else values,                             names=argnames,                             names_len=len(argnames),-                            values_len=len(param.values),+                            values_len=len(values),                         ),                         pytrace=False,                     )testing/test_mark.pyindex 706ae8120..ac64ff2bd 100644@@ -612,6 +612,69 @@ def test_parametrized_collect_with_wrong_args(pytester: Pytester) -> None:     )  +@pytest.mark.parametrize(+    ("argnames", "value_repr", "type_name"),+    [+        # Spurious trailing comma: values must be tuples (#14619).+        ('"x,"', "None", "NoneType"),+        ('"x,"', "1", "int"),+        ('("x",)', "None", "NoneType"),+        ('("x", "y")', "1", "int"),+    ],+)+def test_parametrized_collect_with_non_sequence_values(+    pytester: Pytester, argnames: str, value_repr: str, type_name: str+) -> None:+    """A value that is not a sequence when one is expected fails cleanly,+    pointing at the offending test, instead of crashing with a TypeError."""+    pytester.makepyfile(+        f"""+        import pytest++        @pytest.mark.parametrize({argnames}, [{value_repr}, ("foo",)])+        def test_func(x, y=None):+            pass+    """+    )++    result = pytester.runpytest()+    result.stdout.fnmatch_lines(+        [+            '*::test_func: in "parametrize" the number of names (*):',+            "must be equal to the number of values, but got a value of type "+            f"{type_name} which is not a sequence:",+            f"  {value_repr}",+        ]+    )+    result.stdout.no_fnmatch_line("*TypeError*")+    result.assert_outcomes(errors=1)+++def test_parametrized_collect_with_string_value_shows_unpacked_chars(+    pytester: Pytester,+) -> None:+    """A string value unpacked into its characters is shown as such (#14619)."""+    py_file = pytester.makepyfile(+        """+        import pytest++        @pytest.mark.parametrize("x,", ["foo", "bar"])+        def test_func(x):+            pass+    """+    )++    result = pytester.runpytest(py_file)+    result.stdout.fnmatch_lines(+        [+            '*::test_func: in "parametrize" the number of names (1):',+            "  ['x']",+            "must be equal to the number of values (3):",+            "  ['f', 'o', 'o']",+        ]+    )++ def test_parametrized_with_kwargs(pytester: Pytester) -> None:     """Test collect parametrized func with wrong number of args."""     py_file = pytester.makepyfile( 
Merged fix · PR #14978+41 −2
AUTHORSindex daa6a471e..ad4c20938 100644@@ -343,6 +343,7 @@ Mihail Milushev Mike Fiedler (miketheman) Mike Hoyle (hoylemd) Mike Lundy+Mike Ma Milan Lesnek minbang930 Miro Hrončokchangelog/14619.bugfix.rstnew file mode 100644index 000000000..8f7f788af@@ -0,0 +1 @@+Fixed a crash in ``pytest.mark.parametrize`` when a non-sequence value is passed for a parameter set. It now provides a clear error message during collection.src/_pytest/mark/structures.pyindex 635cf9d47..34a004f48 100644@@ -217,7 +217,20 @@ class ParameterSet(NamedTuple):         if parameters:             # Check all parameter sets have the correct number of values.             for param in parameters:-                if len(param.values) != len(argnames):+                try:+                    values_len = len(param.values)+                except TypeError:+                    values_len = None+                if values_len is None:+                    fail(+                        f'{nodeid}: in "parametrize" expected a sequence of values, '+                        "for a single value use a one-element tuple like "+                        "('value',), "+                        f"got {type(param.values).__name__}:\n"+                        f"  {param.values}",+                        pytrace=False,+                    )+                if values_len != len(argnames):                     msg = (                         '{nodeid}: in "parametrize" the number of names ({names_len}):\n'                         "  {names}\n"@@ -230,7 +243,7 @@ class ParameterSet(NamedTuple):                             values=param.values,                             names=argnames,                             names_len=len(argnames),-                            values_len=len(param.values),+                            values_len=values_len,                         ),                         pytrace=False,                     )testing/test_mark.pyindex 706ae8120..9698aca35 100644@@ -612,6 +612,30 @@ def test_parametrized_collect_with_wrong_args(pytester: Pytester) -> None:     )  +def test_parametrized_collect_with_non_sequence_values(pytester: Pytester) -> None:+    """Test collect parametrized func with tuple-style argnames and scalar values."""+    py_file = pytester.makepyfile(+        """+        import pytest++        @pytest.mark.parametrize("x,", [None])+        def test_func(x):+            pass+    """+    )++    result = pytester.runpytest(py_file)+    result.stdout.fnmatch_lines(+        [+            "test_parametrized_collect_with_non_sequence_values.py::test_func: "+            'in "parametrize" expected a sequence of values, '+            "for a single value use a one-element tuple like ('value',), "+            "got NoneType:",+            "  None",+        ]+    )++ def test_parametrized_with_kwargs(pytester: Pytester) -> None:     """Test collect parametrized func with wrong number of args."""     py_file = pytester.makepyfile(