Backtestspytestrun-20261106-234554#14619
Spurious comma in parametrize causes obfuscated crash
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Both patches fix the root crash in ParameterSet._for_parametrize by detecting a non-sized value before calling len() and reporting a clean collection error that names the offending test; A does it via try/except TypeError, B via an isinstance(Sized) check, and either works for the None/int cases in the issue (independent tests 1 and 2 pass on both). The issue has a second explicit ask: when the value is a string, show it as the unpacked character list (['f','o','o']) so the user can see it was treated as a sequence. Patch B implements this and tests it; Patch A ignores it entirely, and the independent test for it fails on A. B's tests are also stronger: a parametrized case covering None/int with both the '"x,"' and tuple-argnames forms, a no_fnmatch_line('*TypeError*') guard, and assert_outcomes(errors=1), versus A's single None case. B's error message keeps the existing names/count framing, which is consistent with the adjacent mismatch message, and it updates the stale TODO comment accurately. Scope and craft are comparable otherwise (A touches AUTHORS, neutral). B is the more complete fix of the reported issue.
Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.
Solve
3m 34s
Wall-clock time in the container
Full test suite
No new failures
123 passed · 1 skipped · 124 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
98vs 43 merged
Changed lines, ours vs merged
The issue
View on GitHubThis is a regression in pytest 9.1.
@pytest.mark.parametrize("x,", [None, "foo", "bar"]) # Spurious comma after parameter name
def test1(x):
pass
output:
________________________________________________________________ ERROR collecting test.py ________________________________________________________________
.pixi/envs/py314/lib/python3.14/site-packages/pluggy/_hooks.py:512: in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.pixi/envs/py314/lib/python3.14/site-packages/pluggy/_manager.py:120: in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.pixi/envs/py314/lib/python3.14/site-packages/_pytest/python.py:250: in pytest_pycollect_makeitem
return list(collector._genfunctions(name, obj))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.pixi/envs/py314/lib/python3.14/site-packages/_pytest/python.py:476: in _genfunctions
self.ihook.pytest_generate_tests.call_extra(methods, dict(metafunc=metafunc))
.pixi/envs/py314/lib/python3.14/site-packages/pluggy/_hooks.py:573: in call_extra
return self._hookexec(self.name, hookimpls, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.pixi/envs/py314/lib/python3.14/site-packages/pluggy/_manager.py:120: in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.pixi/envs/py314/lib/python3.14/site-packages/_pytest/python.py:124: in pytest_generate_tests
metafunc.parametrize(*marker.args, **marker.kwargs, _param_mark=marker)
.pixi/envs/py314/lib/python3.14/site-packages/_pytest/python.py:1316: in parametrize
argnames, parametersets = ParameterSet._for_parametrize(
.pixi/envs/py314/lib/python3.14/site-packages/_pytest/mark/structures.py:218: in _for_parametrize
if len(param.values) != len(argnames):
^^^^^^^^^^^^^^^^^
E TypeError: object of type 'NoneType' has no len()
As there is no hint as to where the offending test is, the only way to find it is to bisect the whole module.
If the first value is a string, pytest cleanly points at the offending test, but IMHO the error message is improvable as it could better clarify that it's treating the first string as a sequence:
@pytest.mark.parametrize("x,", ["foo", "bar"])
def test1(x):
pass
output:
test.py::test1: in "parametrize" the number of names (1):
['x']
must be equal to the number of values (3):
foo
suggested output:
test.py::test1: in "parametrize" the number of names (1):
['x']
must be equal to the number of values (3):
['f', 'o', 'o']The two patches
changelog/14619.bugfix.rstnew file mode 100644index 000000000..c46fb0365@@ -0,0 +1 @@+Fixed an obscure ``TypeError: object of type 'NoneType' has no len()`` crash during collection when a ``parametrize`` value is not a sequence although one is expected, for example due to a spurious trailing comma in ``@pytest.mark.parametrize("x,", [None])``. A clear error pointing at the offending test is now reported instead, and string values unpacked into their characters are now shown as such in the error message.src/_pytest/mark/structures.pyindex 635cf9d47..2afd71c35 100644@@ -159,7 +159,8 @@ class ParameterSet(NamedTuple): return cls.param(parameterset) else: # TODO: Refactor to fix this type-ignore. Currently the following- # passes type-checking but crashes:+ # passes type-checking but fails at collection time (see the+ # non-sequence check in _for_parametrize): # # @pytest.mark.parametrize(('x', 'y'), [1, 2]) # def test_foo(x, y): pass@@ -217,7 +218,30 @@ class ParameterSet(NamedTuple): if parameters: # Check all parameter sets have the correct number of values. for param in parameters:- if len(param.values) != len(argnames):+ # Typed as object: values is not guaranteed to be a sequence,+ # see extract_from.+ values: object = param.values+ if not isinstance(values, collections.abc.Sized):+ # A non-sequence value was not wrapped into a tuple, e.g.+ # ``parametrize("x,", [None])`` or ``parametrize(("x",), [1])``.+ msg = (+ '{nodeid}: in "parametrize" the number of names ({names_len}):\n'+ " {names}\n"+ "must be equal to the number of values, but got a value "+ "of type {type_name} which is not a sequence:\n"+ " {values!r}"+ )+ fail(+ msg.format(+ nodeid=nodeid,+ values=values,+ type_name=type(values).__name__,+ names=argnames,+ names_len=len(argnames),+ ),+ pytrace=False,+ )+ if len(values) != len(argnames): msg = ( '{nodeid}: in "parametrize" the number of names ({names_len}):\n' " {names}\n"@@ -227,10 +251,12 @@ class ParameterSet(NamedTuple): fail( msg.format( nodeid=nodeid,- values=param.values,+ # Show strings as the sequence of characters they+ # were unpacked into, to make that unpacking visible.+ values=list(values) if isinstance(values, str) else values, names=argnames, names_len=len(argnames),- values_len=len(param.values),+ values_len=len(values), ), pytrace=False, )testing/test_mark.pyindex 706ae8120..ac64ff2bd 100644@@ -612,6 +612,69 @@ def test_parametrized_collect_with_wrong_args(pytester: Pytester) -> None: ) +@pytest.mark.parametrize(+ ("argnames", "value_repr", "type_name"),+ [+ # Spurious trailing comma: values must be tuples (#14619).+ ('"x,"', "None", "NoneType"),+ ('"x,"', "1", "int"),+ ('("x",)', "None", "NoneType"),+ ('("x", "y")', "1", "int"),+ ],+)+def test_parametrized_collect_with_non_sequence_values(+ pytester: Pytester, argnames: str, value_repr: str, type_name: str+) -> None:+ """A value that is not a sequence when one is expected fails cleanly,+ pointing at the offending test, instead of crashing with a TypeError."""+ pytester.makepyfile(+ f"""+ import pytest++ @pytest.mark.parametrize({argnames}, [{value_repr}, ("foo",)])+ def test_func(x, y=None):+ pass+ """+ )++ result = pytester.runpytest()+ result.stdout.fnmatch_lines(+ [+ '*::test_func: in "parametrize" the number of names (*):',+ "must be equal to the number of values, but got a value of type "+ f"{type_name} which is not a sequence:",+ f" {value_repr}",+ ]+ )+ result.stdout.no_fnmatch_line("*TypeError*")+ result.assert_outcomes(errors=1)+++def test_parametrized_collect_with_string_value_shows_unpacked_chars(+ pytester: Pytester,+) -> None:+ """A string value unpacked into its characters is shown as such (#14619)."""+ py_file = pytester.makepyfile(+ """+ import pytest++ @pytest.mark.parametrize("x,", ["foo", "bar"])+ def test_func(x):+ pass+ """+ )++ result = pytester.runpytest(py_file)+ result.stdout.fnmatch_lines(+ [+ '*::test_func: in "parametrize" the number of names (1):',+ " ['x']",+ "must be equal to the number of values (3):",+ " ['f', 'o', 'o']",+ ]+ )++ def test_parametrized_with_kwargs(pytester: Pytester) -> None: """Test collect parametrized func with wrong number of args.""" py_file = pytester.makepyfile(
AUTHORSindex daa6a471e..ad4c20938 100644@@ -343,6 +343,7 @@ Mihail Milushev Mike Fiedler (miketheman) Mike Hoyle (hoylemd) Mike Lundy+Mike Ma Milan Lesnek minbang930 Miro Hrončokchangelog/14619.bugfix.rstnew file mode 100644index 000000000..8f7f788af@@ -0,0 +1 @@+Fixed a crash in ``pytest.mark.parametrize`` when a non-sequence value is passed for a parameter set. It now provides a clear error message during collection.src/_pytest/mark/structures.pyindex 635cf9d47..34a004f48 100644@@ -217,7 +217,20 @@ class ParameterSet(NamedTuple): if parameters: # Check all parameter sets have the correct number of values. for param in parameters:- if len(param.values) != len(argnames):+ try:+ values_len = len(param.values)+ except TypeError:+ values_len = None+ if values_len is None:+ fail(+ f'{nodeid}: in "parametrize" expected a sequence of values, '+ "for a single value use a one-element tuple like "+ "('value',), "+ f"got {type(param.values).__name__}:\n"+ f" {param.values}",+ pytrace=False,+ )+ if values_len != len(argnames): msg = ( '{nodeid}: in "parametrize" the number of names ({names_len}):\n' " {names}\n"@@ -230,7 +243,7 @@ class ParameterSet(NamedTuple): values=param.values, names=argnames, names_len=len(argnames),- values_len=len(param.values),+ values_len=values_len, ), pytrace=False, )testing/test_mark.pyindex 706ae8120..9698aca35 100644@@ -612,6 +612,30 @@ def test_parametrized_collect_with_wrong_args(pytester: Pytester) -> None: ) +def test_parametrized_collect_with_non_sequence_values(pytester: Pytester) -> None:+ """Test collect parametrized func with tuple-style argnames and scalar values."""+ py_file = pytester.makepyfile(+ """+ import pytest++ @pytest.mark.parametrize("x,", [None])+ def test_func(x):+ pass+ """+ )++ result = pytester.runpytest(py_file)+ result.stdout.fnmatch_lines(+ [+ "test_parametrized_collect_with_non_sequence_values.py::test_func: "+ 'in "parametrize" expected a sequence of values, '+ "for a single value use a one-element tuple like ('value',), "+ "got NoneType:",+ " None",+ ]+ )++ def test_parametrized_with_kwargs(pytester: Pytester) -> None: """Test collect parametrized func with wrong number of args.""" py_file = pytester.makepyfile(