Backtestspytestrun-20261106-234554#14942

Issue #14942 Merged fix PR #14984 Run 6 Nov 2026 · run-20261106-234554 pytest-dev/pytest

Test collection is 10x slower on 9.1.1 compared to 9.0.3

Better

Our patch was judged better than the merged fix.

In this judgment, Patch A = our agent, Patch B = the merged fix.

Both patches eliminate the quadratic visibility scan in `_register_fixture` and both pass all independent scaling tests. Patch A keeps the existing `_arg2fixturedefs` list and adds a per-name ancestor set so the common case (new fixturedef's node is not a strict ancestor of any existing one) appends in O(depth) without calling `is_visibility_more_specific`; the fallback to the old insertion scan when the node is an ancestor or when legacy nodeid-based defs are involved keeps the ordering semantics exactly as before. The logic is sound: if the new node is not an ancestor of any existing def's node, no existing def can be more specific, so appending preserves the partial order. A also adds a genuine regression test that counts calls to `is_visibility_more_specific` (would fail on base, roughly n^2/2 calls) plus an ordering test for the ancestor-registered-after-descendant path. Patch B goes further, re-indexing fixturedefs by node so both registration and lookup become O(depth), which is a nicer design, but it removes the `_arg2fixturedefs` attribute and `_matchfactories`, admittedly breaks pytest-bdd, and 'fixes' that by commenting out the plugin integration test in tox.ini. Knowingly breaking a widely used plugin and disabling the test that would catch it is real added risk, not neutral scope. B also changes the `getfixturedefs` return type from tuple to list and adds no test that exercises the performance regression itself (its new tests cover only the legacy nodeid fallbacks it had to re-plumb). On correctness both resolve the reported regression; on tests A is clearly stronger; on scope/risk B's redesign introduces a known compatibility break. A is the better fix.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

6m 51s

Wall-clock time in the container

Full test suite

No new failures

1,279 passed · 25 skipped · 1,324 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

131vs 223 merged

Changed lines, ours vs merged

Hello!

After upgrading to 9.1.1 from 9.0.3 (and no other code changes, only one config change), running pytest --collect-only is around 10x slower.

On 9.0.3: 30406/30495 tests collected (89 deselected) in 30.22s

On 9.1.1: 30406/30495 tests collected (89 deselected) in 303.61s (0:05:03)

The only thing that changed between the two runs was adding this to filterwarnings (as we have a lot of class based fixtures)

ignore:.*Class-scoped fixture defined as instance method.*:pytest.PytestRemovedIn10Warning

I did some profiling with pyinstrument, it seems that most of the time is now spent in is_visibility_more_specific from FixtureManager.parsefactories

The codebase has a lot of class-based tests, but I can't share the code unfortunately.

I do appreciate there is not a lot to go on here, please lmk what other info I could provide to make the debugging easier!

The two patches

Our patch+126 −5
changelog/14942.bugfix.rstnew file mode 100644index 000000000..cb3bbaef8@@ -0,0 +1 @@+Fixed a collection performance regression in pytest 9.1 when many fixtures share the same name, e.g. a fixture defined in a base class inherited by many test classes.src/_pytest/fixtures.pyindex 05537ec01..463079139 100644@@ -1792,6 +1792,17 @@ class FixtureManager:         # TODO: The order of the FixtureDefs list of each arg is significant,         #       explain.         self._arg2fixturedefs: Final[dict[str, list[FixtureDef[Any]]]] = {}+        # Maps a fixture name (argname) to the set of strict ancestors (in the+        # collection tree) of the nodes of all of its node-based FixtureDefs.+        # Used to quickly determine whether a newly registered FixtureDef has+        # any existing FixtureDef with more specific visibility, which allows to+        # avoid a quadratic scan of the FixtureDefs list in the common case.+        self._arg2fixturedef_ancestors: Final[dict[str, set[nodes.Node]]] = {}+        # Fixture names which have at least one FixtureDef registered with a+        # legacy string nodeid rather than a node. For these, the fast path+        # based on ``_arg2fixturedef_ancestors`` cannot be used.+        # This can be removed once baseid deprecation is done (pytest 10).+        self._legacy_visibility_argnames: Final[set[str]] = set()         # A mapping from a node to a list of autouse fixture names it defines.         # The Session entry holds global usefixtures from config.         self._node_autousenames: Final[dict[nodes.Node, list[str]]] = {@@ -2107,12 +2118,38 @@ class FixtureManager:         # cannot be visible together.         # The idea is that a fixture that is defined closer to the item should         # take precedence.-        for i, existing in enumerate(faclist):-            if is_visibility_more_specific(existing, fixture_def):-                faclist.insert(i, fixture_def)-                break-        else:+        #+        # Comparing visibility requires walking the collection tree, so to avoid+        # quadratic behavior when many fixturedefs share a name (e.g. a base+        # class fixture inherited by many test classes), first check whether+        # any existing fixturedef can be more specific at all: this is only+        # possible if the new fixturedef's node is a strict ancestor of an+        # existing fixturedef's node. Usually it is not, since parents are+        # collected before their children, and we can simply append.+        ancestors = self._arg2fixturedef_ancestors.setdefault(name, set())+        if fixture_def.node is None:+            self._legacy_visibility_argnames.add(name)+        if (+            fixture_def.node is not None+            and name not in self._legacy_visibility_argnames+            and fixture_def.node not in ancestors+        ):             faclist.append(fixture_def)+        else:+            for i, existing in enumerate(faclist):+                if is_visibility_more_specific(existing, fixture_def):+                    faclist.insert(i, fixture_def)+                    break+            else:+                faclist.append(fixture_def)+        if fixture_def.node is not None:+            for parent in fixture_def.node.iter_parents():+                if parent is fixture_def.node:+                    continue+                if parent in ancestors:+                    # All further parents were already added.+                    break+                ancestors.add(parent)         if autouse:             if node is not NOTSET:                 self._node_autousenames.setdefault(node, []).append(name)testing/python/fixtures.pyindex c0b499481..bcd5a2666 100644@@ -1950,6 +1950,89 @@ class TestFixtureManagerParseFactories:         reprec = pytester.inline_run()         reprec.assertoutcome(passed=1) +    def test_register_fixture_ordered_by_visibility_node_only(+        self, pytester: Pytester+    ) -> None:+        """Like test_register_fixture_ordered_by_visibility, but with only+        node-based fixturedefs, where an ancestor's fixturedef is registered+        after its descendants' fixturedefs (#14942)."""+        pytester.makeconftest(+            """+            import pytest++            @pytest.hookimpl(wrapper=True)+            def pytest_collection(session):+                result = yield+                fm = session._fixturemanager+                item = session.items[0]+                module = item.parent+                fm._register_fixture(name="fix", func=lambda fix: f"item-{fix}", node=item)+                fm._register_fixture(name="fix", func=lambda fix: f"module-{fix}", node=module)+                fm._register_fixture(name="fix", func=lambda: "session", node=session)+                return result+            """+        )+        pytester.makepyfile(+            """+            def test(fix):+                assert fix == "item-module-session"+            """+        )+        reprec = pytester.inline_run()+        reprec.assertoutcome(passed=1)++    def test_register_many_same_name_fixtures_not_quadratic(+        self, pytester: Pytester, monkeypatch: MonkeyPatch+    ) -> None:+        """Registering many fixtures with the same name in non-comparable+        nodes (e.g. a base class fixture inherited by many test classes) should+        not compare visibility against every existing fixturedef (#14942)."""+        import _pytest.fixtures++        calls = 0+        orig = _pytest.fixtures.is_visibility_more_specific++        def counting(candidate, other) -> bool:+            nonlocal calls+            calls += 1+            return orig(candidate, other)++        monkeypatch.setattr(_pytest.fixtures, "is_visibility_more_specific", counting)++        num_classes = 200+        pytester.makeconftest(+            """+            import pytest++            @pytest.fixture+            def fix():+                return "conftest"+            """+        )+        source = textwrap.dedent(+            """+            import pytest++            class Base:+                @pytest.fixture+                def fix(self, fix):+                    return "base-" + fix+            """+        )+        for i in range(num_classes):+            source += textwrap.dedent(+                f"""+                class TestSub{i}(Base):+                    def test(self, fix):+                        assert fix == "base-conftest"+                """+            )+        pytester.makepyfile(source)+        reprec = pytester.inline_run()+        reprec.assertoutcome(passed=num_classes)+        # Without the fast path this is in the order of num_classes**2 / 2.+        assert calls < num_classes+     def test_parsefactories_relative_node_ids(         self, pytester: Pytester, monkeypatch: MonkeyPatch     ) -> None: 
Merged fix · PR #14984+150 −73
changelog/14942.bugfix.rstnew file mode 100644index 000000000..bc82d6038@@ -0,0 +1,2 @@+Fixed a regression in 9.1.0 that made collection very slow (quadratic) when many fixtures are defined with the same name.+This can happen particularly when many tests are defined in separate classes which inherit from a base class which defines fixtures (the fixtures are repeated per class).src/_pytest/fixtures.pyindex 05537ec01..7656fca2f 100644@@ -473,9 +473,9 @@ class FuncFixtureInfo:     # Note: can't include dynamic dependencies (`request.getfixturevalue` calls).     names_closure: list[str]     # A map from a fixture name in the transitive closure to the FixtureDefs-    # matching the name which are applicable to this function.+    # matching the name which are visible to this item.     # There may be multiple overriding fixtures with the same name. The-    # sequence is ordered from furthest to closes to the function.+    # sequence is ordered from furthest to closes to the item.     name2fixturedefs: dict[str, Sequence[FixtureDef[Any]]]      def prune_dependency_tree(self) -> None:@@ -726,7 +726,7 @@ class FixtureRequest(abc.ABC):         # No fixtures defined with this name.         if fixturedefs is None:             raise FixtureLookupError(argname, self)-        # The are no fixtures with this name applicable for the function.+        # The are no fixtures with this name visible for the item.         if not fixturedefs:             raise FixtureLookupError(argname, self) @@ -999,10 +999,8 @@ class FixtureLookupError(LookupError):             available = set()             parent = self.request._pyfuncitem.parent             assert parent is not None-            for name, fixturedefs in fm._arg2fixturedefs.items():-                faclist = list(fm._matchfactories(fixturedefs, parent))-                if faclist:-                    available.add(name)+            for fixturedef in fm._get_all_fixture_defs_for_node(parent):+                available.add(fixturedef.argname)             if self.argname in available:                 msg = (                     f" recursive dependency involving fixture '{self.argname}' detected"@@ -1788,10 +1786,20 @@ class FixtureManager:         self.session = session         self.config: Config = session.config         # Maps a fixture name (argname) to all of the FixtureDefs in the test-        # suite/plugins defined with this name. Populated by parsefactories().-        # TODO: The order of the FixtureDefs list of each arg is significant,-        #       explain.-        self._arg2fixturedefs: Final[dict[str, list[FixtureDef[Any]]]] = {}+        # suite/plugins defined with this name.+        # For each name, there is a mapping from a Node to the fixtures with the+        # name registered under that Node. The node determines the FixtureDef's+        # visibility (equal to the fixturedef.node).+        # Populated by parsefactories().+        self._arg2node2fixturedefs: Final[+            dict[str, dict[nodes.Node, list[FixtureDef[Any]]]]+        ] = {}+        # Legacy fallback, for plugins still using the deprecated nodeid-based+        # API without a node reference.+        # Part of FIXTURE_NODEID_DEPRECATED deprecation.+        self._arg2nodeid2fixturedefs: Final[+            dict[str, dict[str, list[FixtureDef[Any]]]]+        ] = {}         # A mapping from a node to a list of autouse fixture names it defines.         # The Session entry holds global usefixtures from config.         self._node_autousenames: Final[dict[nodes.Node, list[str]]] = {@@ -1799,6 +1807,7 @@ class FixtureManager:         }         # Legacy fallback: nodeid string -> autouse names, for plugins still         # using the deprecated nodeid-based API without a node reference.+        # Part of FIXTURE_NODEID_DEPRECATED deprecation.         self._nodeid_autousenames: Final[dict[str, list[str]]] = {}         # Pending conftest modules waiting to be parsed when their Directory is collected.         # Maps directory path -> conftest plugin module.@@ -1928,7 +1937,7 @@ class FixtureManager:         self._pending_conftests.clear()      def _getautousenames(self, node: nodes.Node) -> Iterator[str]:-        """Return the names of autouse fixtures applicable to node."""+        """Return the names of autouse fixtures visible to node."""         for parentnode in node.listchain():             basenames = self._node_autousenames.get(parentnode)             if basenames:@@ -1939,7 +1948,7 @@ class FixtureManager:                 yield from nodeid_basenames      def _getusefixturesnames(self, node: nodes.Item) -> Iterator[str]:-        """Return the names of usefixtures fixtures applicable to node."""+        """Return the names of usefixtures fixtures visible to node."""         for marker_node, mark in node.iter_markers_with_node(name="usefixtures"):             if not mark.args:                 marker_node.warn(@@ -2095,24 +2104,16 @@ class FixtureManager:             node=node,         ) -        faclist = self._arg2fixturedefs.setdefault(name, [])-        # Insert the fixturedef into the list while maintaining a partial order-        # based on visibility: a fixturedef whose visibility is more specific-        # sorts after a more general one, so that it takes precedence in the-        # override chain (the last applicable fixturedef in the list is used-        # first, see getfixturedefs).-        # fixturedefs with the same visibility keep registration order, i.e. the-        # last registered wins.-        # The order between non-comparable fixturedefs doesn't matter since they-        # cannot be visible together.-        # The idea is that a fixture that is defined closer to the item should-        # take precedence.-        for i, existing in enumerate(faclist):-            if is_visibility_more_specific(existing, fixture_def):-                faclist.insert(i, fixture_def)-                break+        if node is not NOTSET:+            node2fixturedefs = self._arg2node2fixturedefs.setdefault(name, {})+            node2fixturedefs.setdefault(node, []).insert(0, fixture_def)+        elif nodeid is not NOTSET and nodeid is not None:+            nodeid2fixturedefs = self._arg2nodeid2fixturedefs.setdefault(name, {})+            nodeid2fixturedefs.setdefault(nodeid, []).insert(0, fixture_def)         else:-            faclist.append(fixture_def)+            # Global plugin autouse fixtures go under Session.+            node2fixturedefs = self._arg2node2fixturedefs.setdefault(name, {})+            node2fixturedefs.setdefault(self.session, []).insert(0, fixture_def)         if autouse:             if node is not NOTSET:                 self._node_autousenames.setdefault(node, []).append(name)@@ -2338,41 +2339,64 @@ class FixtureManager:                     nodeid=effective_nodeid,                 ) +    def _get_all_fixture_defs(self) -> Iterable[FixtureDef[Any]]:+        """Get all FixtureDefs.++        The order is not guaranteed.+        """+        for node2fixturedefs in self._arg2node2fixturedefs.values():+            for fixturedefs in node2fixturedefs.values():+                yield from fixturedefs+        for nodeid2fixturedefs in self._arg2nodeid2fixturedefs.values():+            for fixturedefs in nodeid2fixturedefs.values():+                yield from fixturedefs++    def _get_all_fixture_defs_for_node(+        self, node: nodes.Node+    ) -> Iterable[FixtureDef[Any]]:+        """Get all FixtureDefs visible to a node.++        The order is not guaranteed.+        """+        for node2fixturedefs in self._arg2node2fixturedefs.values():+            for parent in node.iter_parents():+                yield from node2fixturedefs.get(parent, ())+        for nodeid2fixturedefs in self._arg2nodeid2fixturedefs.values():+            for parent in node.iter_parents():+                yield from nodeid2fixturedefs.get(parent.nodeid, ())+     def getfixturedefs(         self, argname: str, node: nodes.Node     ) -> Sequence[FixtureDef[Any]] | None:-        """Get FixtureDefs for a fixture name which are applicable+        """Get FixtureDefs for a fixture name which are visible         to a given node.          Returns None if there are no fixtures at all defined with the given         name. (This is different from the case in which there are fixtures-        with the given name, but none applicable to the node. In this case,+        with the given name, but none visible to the node. In this case,         an empty result is returned). +        The returned FixtureDefs are ordered from least specific (registered+        higher in the collection tree) to most specific. For FixtureDefs+        registered at the same Node, registered later => more specific.+         :param argname: Name of the fixture to search for.         :param node: The requesting Node.         """-        try:-            fixturedefs = self._arg2fixturedefs[argname]-        except KeyError:+        node2fixturedefs = self._arg2node2fixturedefs.get(argname, {})+        nodeid2fixturedefs = self._arg2nodeid2fixturedefs.get(argname, {})+        if not node2fixturedefs and not nodeid2fixturedefs:             return None-        return tuple(self._matchfactories(fixturedefs, node))--    def _matchfactories(-        self, fixturedefs: Iterable[FixtureDef[Any]], node: nodes.Node-    ) -> Iterator[FixtureDef[Any]]:-        # Collect parent nodes and their IDs for matching-        parent_nodes = set(node.iter_parents())-        parentnodeids = {n.nodeid for n in parent_nodes}--        for fixturedef in fixturedefs:-            if fixturedef.node is not None:-                # Node-based matching: check if fixture's node is a parent-                if fixturedef.node in parent_nodes:-                    yield fixturedef-            elif fixturedef.baseid in parentnodeids:-                # Fallback to string-based matching for legacy/plugins-                yield fixturedef+        fixturedefs = [+            fixturedef+            for parent in node.iter_parents()+            for fixturedef in [+                *node2fixturedefs.get(parent, ()),+                *nodeid2fixturedefs.get(parent.nodeid, ()),+            ]+        ]+        fixturedefs.reverse()+        return fixturedefs   def show_fixtures_per_test(config: Config) -> int | ExitCode:@@ -2487,30 +2511,24 @@ def _showfixtures_main(config: Config, session: Session) -> None:     verbose = config.get_verbosity()      fm = session._fixturemanager-     available = []     seen: set[tuple[str, str]] = set()--    for argname, fixturedefs in fm._arg2fixturedefs.items():-        assert fixturedefs is not None-        if not fixturedefs:+    for fixturedef in fm._get_all_fixture_defs():+        loc = getlocation(fixturedef.func, invocation_dir)+        if (fixturedef.argname, loc) in seen:             continue-        for fixturedef in fixturedefs:-            loc = getlocation(fixturedef.func, invocation_dir)-            if (fixturedef.argname, loc) in seen:-                continue-            seen.add((fixturedef.argname, loc))-            available.append(-                (-                    len(fixturedef.baseid),-                    fixturedef.func.__module__,-                    _pretty_fixture_path(invocation_dir, fixturedef.func),-                    fixturedef.argname,-                    fixturedef,-                )+        seen.add((fixturedef.argname, loc))+        available.append(+            (+                len(fixturedef.baseid),+                fixturedef.func.__module__,+                _pretty_fixture_path(invocation_dir, fixturedef.func),+                fixturedef.argname,+                fixturedef,             )-+        )     available.sort()+     currentmodule = None     for baseid, module, prettypath, argname, fixturedef in available:         if currentmodule != module:testing/deprecated_test.pyindex 96c6fe61d..ee91880d2 100644@@ -323,7 +323,7 @@ class TestFixtureNodeidDeprecations:                 defs = request.session._fixturemanager.getfixturedefs(                     "scoped_legacy", request._pyfuncitem                 )-                assert defs == ()+                assert defs == []             """         )         result = pytester.runpytest("-W", "ignore::pytest.PytestRemovedIn10Warning")testing/python/fixtures.pyindex c0b499481..8e779a93d 100644@@ -146,6 +146,37 @@ class TestFillFixtures:             """         ) +    def test_fixture_not_found_nodeid_fallback(self, pytester: Pytester) -> None:+        """Test for fallback string nodeid handling in fixture not found error.++        This test can be deleted with FIXTURE_NODEID_DEPRECATED deprecation.+        """+        pytester.makeconftest(+            """+            import pytest++            def pytest_collection_finish(session):+                session._fixturemanager._register_fixture(+                    name="does_exist",+                    func=lambda: 0,+                    nodeid="",+                )+            """+        )+        pytester.makepyfile(+            """+            def test_it(does_not_exist): pass+            """+        )+        result = pytester.runpytest()+        assert result.ret == ExitCode.TESTS_FAILED+        result.stdout.fnmatch_lines(+            [+                "*fixture 'does_not_exist' not found*",+                "*available fixtures: *does_exist*",+            ]+        )+     def test_detect_recursive_dependency_error(self, pytester: Pytester) -> None:         pytester.copy_example()         result = pytester.runpytest()@@ -4095,6 +4126,31 @@ class TestShowFixtures:             def foo():                 raise NotImplementedError() +    def test_show_fixtures_deprecated_nodeid_fixture(self, pytester: Pytester) -> None:+        """Test for fallback string nodeid handling in showfixtures.++        This test can be deleted with FIXTURE_NODEID_DEPRECATED deprecation.+        """+        pytester.makeconftest(+            """+            import pytest++            def pytest_collection_finish(session):+                session._fixturemanager._register_fixture(+                    name="does_exist",+                    func=lambda: 0,+                    nodeid="",+                )+            """+        )++        result = pytester.runpytest("--fixtures")+        result.stdout.fnmatch_lines(+            [+                "*does_exist -- conftest.py:*",+            ]+        )+  class TestContextManagerFixtureFuncs:     def test_simple(self, pytester: Pytester) -> None:tox.iniindex e831bde84..75d22d6e2 100644@@ -187,7 +187,8 @@ setenv =     PYTHONPATH=. commands =     uv pip check-    pytest bdd_wallet.py+    ; Currently broken by _arg2fixturedefs -> _arg2node2fixturedefs change.+    ; pytest bdd_wallet.py     pytest --cov=. simple_integration.py     pytest --ds=django_settings simple_integration.py     pytest --html=simple.html simple_integration.py