Backtestspytestrun-20261106-234554#14036
ExceptionGroups do not respect __tracebackhide__
Our patch was judged better than the merged fix.
In this judgment, Patch A = our agent, Patch B = the merged fix.
Both patches pass every independent test, but the structural differences favor A on correctness. Patch A builds the group's native traceback from the already-pruned `traceback` (via `_build_raw_traceback`), so the cut performed by `Function._traceback_filter` (dropping pytest runner/pluggy frames above the test function) is preserved, exactly as the old `traceback[0]._rawentry` approach did. Patch B instead calls `TracebackException.from_exception(excinfo.value)`, which starts from the raw `__traceback__` of the exception, and then filters it against a fresh `ExceptionInfo.from_exception(e)` whose traceback is also un-cut and only `__tracebackhide__`-filtered. The computed `traceback` variable is used only for `extraline`. That means the group's own stack in B will include pytest-internal frames (runner `from_call`, pluggy callers, `pytest_pyfunc_call`) that the expected output in the issue does not show; the independent tests only assert absence of f1/f2/f3/g1 and presence of the test frame, so they would not catch this regression. B also keeps using `excinfo` rather than the per-chain `excinfo_`, so when a group is the cause of another exception it renders the outer exception again (A explicitly fixes that and tests it), and B's `zip(..., strict=True)` plus `(filename, lineno)` string matching are more fragile (e.g. `Code.path` normalization on Windows could drop frames entirely, while A matches by position with a length guard for `sys.tracebacklimit`). On tests, A is stronger: it parametrizes over long/short/auto/full-trace, asserts the exact native layout consecutively, checks that --full-trace still shows everything, and adds a chain/duplication test. B has a single --tb=short test with loose fnmatch checks. Scope and craft: A's extra chain fix is tightly related to the same code path and documented in the changelog; its cycle-safe traversal with a visited set and the tracebacklimit guard show care at the edges. B is more compact but its redesign of how the group stack is obtained introduces the internal-frame regression. A wins.
Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.
Solve
9m 24s
Wall-clock time in the container
Full test suite
No new failures
299 passed · 13 skipped · 312 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
244vs 100 merged
Changed lines, ours vs merged
The issue
View on GitHubrepro:
def g1():
__tracebackhide__ = True
str.does_not_exist
def f3():
__tracebackhide__ = True
1 / 0
def f2():
__tracebackhide__ = True
exc = None
try:
f3()
except Exception as e:
exc = e
exc2 = None
try:
g1()
except Exception as e:
exc2 = e
raise ExceptionGroup("blah", [exc, exc2])
def f1():
__tracebackhide__ = True
f2()
def test():
f1()
actual output:
____________________________________ test _____________________________________
+ Exception Group Traceback (most recent call last):
| File "/home/h/Git/pytest/hide_tb/foo.py", line 33, in test
| f1()
| ~~^^
| File "/home/h/Git/pytest/hide_tb/foo.py", line 30, in f1
| f2()
| ~~^^
| File "/home/h/Git/pytest/hide_tb/foo.py", line 25, in f2
| raise ExceptionGroup("blah", [exc, exc2])
| ExceptionGroup: blah (2 sub-exceptions)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/home/h/Git/pytest/hide_tb/foo.py", line 17, in f2
| f3()
| ~~^^
| File "/home/h/Git/pytest/hide_tb/foo.py", line 11, in f3
| f4()
| ~~^^
| File "/home/h/Git/pytest/hide_tb/foo.py", line 7, in f4
| 1 / 0
| ~~^~~
| ZeroDivisionError: division by zero
+---------------- 2 ----------------
| Traceback (most recent call last):
| File "/home/h/Git/pytest/hide_tb/foo.py", line 22, in f2
| g1()
| ~~^^
| File "/home/h/Git/pytest/hide_tb/foo.py", line 3, in g1
| str.does_not_exist
| AttributeError: type object 'str' has no attribute 'does_not_exist'
+------------------------------------
expected output
____________________________________ test _____________________________________
+ Exception Group Traceback (most recent call last):
| File "/home/h/Git/pytest/hide_tb/foo.py", line 33, in test
| f1()
| ~~^^
| ExceptionGroup: blah (2 sub-exceptions)
+-+---------------- 1 ----------------
| ZeroDivisionError: division by zero
+---------------- 2 ----------------
| AttributeError: type object 'str' has no attribute 'does_not_exist'
+------------------------------------
This is as much a feature request as it is a bug - the logic has simply not been implemented given that exception groups always fall back to displaying using tbstyle "native". See #9159
Designing the output of exception groups to be as pretty as the other "tbstyle"s would require a lot of design decisions, but a minimal implementation could keep the look of "native" except add features such as traceback filtering.
The two patches
changelog/14036.bugfix.rstnew file mode 100644index 000000000..2e03a0bda@@ -0,0 +1,3 @@+Frames hidden with ``__tracebackhide__`` are now also hidden in the tracebacks of exception groups, including the tracebacks of their sub-exceptions and of the exceptions chained to them -- previously only the leading hidden frames of the group itself were removed.++Also, an exception group which is the cause or context of another exception no longer causes that other exception to be shown twice.src/_pytest/_code/code.pyindex fee4468cf..03f7187b9 100644@@ -17,6 +17,8 @@ from traceback import extract_tb from traceback import format_exception from traceback import format_exception_only from traceback import FrameSummary+from traceback import StackSummary+from traceback import TracebackException from types import CodeType from types import FrameType from types import TracebackType@@ -1200,6 +1202,27 @@ class ExceptionInfoFormatter: return traceback, extraline + def _format_exception_group_native(+ self, excinfo: ExceptionInfo[BaseException], traceback: Traceback+ ) -> list[str]:+ """Format an exception group (with its sub-exceptions and chained+ exceptions) in native style, honoring traceback filtering.++ The group's own frames are limited to the already filtered+ ``traceback``. Unless filtering is disabled, frames which set+ ``__tracebackhide__`` are also removed from the tracebacks of the+ sub-exceptions and of the chained exceptions rendered along with them.+ """+ tb_exc = TracebackException(+ excinfo.type,+ excinfo.value,+ _build_raw_traceback(traceback),+ compact=True,+ )+ if self.tbfilter is not False:+ _hide_frames_of_related_exceptions(tb_exc, excinfo.value)+ return list(tb_exc.format())+ def repr_excinfo(self, excinfo: ExceptionInfo[BaseException]) -> ExceptionChainRepr: repr_chain: list[tuple[ReprTraceback, ReprFileLocation | None, str | None]] = [] e: BaseException | None = excinfo.value@@ -1215,19 +1238,14 @@ class ExceptionInfoFormatter: # See https://github.com/pytest-dev/pytest/issues/9159 reprtraceback: ReprTraceback | ReprTracebackNative if isinstance(e, BaseExceptionGroup):- # don't filter any sub-exceptions since they shouldn't have any internal frames- traceback = filter_excinfo_traceback(self.tbfilter, excinfo)+ traceback = filter_excinfo_traceback(self.tbfilter, excinfo_) extraline = ( "All traceback entries are hidden. Pass `--full-trace` to see hidden and internal frames." if not traceback else None ) reprtraceback = ReprTracebackNative(- format_exception(- type(excinfo.value),- excinfo.value,- traceback[0]._rawentry if traceback else None,- ),+ self._format_exception_group_native(excinfo_, traceback), extraline=extraline, ) @@ -1635,6 +1653,62 @@ def filter_traceback(entry: TracebackEntry) -> bool: return True +def _build_raw_traceback(traceback: Traceback) -> TracebackType | None:+ """Build a raw traceback object chaining exactly the given entries."""+ tb: TracebackType | None = None+ for entry in reversed(traceback):+ raw = entry._rawentry+ tb = TracebackType(tb, raw.tb_frame, raw.tb_lasti, raw.tb_lineno)+ return tb+++def _filter_hidden_stack(stack: StackSummary, exc: BaseException) -> StackSummary:+ """Remove the frames of ``exc``'s traceback which are hidden by+ ``__tracebackhide__`` from ``stack``, its native summary."""+ if exc.__traceback__ is None:+ return stack+ excinfo = ExceptionInfo.from_exception(exc)+ entries = excinfo.traceback+ if len(entries) != len(stack):+ # The summary was truncated by ``sys.tracebacklimit``, so its frames+ # can't be matched reliably with the traceback entries.+ return stack+ return StackSummary.from_list(+ [+ frame_summary+ for frame_summary, entry in zip(stack, entries)+ if not entry.ishidden(excinfo)+ ]+ )+++def _hide_frames_of_related_exceptions(+ tb_exc: TracebackException, exc: BaseException+) -> None:+ """Remove hidden frames from the native summaries of all sub-exceptions+ and chained exceptions reachable from ``tb_exc`` (whose own stack is+ left untouched)."""+ pending: list[tuple[TracebackException, BaseException]] = [(tb_exc, exc)]+ visited: set[int] = {id(tb_exc)}+ while pending:+ parent_te, parent = pending.pop()+ related: list[tuple[TracebackException | None, BaseException | None]] = [+ (parent_te.__cause__, parent.__cause__),+ (parent_te.__context__, parent.__context__),+ ]+ if isinstance(parent, BaseExceptionGroup):+ # ``exceptions`` is missing when the ``exceptiongroup`` backport+ # did not patch ``TracebackException`` (Python < 3.11).+ sub_tes = getattr(parent_te, "exceptions", None) or []+ related.extend(zip(sub_tes, parent.exceptions))+ for child_te, child in related:+ if child_te is None or child is None or id(child_te) in visited:+ continue+ visited.add(id(child_te))+ child_te.stack = _filter_hidden_stack(child_te.stack, child)+ pending.append((child_te, child))++ def filter_excinfo_traceback( tbfilter: TracebackFilter, excinfo: ExceptionInfo[BaseException] ) -> Traceback:testing/code/test_excinfo.pyindex f4cc6d7e7..f34745379 100644@@ -2167,6 +2167,159 @@ def test_hidden_entries_of_chained_exceptions_are_not_shown(pytester: Pytester) ) +@pytest.mark.parametrize("tbstyle", ("long", "short", "auto", "full-trace"))+def test_hidden_entries_of_exception_groups_are_not_shown(+ pytester: Pytester, tbstyle: str+) -> None:+ """Hidden entries of exception groups and their sub-exceptions are not+ shown, unless --full-trace is given."""+ p = pytester.makepyfile(+ """+ import sys+ if sys.version_info < (3, 11):+ from exceptiongroup import ExceptionGroup++ def g1():+ __tracebackhide__ = True+ str.does_not_exist++ def f3():+ __tracebackhide__ = True+ 1 / 0++ def f2():+ __tracebackhide__ = True+ exc = None+ try:+ f3()+ except Exception as e:+ exc = e+ exc2 = None+ try:+ g1()+ except Exception as e:+ exc2 = e+ raise ExceptionGroup("blah", [exc, exc2])++ def f1():+ __tracebackhide__ = True+ f2()++ def test():+ f1()+ """+ )+ if tbstyle == "full-trace":+ result = pytester.runpytest(str(p), "--full-trace")+ else:+ result = pytester.runpytest(str(p), f"--tb={tbstyle}")+ assert result.ret == 1+ result.stdout.re_match_lines(+ [+ r" \+ Exception Group Traceback \(most recent call last\):",+ r' \| File ".*", line 32, in test',+ r" \| f1\(\)",+ ]+ )+ if tbstyle == "full-trace":+ # Nothing is hidden with --full-trace.+ result.stdout.fnmatch_lines(+ [+ "*, line 29, in f1",+ "*, line 25, in f2",+ "*ExceptionGroup: blah (2 sub-exceptions)",+ "*, line 17, in f2",+ "*, line 11, in f3",+ "*ZeroDivisionError: division by zero",+ "*, line 22, in f2",+ "*, line 7, in g1",+ "*AttributeError:*'does_not_exist'",+ ]+ )+ else:+ result.stdout.re_match_lines(+ [+ r" \| (exceptiongroup\.)?ExceptionGroup: blah \(2 sub-exceptions\)",+ r" \+-\+---------------- 1 ----------------",+ r" \| ZeroDivisionError: division by zero",+ r" \+---------------- 2 ----------------",+ r" \| AttributeError: .*'does_not_exist'",+ ],+ consecutive=True,+ )+ # All frames of the sub-exceptions are hidden.+ result.stdout.no_fnmatch_line("*| Traceback (most recent call last):")+ for name in ("f1", "f2", "f3", "g1"):+ result.stdout.no_fnmatch_line(f"*, line *, in {name}")+++def test_hidden_entries_of_exception_group_chains_are_not_shown(+ pytester: Pytester,+) -> None:+ """Hidden entries are also filtered from the exceptions chained to an+ exception group and to its sub-exceptions, and an exception group which+ is the cause of another exception is shown only once."""+ p = pytester.makepyfile(+ """+ import sys+ if sys.version_info < (3, 11):+ from exceptiongroup import ExceptionGroup++ def hidden_root():+ __tracebackhide__ = True+ raise KeyError("root")++ def visible_sub():+ try:+ hidden_root()+ except KeyError as e:+ raise ValueError("sub") from e++ def hidden_group():+ __tracebackhide__ = True+ try:+ visible_sub()+ except ValueError as e:+ raise ExceptionGroup("group", [e])++ def test():+ try:+ hidden_group()+ except ExceptionGroup as e:+ raise IndexError("outer") from e+ """+ )+ result = pytester.runpytest(str(p), "--tb=short")+ assert result.ret == 1+ result.stdout.re_match_lines(+ [+ r" \+ Exception Group Traceback \(most recent call last\):",+ r' \| File ".*", line 24, in test',+ r" \| hidden_group\(\)",+ r" \| (exceptiongroup\.)?ExceptionGroup: group \(1 sub-exception\)",+ r" \+-\+---------------- 1 ----------------",+ r" \| Traceback \(most recent call last\):",+ r' \| File ".*", line 11, in visible_sub',+ r" \| hidden_root\(\)",+ r" \| KeyError: 'root'",+ r" \| The above exception was the direct cause of the following exception:",+ r" \| Traceback \(most recent call last\):",+ r' \| File ".*", line 13, in visible_sub',+ r' \| raise ValueError\("sub"\) from e',+ r" \| ValueError: sub",+ r" \+------------------------------------",+ r"The above exception was the direct cause of the following exception:",+ r".*test_hidden_entries_of_exception_group_chains_are_not_shown.py:26: in test",+ r' raise IndexError\("outer"\) from e',+ r"E IndexError: outer",+ ]+ )+ result.stdout.no_fnmatch_line("*, line *, in hidden_root")+ result.stdout.no_fnmatch_line("*, line *, in hidden_group")+ # The outer exception is not rendered a second time along with the group.+ assert result.stdout.str().count('raise IndexError("outer") from e') == 1++ def add_note(err: BaseException, msg: str) -> None: """Adds a note to an exception inplace.""" if sys.version_info < (3, 11):
changelog/14036.bugfix.rstnew file mode 100644index 000000000..2c82fe383@@ -0,0 +1 @@+``__tracebackhide__`` is now respected in ``ExceptionGroup`` tracebacks, both for the group itself and for its sub-exceptions.src/_pytest/_code/code.pyindex fee4468cf..e7712c48b 100644@@ -17,6 +17,8 @@ from traceback import extract_tb from traceback import format_exception from traceback import format_exception_only from traceback import FrameSummary+from traceback import StackSummary+from traceback import TracebackException from types import CodeType from types import FrameType from types import TracebackType@@ -1215,19 +1217,16 @@ class ExceptionInfoFormatter: # See https://github.com/pytest-dev/pytest/issues/9159 reprtraceback: ReprTraceback | ReprTracebackNative if isinstance(e, BaseExceptionGroup):- # don't filter any sub-exceptions since they shouldn't have any internal frames traceback = filter_excinfo_traceback(self.tbfilter, excinfo) extraline = ( "All traceback entries are hidden. Pass `--full-trace` to see hidden and internal frames." if not traceback else None )+ tb_exc = TracebackException.from_exception(excinfo.value)+ _filter_tracebackexception(tb_exc, excinfo.value, self.tbfilter) reprtraceback = ReprTracebackNative(- format_exception(- type(excinfo.value),- excinfo.value,- traceback[0]._rawentry if traceback else None,- ),+ list(tb_exc.format()), extraline=extraline, ) @@ -1645,3 +1644,38 @@ def filter_excinfo_traceback( return excinfo.traceback.filter(excinfo) else: return excinfo.traceback+++def _filter_tracebackexception(+ tb_exc: TracebackException,+ e: BaseException,+ tbfilter: TracebackFilter,+) -> None:+ """Filter a ``TracebackException`` in-place, respecting ``__tracebackhide__``.++ This is used to filter native-style tracebacks (currently the only style+ used for ``BaseExceptionGroup``) without mutating the original exception+ objects. It recurses into exception group sub-exceptions and into+ ``__cause__`` / ``__context__`` chains.++ Frames are matched by ``(filename, lineno)``: ``TracebackEntry._rawentry.tb_lineno``+ is 1-based absolute, matching ``FrameSummary.lineno``.+ """+ if e.__traceback__ is not None:+ excinfo = ExceptionInfo.from_exception(e)+ filtered = filter_excinfo_traceback(tbfilter, excinfo)+ kept = {+ (str(entry.frame.code.path), entry._rawentry.tb_lineno)+ for entry in filtered+ }+ tb_exc.stack = StackSummary.from_list(+ [fs for fs in tb_exc.stack if (fs.filename, fs.lineno) in kept]+ )+ if isinstance(e, BaseExceptionGroup):+ sub_tb_excs = getattr(tb_exc, "exceptions", None) or []+ for sub_tb_exc, sub_e in zip(sub_tb_excs, e.exceptions, strict=True):+ _filter_tracebackexception(sub_tb_exc, sub_e, tbfilter)+ if tb_exc.__cause__ is not None and e.__cause__ is not None:+ _filter_tracebackexception(tb_exc.__cause__, e.__cause__, tbfilter)+ if tb_exc.__context__ is not None and e.__context__ is not None:+ _filter_tracebackexception(tb_exc.__context__, e.__context__, tbfilter)testing/code/test_excinfo.pyindex f4cc6d7e7..ec9f584df 100644@@ -2167,6 +2167,59 @@ def test_hidden_entries_of_chained_exceptions_are_not_shown(pytester: Pytester) ) +def test_tracebackhide_in_exceptiongroup_is_respected(pytester: Pytester) -> None:+ """ExceptionGroup tracebacks respect __tracebackhide__ (#14036)."""+ p = pytester.makepyfile(+ """+ import sys+ if sys.version_info < (3, 11):+ from exceptiongroup import ExceptionGroup++ def g1():+ __tracebackhide__ = True+ str.does_not_exist++ def f3():+ __tracebackhide__ = True+ 1 / 0++ def f2():+ __tracebackhide__ = True+ exc = None+ try:+ f3()+ except Exception as e:+ exc = e+ exc2 = None+ try:+ g1()+ except Exception as e:+ exc2 = e+ raise ExceptionGroup("blah", [exc, exc2])++ def f1():+ __tracebackhide__ = True+ f2()++ def test():+ f1()+ """+ )+ result = pytester.runpytest(str(p), "--tb=short")+ assert result.ret == 1+ result.stdout.fnmatch_lines(+ [+ "*ExceptionGroup: blah (2 sub-exceptions)*",+ "*ZeroDivisionError: division by zero*",+ "*AttributeError*does_not_exist*",+ ]+ )+ result.stdout.no_fnmatch_line("*in f1*")+ result.stdout.no_fnmatch_line("*in f2*")+ result.stdout.no_fnmatch_line("*in f3*")+ result.stdout.no_fnmatch_line("*in g1*")++ def add_note(err: BaseException, msg: str) -> None: """Adds a note to an exception inplace.""" if sys.version_info < (3, 11):