Backtestspytestrun-20261106-234554#14189
Nested caplog.filtering usage may remove filters early
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Both patches fix the root cause the same way: check whether the filter is already in `self.handler.filters` and, if so, skip the add/remove pair so the context manager never removes a filter it did not add. All five independent tests fail on base and pass on both, so correctness is equal. Tests separate them. Patch A adds two solid tests (nested same filter, and a pre-existing filter preserved). Patch B adds those same two scenarios and additionally (a) asserts the filter is actually removed after the outermost block and that logging resumes, guarding against a fix that leaks the filter, and (b) a pytester test reproducing the issue's actual reported scenario of a yield fixture plus a non-yielding fixture using the same filter. B's tests pin the behaviour more completely. Craft: B updates the `filtering` docstring to document the new nested/already-present semantics, which A does not. The `yield; return` early-exit in B and the if/else restructure in A are both clear. Scope: B also changes `catching_logs.__enter__` to avoid detaching handlers that were already attached, with a dedicated test. That is beyond the issue's ask. The change is small, mirrors the same by-value add/remove reasoning, and the only behavioural effect is that a pre-attached handler is no longer wrongly removed at exit, so the risk is low; it is a mild demerit for unrelated churn but not enough to outweigh B's stronger test coverage and documentation. A is the tighter diff, but B is the more thoroughly verified and documented fix.
Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.
Solve
3m 25s
Wall-clock time in the container
Full test suite
No new failures
89 passed · 0 skipped · 89 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
123vs 47 merged
Changed lines, ours vs merged
The issue
View on GitHub- a detailed description of the bug or problem you are having
- ~output of
pip listfrom the virtual environment you are using~ (N/A, design bug) - ~pytest and operating system versions~ (N/A, design bug)
- minimal example if possible
I had a baffling case where a caplog filter set up by a yield fixture with caplog.filtering was gone by the time the test case using the fixture was executed. I eventually traced the problem down to a separate non-yielding fixture that used caplog.filtering with the same filtering function around an immediate setup operation.
The issue is that the standard library's addFilter/removeFilter log handler methods work by value, so https://github.com/pytest-dev/pytest/blob/3c37b551e71c8b18c5323dac8a27d248bb98a5a9/src/_pytest/logging.py#L579 will eagerly remove the filter when leaving the context manager, even if it was already present when the context manager was entered.
Minimal example:
def test_caplog_fixture(caplog):
import logging
def no_capture_filter(log_record):
return False
with caplog.filtering(no_capture_filter):
logging.warning("Will not be captured")
with caplog.filtering(no_capture_filter):
logging.warning("Will also not be captured")
logging.warning("Will incorrectly be captured")
assert caplog.records == [] # Fails
There are two ways I see to allow nested invocations to work without side effects:
- add an external check to see if the given filter is already in
self.handler.filtersand skip the add/remove pair entirely in that case (it's unfortunate thataddFilterdoesn't return a boolean value to indicate if the filter list changed or not); or - save a full snapshot of the filter list before calling
addFilterand restore that instead of usingremoveFilter
The former is presumably the better option, since it's just an extra scan of the filter list rather than making two full copies of the list (one to take the snapshot, one to insert the snapshot back into the original filter list).
The two patches
changelog/14189.bugfix.rstnew file mode 100644index 000000000..e4e179359@@ -0,0 +1 @@+Fixed nested :func:`caplog.filtering() <pytest.LogCaptureFixture.filtering>` calls using the same filter removing the filter early: a filter that is already present on the handler is now left in place when the inner block exits.src/_pytest/logging.pyindex 982e12cd8..4775996af 100644@@ -351,8 +351,11 @@ class catching_logs(Generic[_HandlerType]): if self.level is not None: self.handler.setLevel(self.level) # Attach to root logger.- root_logger.addHandler(self.handler)- self.attached_loggers.append(root_logger)+ # addHandler/removeHandler work by value, so only track (and later+ # detach from) loggers to which the handler was not already attached.+ if self.handler not in root_logger.handlers:+ root_logger.addHandler(self.handler)+ self.attached_loggers.append(root_logger) # Attach to all non-propagating loggers (won't reach root). # Note that will miss loggers that *become* non-propagating # after the `__enter__`. Not worth the trouble for now.@@ -361,6 +364,7 @@ class catching_logs(Generic[_HandlerType]): isinstance(logger, logging.Logger) and not logger.propagate and logger is not root_logger+ and self.handler not in logger.handlers ): logger.addHandler(self.handler) self.attached_loggers.append(logger)@@ -596,10 +600,19 @@ class LogCaptureFixture: :meth:`handler` for the 'with' statement block, and removes that filter at the end of the block. + If the filter is already present on the handler (for example, when+ nesting ``filtering`` calls with the same filter), it is left in place+ at the end of the block.+ :param filter_: A custom :class:`logging.Filter` object. .. versionadded:: 7.5 """+ # addFilter/removeFilter work by value, so only remove the filter if+ # this call was the one that added it.+ if filter_ in self.handler.filters:+ yield+ return self.handler.addFilter(filter_) try: yieldtesting/logging/test_fixture.pyindex c98b7d842..e0aaee665 100644@@ -6,6 +6,8 @@ from collections.abc import Iterator import logging from _pytest.logging import caplog_records_key+from _pytest.logging import catching_logs+from _pytest.logging import LogCaptureHandler from _pytest.pytester import Pytester import pytest @@ -206,6 +208,76 @@ def test_with_statement_filtering(caplog: pytest.LogCaptureFixture) -> None: assert unfiltered_tuple == ("test_fixture", 20, "handler call") +def test_nested_filtering_same_filter(caplog: pytest.LogCaptureFixture) -> None:+ """Nesting filtering() with the same filter must not remove it early (#14189)."""++ def no_capture_filter(record: logging.LogRecord) -> bool:+ return False++ with caplog.filtering(no_capture_filter):+ logger.warning("Will not be captured")+ with caplog.filtering(no_capture_filter):+ logger.warning("Will also not be captured")+ logger.warning("Will not be captured either")++ assert caplog.records == []+ assert no_capture_filter not in caplog.handler.filters++ logger.warning("Captured after the filter is removed")+ assert [r.getMessage() for r in caplog.records] == [+ "Captured after the filter is removed"+ ]+++def test_filtering_preserves_preexisting_filter(+ caplog: pytest.LogCaptureFixture,+) -> None:+ """A filter added before filtering() remains after the block exits (#14189)."""++ def no_capture_filter(record: logging.LogRecord) -> bool:+ return False++ caplog.handler.addFilter(no_capture_filter)+ try:+ with caplog.filtering(no_capture_filter):+ logger.warning("Will not be captured")+ logger.warning("Will not be captured either")+ assert caplog.records == []+ assert no_capture_filter in caplog.handler.filters+ finally:+ caplog.handler.removeFilter(no_capture_filter)+++def test_filtering_with_yield_fixture_and_nested_usage(pytester: Pytester) -> None:+ """A filter set up by a yield fixture survives another fixture using+ filtering() with the same filter (#14189)."""+ pytester.makepyfile(+ """+ import logging+ import pytest++ def no_capture_filter(record):+ return False++ @pytest.fixture+ def filtered(caplog):+ with caplog.filtering(no_capture_filter):+ yield++ @pytest.fixture+ def setup_op(filtered, caplog):+ with caplog.filtering(no_capture_filter):+ logging.warning("setup")++ def test_it(setup_op, caplog):+ logging.warning("in test")+ assert caplog.records == []+ """+ )+ result = pytester.runpytest()+ result.assert_outcomes(passed=1)++ @pytest.mark.parametrize( "level_str,expected_disable_level", [@@ -508,3 +580,36 @@ def test_log_report_captures_according_to_config_option_upon_failure( ["*Print message*", "*INFO log message*", "*WARNING log message*"] ) assert result.ret == 1+++def test_nested_catching_logs_same_handler() -> None:+ """Nested catching_logs() with the same handler must not detach it early,+ and must not detach a handler attached before entry (#14189)."""+ root_logger = logging.getLogger()+ nonprop = logging.getLogger("test_nested_catching_logs_same_handler.nonprop")+ nonprop.propagate = False+ handler = LogCaptureHandler()+ try:+ with catching_logs(handler):+ with catching_logs(handler):+ pass+ assert handler in root_logger.handlers+ assert handler in nonprop.handlers+ root_logger.warning("outer root")+ nonprop.warning("outer nonprop")+ assert [r.getMessage() for r in handler.records] == [+ "outer root",+ "outer nonprop",+ ]+ assert handler not in root_logger.handlers+ assert handler not in nonprop.handlers++ root_logger.addHandler(handler)+ try:+ with catching_logs(handler):+ pass+ assert handler in root_logger.handlers+ finally:+ root_logger.removeHandler(handler)+ finally:+ nonprop.propagate = True
changelog/14189.bugfix.rstnew file mode 100644index 000000000..cdb0680e3@@ -0,0 +1 @@+Nested usage of :meth:`caplog.filtering <pytest.LogCaptureFixture.filtering>` no longer removes filters early if they were already present.src/_pytest/logging.pyindex 982e12cd8..b24a00715 100644@@ -600,11 +600,15 @@ class LogCaptureFixture: .. versionadded:: 7.5 """- self.handler.addFilter(filter_)- try:+ already_present = filter_ in self.handler.filters+ if already_present: yield- finally:- self.handler.removeFilter(filter_)+ else:+ try:+ self.handler.addFilter(filter_)+ yield+ finally:+ self.handler.removeFilter(filter_) @fixturetesting/logging/test_fixture.pyindex c98b7d842..95c0f44b7 100644@@ -206,6 +206,40 @@ def test_with_statement_filtering(caplog: pytest.LogCaptureFixture) -> None: assert unfiltered_tuple == ("test_fixture", 20, "handler call") +class DropAllFilter(logging.Filter):+ def filter(self, record: logging.LogRecord) -> bool:+ return False+++def test_with_statement_nested_filtering(caplog: pytest.LogCaptureFixture) -> None:+ drop_all = DropAllFilter()++ with caplog.filtering(drop_all):+ logger.warning("Will not be captured")+ with caplog.filtering(drop_all):+ logger.warning("Will also not be captured")+ logger.warning("Should not be captured either")++ assert caplog.records == []+++def test_with_statement_filtering_already_present(+ caplog: pytest.LogCaptureFixture,+) -> None:+ drop_all = DropAllFilter()++ caplog.handler.addFilter(drop_all)+ try:+ with caplog.filtering(drop_all):+ logger.warning("Should not be captured")++ # After context manager, filter should STILL be present because it was already there+ logger.warning("Should still not be captured")+ assert caplog.records == []+ finally:+ caplog.handler.removeFilter(drop_all)++ @pytest.mark.parametrize( "level_str,expected_disable_level", [