Backtestsopenai-agents-pythonrun-20261024-184640#5213
Streamed runs swallow consumer cancellation and skip run cleanup while a parallel input guardrail is running
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Both patches make the same core change: `_await_task_safely` and `input_guardrail_tripwire_triggered_for_stream` switch from `await task` to `asyncio.wait((task,))`, forward consumer cancellation to the task, and re-raise, so a cancelled guardrail task no longer looks like the waiter's own cancellation. Both pass the four independent tests for the issue's two repro scenarios (consumer timeout/cancel swallowed; result.cancel() during an in-flight tool skipping cleanup). The decisive difference is the fifth independent test: `result.cancel()` called after the model finished, while the run loop's `finally` is already waiting on the guardrail. Here the CancelledError genuinely belongs to the run-loop task, so both patches' guardrail helper re-raises it. Patch A then lets it abort the rest of `start_streaming`'s `finally`, so computers are never disposed and the trace/spans never finish (independent test: FAIL-ASSERT on A, 'computer created for the run was not disposed'). Patch B explicitly catches that CancelledError in run_loop.py, finishes `dispose_resolved_computers()`, span/trace finish, and the completion sentinel, then re-raises it. The issue's stated expectation is general: 'cancelling the run should still finish the trace/spans and dispose per-run computers', and this is a very plausible ordering (fast model, slow moderation guardrail, user cancels). B resolves it; A does not. B also adds a regression test (`test_cancel_while_run_loop_awaits_input_guardrail_finishes_run_cleanup`) for exactly this path. Patch A's additional work is mixed. The try/finally around model-provider/sandbox cleanup in `stream_events` is reasonable hardening now that cancellation propagates, but the base already skips those cleanups on the `cancelled` path, so B's behaviour is consistent with existing semantics rather than a regression. A's `ignore_cancelled` keyword on `input_guardrail_tripwire_triggered_for_stream` adds API surface and forces a signature change in an unrelated existing test's monkeypatched finalizer; B's unconditional `if not task.cancelled(): task.result()` is simpler and matches the issue's root-cause description. A's extra `await asyncio.wait((task,))` after `task.cancel()` preserves old direct-await settlement, which is defensible but not required by the issue. Tests: both add deterministic, event-gated tests that would fail on base. B's are more directly mapped to the issue's scenarios (consumer cancel, consumer timeout, result.cancel during tool, consumer cancel during tool, cancel while run loop awaits guardrail, consumer cancel during run-loop cleanup wait). B's only skip is the `asyncio.timeout()` test on Python < 3.11, and it is paired with a non-skipped CancelledError equivalent, so coverage holds. A's parametrized consumer-cancellation test and cleanup-ordering test are solid but one of A's tests relies on fairly intricate monkeypatching of internals. On correctness, B covers strictly more of the issue's stated expected behaviour, confirmed by the independent run, with a smaller and clearer change. B wins.
Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.
Solve
18m 48s
Wall-clock time in the container
Full test suite
No new failures
2,626 passed · 1 skipped · 2,627 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
316vs 347 merged
Changed lines, ours vs merged
The issue
View on GitHubPlease read this first
- Have you read the docs? Yes, including the streaming cancellation section.
- Have you searched for related issues? Yes. #2222 fixed cancellation while
stream_events()waits for the next event; this report covers the remaining waits on the input guardrail task and the run-loop task, which still swallow cancellation.
Describe the bug
Streamed runs mishandle cancellation while a parallel input guardrail is still running. Parallel is the default for input guardrails, and a guardrail that outlasts the model, such as a moderation call, is common.
- The consumer's cancellation is swallowed. When the model has finished but the guardrail has not,
stream_events()waits for the guardrail after the final event. If the consumer is cancelled or itsasyncio.timeout()expires during that wait, the guardrail is cancelled, butstream_events()returns normally. The caller never seesCancelledError/TimeoutErrorand continues withresult.final_output, even though the input guardrail never produced a verdict. - Run cleanup is skipped after
result.cancel(). Ifresult.cancel()(or consumer cancellation) happens while the guardrail is running, the run loop'sfinallyawaits the cancelled guardrail task, gets itsCancelledError, and aborts the rest of cleanup. The trace plus the task and agent spans are never finished, and computers created by aComputerProviderfor the run are never disposed.
Debug information
- Agents SDK version: v0.22.3 and
mainat 08e5c431 - Python version: 3.12.13 (the regression tests also run on 3.10)
- Operating system: macOS
- Model and model provider: none; reproduced with
agents.testing.ScriptedModel - Does the issue reproduce with the latest Agents SDK release? Yes
- Does the issue occur consistently or intermittently? Consistently
Repro steps
import asyncio
from agents import (
Agent,
GuardrailFunctionOutput,
Runner,
add_trace_processor,
function_tool,
input_guardrail,
)
from agents.testing import ScriptedModel, assistant_message, function_call
from agents.tracing import TracingProcessor
class OpenTraces(TracingProcessor):
def __init__(self) -> None:
self.open: set[str] = set()
def on_trace_start(self, trace) -> None:
self.open.add(trace.trace_id)
def on_trace_end(self, trace) -> None:
self.open.discard(trace.trace_id)
def on_span_start(self, span) -> None: ...
def on_span_end(self, span) -> None: ...
def shutdown(self) -> None: ...
def force_flush(self) -> None: ...
traces = OpenTraces()
add_trace_processor(traces)
def make_agent(steps, tools=()):
# Parallel input guardrails are the default; this one outlasts the model.
@input_guardrail
async def slow_guardrail(ctx, agent, input):
await asyncio.sleep(3600)
return GuardrailFunctionOutput(output_info=None, tripwire_triggered=False)
return Agent(
name="assistant",
model=ScriptedModel(steps),
tools=list(tools),
input_guardrails=[slow_guardrail],
)
async def timeout_is_swallowed() -> None:
result = Runner.run_streamed(make_agent([[assistant_message("done")]]), "hi")
try:
async with asyncio.timeout(0.5):
async for _ in result.stream_events():
pass
print("1) timeout: stream_events() returned normally, final_output =", repr(result.final_output))
except TimeoutError:
print("1) timeout: TimeoutError raised")
async def cancel_skips_cleanup() -> None:
tool_started = asyncio.Event()
@function_tool
async def slow_tool() -> str:
tool_started.set()
await asyncio.sleep(3600)
return ""
traces.open.clear()
agent = make_agent([[function_call("slow_tool", {}, call_id="call_1")]], tools=[slow_tool])
result = Runner.run_streamed(agent, "hi")
async def consume() -> None:
async for _ in result.stream_events():
pass
consumer = asyncio.create_task(consume())
await tool_started.wait()
result.cancel()
await consumer
await asyncio.sleep(0.1)
print("2) result.cancel(): unfinished traces =", len(traces.open))
async def main() -> None:
await timeout_is_swallowed()
await cancel_skips_cleanup()
asyncio.run(main())
Output on main:
1) timeout: stream_events() returned normally, final_output = 'done'
2) result.cancel(): unfinished traces = 1
Expected behavior
1) timeout: TimeoutError raised
2) result.cancel(): unfinished traces = 0
Cancelling a stream consumer should propagate to that consumer, and cancelling the run should still finish the trace/spans and dispose per-run computers.
Root cause
Both waits await the shared task directly:
RunResultStreaming._await_task_safely()runsawait taskand suppressesCancelledError. Awaiting a task forwards the waiter's cancellation into the task, so after the consumer is cancelled the task ends cancelled. TheCancelledErrorthe consumer then receives looks like the task's own cancellation and is swallowed.input_guardrail_tripwire_triggered_for_stream(), which the run loop'sfinallycalls before disposing computers and finishing spans, runsawait task. When the guardrail task was cancelled bycancel(), awaiting it raisesCancelledErrorin the run loop and the remaining cleanup is skipped.
Ordering for (1): the model completes, QueueCompleteSentinel is queued, the consumer takes the sentinel and awaits the guardrail task, and the consumer is cancelled. The guardrail is cancelled, the consumer suppresses the resulting CancelledError, and stream_events() ends normally.
Ordering for (2): a tool (or the model) is in flight while the guardrail runs, and result.cancel() cancels the run loop and the guardrail. The run loop enters its finally, awaits the cancelled guardrail task, gets CancelledError, and exits before dispose_resolved_computers(), span.finish(), and trace.finish().
Waiting with asyncio.wait((task,)) avoids both problems: the task's own cancellation is not raised into the waiter, and a CancelledError there always belongs to the waiter. That waiter can then forward its cancellation to the task, as await did, and re-raise it.
Regression test evidence
Two deterministic regression tests through Runner.run_streamed() (event-gated, no sleeps for ordering) fail on main at 08e5c431:
test_cancel_with_pending_parallel_input_guardrail_finishes_run_cleanup
assert len(disposed) == 1 -> assert 0 == 1
test_consumer_cancellation_while_awaiting_input_guardrail_propagates
Failed: DID NOT RAISE <class 'asyncio.exceptions.CancelledError'>
Both pass with the change below. Because CONTRIBUTING.md limits pull requests to collaborators, the fix and tests are offered here for maintainers. They are also available as commit hsusul/openai-agents-python@d1e8466fef15d837a58a38727d16c61efdad097f on branch fix/stream-guardrail-wait-cancellation, based on 08e5c431.
diff --git a/src/agents/result.py b/src/agents/result.py
index 70d48fe3..52276efa 100644
--- a/src/agents/result.py
+++ b/src/agents/result.py
@@ -1166,20 +1166,20 @@ class RunResultStreaming(RunResultBase):
return pretty_print_run_result_streaming(self)
async def _await_task_safely(self, task: asyncio.Task[Any] | None) -> None:
- """Await a task if present, ignoring cancellation and storing exceptions elsewhere.
+ """Wait for a task if present, leaving its cancellation and exceptions to _check_errors().
- This ensures we do not lose late guardrail exceptions while not surfacing
- CancelledError to callers of stream_events.
+ This ensures we do not lose late guardrail exceptions while not surfacing the task's
+ CancelledError to callers of stream_events. Cancellation of the consumer still propagates.
"""
if task and not task.done():
+ # Do not await the task directly: its cancellation (e.g., due to result.cancel())
+ # would be indistinguishable from cancellation of this consumer.
try:
- await task
+ await asyncio.wait((task,))
except asyncio.CancelledError:
- # Task was cancelled (e.g., due to result.cancel()). Nothing to do here.
- pass
- except Exception:
- # The exception will be surfaced via _check_errors() if needed.
- pass
+ # This consumer was cancelled. Forward it to the task, as awaiting it would.
+ task.cancel()
+ raise
def _drain_event_queue(self) -> None:
"""Remove any pending items from the event queue and mark them done."""
diff --git a/src/agents/run_internal/guardrails.py b/src/agents/run_internal/guardrails.py
index dc09f0cd..199b935c 100644
--- a/src/agents/run_internal/guardrails.py
+++ b/src/agents/run_internal/guardrails.py
@@ -233,7 +233,16 @@ async def input_guardrail_tripwire_triggered_for_stream(
return False
if not task.done():
- await task
+ # Do not await the task directly: its cancellation (e.g., due to result.cancel()) would be
+ # indistinguishable from cancellation of this caller and abort the caller's cleanup.
+ try:
+ await asyncio.wait((task,))
+ except asyncio.CancelledError:
+ # This caller was cancelled. Forward it to the task, as awaiting it would.
+ task.cancel()
+ raise
+ if not task.cancelled():
+ task.result()
return any(
guardrail_result.output.tripwire_triggeredThe two patches
src/agents/result.pyindex 70d48fe3..52276efa 100644@@ -1166,20 +1166,20 @@ class RunResultStreaming(RunResultBase): return pretty_print_run_result_streaming(self) async def _await_task_safely(self, task: asyncio.Task[Any] | None) -> None:- """Await a task if present, ignoring cancellation and storing exceptions elsewhere.+ """Wait for a task if present, leaving its cancellation and exceptions to _check_errors(). - This ensures we do not lose late guardrail exceptions while not surfacing- CancelledError to callers of stream_events.+ This ensures we do not lose late guardrail exceptions while not surfacing the task's+ CancelledError to callers of stream_events. Cancellation of the consumer still propagates. """ if task and not task.done():+ # Do not await the task directly: its cancellation (e.g., due to result.cancel())+ # would be indistinguishable from cancellation of this consumer. try:- await task+ await asyncio.wait((task,)) except asyncio.CancelledError:- # Task was cancelled (e.g., due to result.cancel()). Nothing to do here.- pass- except Exception:- # The exception will be surfaced via _check_errors() if needed.- pass+ # This consumer was cancelled. Forward it to the task, as awaiting it would.+ task.cancel()+ raise def _drain_event_queue(self) -> None: """Remove any pending items from the event queue and mark them done."""src/agents/run_internal/guardrails.pyindex dc09f0cd..199b935c 100644@@ -233,7 +233,16 @@ async def input_guardrail_tripwire_triggered_for_stream( return False if not task.done():- await task+ # Do not await the task directly: its cancellation (e.g., due to result.cancel()) would be+ # indistinguishable from cancellation of this caller and abort the caller's cleanup.+ try:+ await asyncio.wait((task,))+ except asyncio.CancelledError:+ # This caller was cancelled. Forward it to the task, as awaiting it would.+ task.cancel()+ raise+ if not task.cancelled():+ task.result() return any( guardrail_result.output.tripwire_triggeredsrc/agents/run_internal/run_loop.pyindex da527bb5..046aa805 100644@@ -2045,6 +2045,7 @@ async def start_streaming( finally: await cleanup_models_after_run(tool_use_tracker) _sync_conversation_tracking_from_tracker()+ cleanup_cancelled_error: asyncio.CancelledError | None = None if streamed_result._input_guardrails_task: try: triggered = await input_guardrail_tripwire_triggered_for_stream(streamed_result)@@ -2059,6 +2060,10 @@ async def start_streaming( ) if first_trigger is not None: raise InputGuardrailTripwireTriggered(first_trigger)+ except asyncio.CancelledError as error:+ # The run was cancelled (e.g., result.cancel()) while waiting for the guardrail.+ # Finish the cleanup below before propagating it.+ cleanup_cancelled_error = error except Exception as e: log_model_and_tool_action_debug( logger,@@ -2084,6 +2089,8 @@ async def start_streaming( if not streamed_result.is_complete: streamed_result.is_complete = True streamed_result._event_queue.put_nowait(QueueCompleteSentinel())+ if cleanup_cancelled_error is not None:+ raise cleanup_cancelled_error async def run_single_turn_streamed(tests/test_stream_input_guardrail_cancellation.pynew file mode 100644index 00000000..b14306f8@@ -0,0 +1,280 @@+from __future__ import annotations++import asyncio+import sys+from typing import Any++import pytest++from agents import (+ Agent,+ ComputerProvider,+ ComputerTool,+ GuardrailFunctionOutput,+ Runner,+ function_tool,+ input_guardrail,+)+from agents.result import RunResultStreaming+from agents.testing import ScriptedModel++from .test_computer_tool_lifecycle import FakeComputer+from .test_responses import get_function_tool_call, get_text_message+from .testing_processor import fetch_events+++def _make_slow_guardrail(started: asyncio.Event) -> Any:+ @input_guardrail+ async def slow_guardrail(ctx: Any, agent: Any, input: Any) -> GuardrailFunctionOutput:+ started.set()+ await asyncio.Event().wait()+ return GuardrailFunctionOutput(output_info=None, tripwire_triggered=False)++ return slow_guardrail+++def _signal_when_awaiting(+ result: RunResultStreaming, target: asyncio.Task[Any] | None+) -> asyncio.Event:+ """Set an event once stream_events() starts waiting on the given background task."""+ entered = asyncio.Event()+ original = result._await_task_safely++ async def wrapped(task: asyncio.Task[Any] | None) -> None:+ if task is not None and task is target:+ entered.set()+ await original(task)++ result._await_task_safely = wrapped # type: ignore[method-assign]+ return entered+++async def _wait_for_run_loop(result: RunResultStreaming) -> None:+ assert result.run_loop_task is not None+ done, _ = await asyncio.wait((result.run_loop_task,), timeout=5)+ assert done, "run loop did not finish"+++def _assert_trace_and_spans_finished() -> None:+ events = fetch_events()+ assert events.count("trace_start") == 1+ assert events.count("trace_end") == 1+ assert events.count("span_start") == events.count("span_end")+++@pytest.mark.asyncio+async def test_consumer_cancellation_while_awaiting_input_guardrail_propagates() -> None:+ guardrail_started = asyncio.Event()+ agent = Agent(+ name="assistant",+ model=ScriptedModel(steps=[[get_text_message("done")]]),+ input_guardrails=[_make_slow_guardrail(guardrail_started)],+ )+ result = Runner.run_streamed(agent, "hi")+ await guardrail_started.wait()+ awaiting_guardrail = _signal_when_awaiting(result, result._input_guardrails_task)++ async def consume() -> None:+ async for _ in result.stream_events():+ pass++ consumer = asyncio.create_task(consume())+ await awaiting_guardrail.wait()+ # Let the consumer suspend inside the wait on the guardrail task.+ await asyncio.sleep(0)+ consumer.cancel()++ with pytest.raises(asyncio.CancelledError):+ await consumer+ assert result._input_guardrails_task is not None+ await asyncio.wait((result._input_guardrails_task,), timeout=5)+ assert result._input_guardrails_task.cancelled()+++@pytest.mark.asyncio+@pytest.mark.skipif(sys.version_info < (3, 11), reason="asyncio.timeout() requires Python 3.11+")+async def test_consumer_timeout_while_awaiting_input_guardrail_raises_timeout() -> None:+ guardrail_started = asyncio.Event()+ agent = Agent(+ name="assistant",+ model=ScriptedModel(steps=[[get_text_message("done")]]),+ input_guardrails=[_make_slow_guardrail(guardrail_started)],+ )+ result = Runner.run_streamed(agent, "hi")++ # The scripted model finishes immediately; the guardrail never does. The timeout must cancel+ # this consumer in its own task (asyncio.wait_for would run it in a separate one).+ with pytest.raises(TimeoutError):+ async with asyncio.timeout(0.5): # type: ignore[attr-defined,unused-ignore]+ async for _ in result.stream_events():+ pass+ assert guardrail_started.is_set()+ assert result._input_guardrails_task is not None+ await asyncio.wait((result._input_guardrails_task,), timeout=5)+ assert result._input_guardrails_task.cancelled()+++@pytest.mark.asyncio+async def test_consumer_cancellation_while_awaiting_run_loop_cleanup_propagates() -> None:+ dispose_started = asyncio.Event()+ release_dispose = asyncio.Event()++ async def create(**_kwargs: Any) -> FakeComputer:+ return FakeComputer()++ async def dispose(**_kwargs: Any) -> None:+ dispose_started.set()+ await release_dispose.wait()++ agent = Agent(+ name="assistant",+ model=ScriptedModel(steps=[[get_text_message("done")]]),+ tools=[ComputerTool(computer=ComputerProvider(create=create, dispose=dispose))],+ )+ result = Runner.run_streamed(agent, "hi")++ async def consume() -> None:+ async for _ in result.stream_events():+ pass++ consumer = asyncio.create_task(consume())+ await dispose_started.wait()+ # The final event has been queued; let the consumer reach the wait on the run loop.+ for _ in range(10):+ await asyncio.sleep(0)+ assert not consumer.done()+ consumer.cancel()++ with pytest.raises(asyncio.CancelledError):+ await consumer+ release_dispose.set()+ await _wait_for_run_loop(result)+++@pytest.mark.asyncio+async def test_cancel_with_pending_parallel_input_guardrail_finishes_run_cleanup() -> None:+ guardrail_started = asyncio.Event()+ tool_started = asyncio.Event()+ disposed: list[Any] = []++ @function_tool+ async def slow_tool() -> str:+ tool_started.set()+ await asyncio.Event().wait()+ return ""++ async def create(**_kwargs: Any) -> FakeComputer:+ return FakeComputer()++ async def dispose(*, computer: Any, **_kwargs: Any) -> None:+ disposed.append(computer)++ agent = Agent(+ name="assistant",+ model=ScriptedModel(steps=[[get_function_tool_call("slow_tool", "{}", call_id="call_1")]]),+ tools=[+ slow_tool,+ ComputerTool(computer=ComputerProvider(create=create, dispose=dispose)),+ ],+ input_guardrails=[_make_slow_guardrail(guardrail_started)],+ )+ result = Runner.run_streamed(agent, "hi")++ async def consume() -> None:+ async for _ in result.stream_events():+ pass++ consumer = asyncio.create_task(consume())+ await guardrail_started.wait()+ await tool_started.wait()+ result.cancel()+ await consumer+ await _wait_for_run_loop(result)++ assert len(disposed) == 1+ _assert_trace_and_spans_finished()+++@pytest.mark.asyncio+async def test_consumer_cancellation_with_pending_input_guardrail_finishes_run_cleanup() -> None:+ guardrail_started = asyncio.Event()+ tool_started = asyncio.Event()+ disposed: list[Any] = []++ @function_tool+ async def slow_tool() -> str:+ tool_started.set()+ await asyncio.Event().wait()+ return ""++ async def create(**_kwargs: Any) -> FakeComputer:+ return FakeComputer()++ async def dispose(*, computer: Any, **_kwargs: Any) -> None:+ disposed.append(computer)++ agent = Agent(+ name="assistant",+ model=ScriptedModel(steps=[[get_function_tool_call("slow_tool", "{}", call_id="call_1")]]),+ tools=[+ slow_tool,+ ComputerTool(computer=ComputerProvider(create=create, dispose=dispose)),+ ],+ input_guardrails=[_make_slow_guardrail(guardrail_started)],+ )+ result = Runner.run_streamed(agent, "hi")++ async def consume() -> None:+ async for _ in result.stream_events():+ pass++ consumer = asyncio.create_task(consume())+ await guardrail_started.wait()+ await tool_started.wait()+ consumer.cancel()+ with pytest.raises(asyncio.CancelledError):+ await consumer+ await _wait_for_run_loop(result)++ assert len(disposed) == 1+ _assert_trace_and_spans_finished()+++@pytest.mark.asyncio+async def test_cancel_while_run_loop_awaits_input_guardrail_finishes_run_cleanup() -> None:+ guardrail_started = asyncio.Event()+ disposed: list[Any] = []++ async def create(**_kwargs: Any) -> FakeComputer:+ return FakeComputer()++ async def dispose(*, computer: Any, **_kwargs: Any) -> None:+ disposed.append(computer)++ agent = Agent(+ name="assistant",+ model=ScriptedModel(steps=[[get_text_message("done")]]),+ tools=[ComputerTool(computer=ComputerProvider(create=create, dispose=dispose))],+ input_guardrails=[_make_slow_guardrail(guardrail_started)],+ )+ result = Runner.run_streamed(agent, "hi")+ await guardrail_started.wait()+ awaiting_guardrail = _signal_when_awaiting(result, result._input_guardrails_task)++ async def consume() -> None:+ async for _ in result.stream_events():+ pass++ consumer = asyncio.create_task(consume())+ # The model has finished, so the run loop is in its cleanup waiting on the guardrail.+ await awaiting_guardrail.wait()+ await asyncio.sleep(0)+ assert result.run_loop_task is not None+ assert not result.run_loop_task.done()+ result.cancel()+ await asyncio.wait((consumer,), timeout=5)+ assert consumer.done()+ await _wait_for_run_loop(result)++ assert len(disposed) == 1+ _assert_trace_and_spans_finished()
src/agents/result.pyindex 70d48fe3e..0dad43e5f 100644@@ -1045,25 +1045,25 @@ class RunResultStreaming(RunResultBase): registered_consumer_task.remove_done_callback(consumer_task_done) unregister_consumer() try:- if cancelled:- # Cancellation should return promptly, so avoid waiting on long-running tasks.- # Tasks have already been cancelled above.- self._cleanup_tasks()- else:- # Ensure main execution completes before cleanup to avoid race conditions- # with session operations.- await self._await_task_safely(self.run_loop_task)- # Re-check for exceptions now that the run loop has fully settled.- # _await_task_safely swallows exceptions; without this call, a run-loop- # failure that races past the sentinel (e.g. early sandbox failures) would- # be silently lost instead of surfaced via _stored_exception.- self._check_errors()- # Safely terminate all background tasks after main execution has finished.- self._cleanup_tasks()-- if not cancelled:- await self._await_model_provider_cleanup()- await self._run_sandbox_cleanup()+ try:+ if cancelled:+ # Queue-wait cancellation should return without awaiting long-running tasks.+ # Tasks have already been cancelled above.+ self._cleanup_tasks()+ else:+ # Ensure main execution completes before cleanup to avoid race conditions+ # with session operations.+ await self._await_task_safely(self.run_loop_task)+ # Re-check for exceptions now that the run loop has fully settled.+ # _await_task_safely leaves errors to _check_errors(); without this call,+ # a run-loop failure racing past the sentinel would be silently lost.+ self._check_errors()+ # Safely terminate background tasks after main execution has finished.+ self._cleanup_tasks()+ finally:+ if not cancelled:+ await self._await_model_provider_cleanup()+ await self._run_sandbox_cleanup() finally: # Allow any pending callbacks (e.g., cancellation handlers) to enqueue their # completion sentinels before we clear the queues for observability.@@ -1166,20 +1166,20 @@ class RunResultStreaming(RunResultBase): return pretty_print_run_result_streaming(self) async def _await_task_safely(self, task: asyncio.Task[Any] | None) -> None:- """Await a task if present, ignoring cancellation and storing exceptions elsewhere.+ """Wait for a task, leaving its cancellation and exceptions to _check_errors(). - This ensures we do not lose late guardrail exceptions while not surfacing- CancelledError to callers of stream_events.+ This ensures we do not lose late guardrail exceptions while not surfacing the task's+ CancelledError to callers of stream_events. Consumer cancellation still propagates. """ if task and not task.done(): try:- await task+ # Waiting directly would conflate child cancellation with consumer cancellation.+ await asyncio.wait((task,)) except asyncio.CancelledError:- # Task was cancelled (e.g., due to result.cancel()). Nothing to do here.- pass- except Exception:- # The exception will be surfaced via _check_errors() if needed.- pass+ task.cancel()+ # Preserve direct-await settlement before the consumer releases owned resources.+ await asyncio.wait((task,))+ raise def _drain_event_queue(self) -> None: """Remove any pending items from the event queue and mark them done."""src/agents/run_internal/guardrails.pyindex dc09f0cd7..d1a1c24d2 100644@@ -226,14 +226,23 @@ async def run_output_guardrails( async def input_guardrail_tripwire_triggered_for_stream( streamed_result: RunResultStreaming,+ *,+ ignore_cancelled: bool = False, ) -> bool:- """Return True if any input guardrail triggered during a streamed run."""+ """Wait for input verdicts, optionally tolerating child cancellation during final cleanup.""" task = streamed_result._input_guardrails_task if task is None: return False if not task.done():- await task+ # A cancelled child must not abort the caller's remaining run cleanup.+ try:+ await asyncio.wait((task,))+ except asyncio.CancelledError:+ task.cancel()+ raise+ if not ignore_cancelled or not task.cancelled():+ task.result() return any( guardrail_result.output.tripwire_triggeredsrc/agents/run_internal/run_loop.pyindex da527bb5f..bad24a835 100644@@ -2047,7 +2047,9 @@ async def start_streaming( _sync_conversation_tracking_from_tracker() if streamed_result._input_guardrails_task: try:- triggered = await input_guardrail_tripwire_triggered_for_stream(streamed_result)+ triggered = await input_guardrail_tripwire_triggered_for_stream(+ streamed_result, ignore_cancelled=True+ ) if triggered: first_trigger = next( (tests/test_agent_runner_streamed.pyindex e2518e93b..addef5e65 100644@@ -1580,7 +1580,7 @@ async def test_streamed_finalizer_failure_follows_both_data_policies( error = RuntimeError("SECRET_STREAM_FINALIZER_ERROR") - async def fail_finalizer(_result: Any) -> bool:+ async def fail_finalizer(_result: Any, *, ignore_cancelled: bool = False) -> bool: raise error monkeypatch.setattr(tests/test_cancel_streaming.pyindex ea7cb8563..bfe26d20a 100644@@ -6,13 +6,19 @@ import time import pytest from openai.types.responses import ResponseCompletedEvent -from agents import Agent, Runner+from agents import Agent, ComputerProvider, ComputerTool, GuardrailFunctionOutput, Runner+from agents.decorators import tool from agents.guardrail import input_guardrail from agents.models.multi_provider import MultiProvider+from agents.result import RunResultStreaming+from agents.run_internal import run_loop from agents.stream_events import RawResponsesStreamEvent from agents.testing import ScriptedModel +from .test_computer_tool_lifecycle import FakeComputer from .test_responses import get_function_tool, get_function_tool_call, get_text_message+from .testing_processor import fetch_events+from .utils.simple_session import SimpleListSession class SlowCompleteScriptedModel(ScriptedModel):@@ -328,3 +334,267 @@ async def test_falsy_input_guardrail_exception_is_surfaced_after_stream() -> Non with pytest.raises(FalsyRuntimeError, match="falsy guardrail boom"): async for _ in result.stream_events(): pass+++@pytest.mark.asyncio+async def test_cancel_with_pending_parallel_input_guardrail_finishes_cleanup() -> None:+ guardrail_started = asyncio.Event()+ tool_started = asyncio.Event()+ disposed: list[FakeComputer] = []++ @input_guardrail+ async def slow_guardrail(context, agent, input):+ guardrail_started.set()+ await asyncio.Event().wait()+ return GuardrailFunctionOutput(output_info=None, tripwire_triggered=False)++ @tool+ async def slow_tool() -> str:+ await guardrail_started.wait()+ tool_started.set()+ await asyncio.Event().wait()+ return "unreachable"++ def create_fake_computer(*, run_context) -> FakeComputer:+ return FakeComputer()++ computer_tool = ComputerTool(+ computer=ComputerProvider[FakeComputer](+ create=create_fake_computer,+ dispose=lambda *, run_context, computer: disposed.append(computer),+ )+ )+ agent = Agent(+ name="A",+ model=ScriptedModel([[get_function_tool_call("slow_tool", "{}", "call_1")]]),+ tools=[slow_tool, computer_tool],+ input_guardrails=[slow_guardrail],+ )+ result = Runner.run_streamed(agent, input="hi")++ async def consume() -> None:+ async for _ in result.stream_events():+ pass++ consumer = asyncio.create_task(consume())+ try:+ await asyncio.wait_for(tool_started.wait(), timeout=2)+ result.cancel()+ await asyncio.wait_for(consumer, timeout=2)+ assert len(disposed) == 1+ events = fetch_events()+ assert events.count("trace_start") == events.count("trace_end") == 1+ assert events.count("span_start") == events.count("span_end")+ finally:+ result.cancel()+ assert result.run_loop_task is not None+ await asyncio.gather(consumer, result.run_loop_task, return_exceptions=True)+++@pytest.mark.asyncio+@pytest.mark.parametrize("wait_target", ["guardrail", "run_loop"])+async def test_consumer_cancellation_at_terminal_wait_propagates(+ monkeypatch: pytest.MonkeyPatch, wait_target: str+) -> None:+ wait_started = asyncio.Event()+ guardrail_started = asyncio.Event()+ disposal_started = asyncio.Event()+ child_cancelled = asyncio.Event()++ @input_guardrail+ async def slow_guardrail(context, agent, input):+ guardrail_started.set()+ try:+ await asyncio.Event().wait()+ finally:+ child_cancelled.set()+ return GuardrailFunctionOutput(output_info=None, tripwire_triggered=False)++ async def dispose(**kwargs) -> None:+ disposal_started.set()+ try:+ await asyncio.Event().wait()+ finally:+ child_cancelled.set()++ def create_fake_computer(*, run_context) -> FakeComputer:+ return FakeComputer()++ computer_tool = ComputerTool(+ computer=ComputerProvider[FakeComputer](create=create_fake_computer, dispose=dispose)+ )+ result = Runner.run_streamed(+ Agent(+ name="A",+ model=ScriptedModel([[get_text_message("done")]]),+ input_guardrails=[slow_guardrail] if wait_target == "guardrail" else [],+ tools=[computer_tool] if wait_target == "run_loop" else [],+ ),+ input="hi",+ )+ original_wait = RunResultStreaming._await_task_safely++ async def observe_wait(self, task) -> None:+ target = self._input_guardrails_task if wait_target == "guardrail" else self.run_loop_task+ if self is result and task is target and task is not None and not task.done():+ wait_started.set()+ await original_wait(self, task)++ # Observe entry to the terminal wait without changing its behavior. Public events do not+ # expose this boundary, and the consumer must already be suspended here before cancellation.+ monkeypatch.setattr(RunResultStreaming, "_await_task_safely", observe_wait)++ async def consume() -> None:+ async for _ in result.stream_events():+ pass++ consumer = asyncio.create_task(consume())+ try:+ child_started = guardrail_started if wait_target == "guardrail" else disposal_started+ await asyncio.wait_for(child_started.wait(), timeout=2)+ await asyncio.wait_for(wait_started.wait(), timeout=2)+ assert result.final_output == "done"+ consumer.cancel()+ with pytest.raises(asyncio.CancelledError):+ await asyncio.wait_for(consumer, timeout=2)+ await asyncio.wait_for(child_cancelled.wait(), timeout=2)+ if wait_target == "guardrail":+ events = fetch_events()+ assert events.count("trace_start") == events.count("trace_end") == 1+ assert events.count("span_start") == events.count("span_end")+ finally:+ result.cancel()+ assert result.run_loop_task is not None+ await asyncio.gather(consumer, result.run_loop_task, return_exceptions=True)+++@pytest.mark.asyncio+async def test_pending_guardrail_cancellation_does_not_accept_or_persist_output(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ verdict_wait_started = asyncio.Event()+ dependency = asyncio.create_task(asyncio.Event().wait())+ session = SimpleListSession()+ original_verdict = run_loop.input_guardrail_tripwire_triggered_for_stream++ @input_guardrail+ async def guardrail(context, agent, input):+ await dependency+ return GuardrailFunctionOutput(output_info=None, tripwire_triggered=False)++ async def observe_verdict(result, **kwargs):+ verdict_wait_started.set()+ return await original_verdict(result, **kwargs)++ monkeypatch.setattr(run_loop, "input_guardrail_tripwire_triggered_for_stream", observe_verdict)+ result = Runner.run_streamed(+ Agent(+ name="A",+ model=ScriptedModel([[get_text_message("done")]]),+ input_guardrails=[guardrail],+ ),+ "hi",+ session=session,+ )++ async def consume() -> None:+ async for _ in result.stream_events():+ pass++ consumer = asyncio.create_task(consume())+ try:+ await asyncio.wait_for(verdict_wait_started.wait(), timeout=2)+ dependency.cancel()+ await asyncio.wait_for(consumer, timeout=2)+ assert result.final_output is None+ assert await session.get_items() == [{"content": "hi", "role": "user"}]+ assert result.run_loop_task is not None+ assert result.run_loop_task.cancelled()+ events = fetch_events()+ assert events.count("trace_start") == events.count("trace_end") == 1+ assert events.count("span_start") == events.count("span_end")+ finally:+ dependency.cancel()+ result.cancel()+ assert result.run_loop_task is not None+ await asyncio.gather(dependency, consumer, result.run_loop_task, return_exceptions=True)+++@pytest.mark.asyncio+async def test_terminal_consumer_cancellation_waits_for_registered_cleanup(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ wait_started = asyncio.Event()+ model_cleanup_started = asyncio.Event()+ provider_cleanup_started = asyncio.Event()+ cleanup_wait_started = asyncio.Event()+ cleanup_release = asyncio.Event()+ completed: list[str] = []++ class CleanupModel(ScriptedModel):+ async def _cleanup_on_run_end(self, owner) -> None:+ model_cleanup_started.set()+ await asyncio.Event().wait()++ async def close_provider(provider: MultiProvider) -> None:+ provider_cleanup_started.set()+ await cleanup_release.wait()+ completed.append("provider")++ async def cleanup_sandbox() -> None:+ await cleanup_release.wait()+ completed.append("sandbox")++ monkeypatch.setattr(MultiProvider, "aclose", close_provider)+ result = Runner.run_streamed(+ Agent(name="A", model=CleanupModel([[get_text_message("done")]])), "hi"+ )+ await asyncio.wait_for(model_cleanup_started.wait(), timeout=2)+ # Register the same cleanup wrapper used by SandboxRuntime without a provider backend.+ result._sandbox_cleanup = cleanup_sandbox+ result.ensure_sandbox_cleanup_on_completion()+ original_wait = RunResultStreaming._await_task_safely+ original_provider_wait = RunResultStreaming._await_model_provider_cleanup++ async def observe_wait(self, task) -> None:+ if self is result and task is self.run_loop_task:+ wait_started.set()+ await original_wait(self, task)++ monkeypatch.setattr(RunResultStreaming, "_await_task_safely", observe_wait)++ async def observe_provider_wait(self) -> None:+ if self is result:+ cleanup_wait_started.set()+ await original_provider_wait(self)++ monkeypatch.setattr(RunResultStreaming, "_await_model_provider_cleanup", observe_provider_wait)++ async def consume() -> None:+ async for _ in result.stream_events():+ pass++ consumer = asyncio.create_task(consume())+ cleanup_waiter = asyncio.create_task(cleanup_wait_started.wait())+ try:+ await asyncio.wait_for(wait_started.wait(), timeout=2)+ consumer.cancel()+ await asyncio.wait_for(provider_cleanup_started.wait(), timeout=2)+ # The consumer must await registered cleanup rather than return while callbacks run.+ await asyncio.wait_for(+ asyncio.wait((consumer, cleanup_waiter), return_when=asyncio.FIRST_COMPLETED), timeout=2+ )+ assert cleanup_wait_started.is_set()+ assert not consumer.done()+ cleanup_release.set()+ with pytest.raises(asyncio.CancelledError):+ await asyncio.wait_for(consumer, timeout=2)+ assert sorted(completed) == ["provider", "sandbox"]+ finally:+ cleanup_release.set()+ cleanup_waiter.cancel()+ result.cancel()+ assert result.run_loop_task is not None+ await asyncio.gather(consumer, cleanup_waiter, result.run_loop_task, return_exceptions=True)+ await result._await_model_provider_cleanup()+ await result._run_sandbox_cleanup()