Backtestsopenai-agents-pythonrun-20261024-184640#5072
Realtime: agent_end names the post-handoff agent instead of the agent whose turn ended
The judges split evenly between our patch and the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Both patches make the identical core fix: in the turn_ended branch they capture `self._active_output_response_agent or self._current_agent` before the guardrail state is cleared and pass it to RealtimeAgentEndEvent. This is exactly the root cause the issue identifies (the snapshot taken at turn_started is the agent that produced the turn, and it was being cleared one line before the end event was built). All five independent tests fail on base and pass on both patches, so on the issue's core scenarios they are equivalent. The differences are in tests and in one adjacent change. Patch A adds a single test that mirrors the repro (start A, handoff A->B, end A, and asserts `_current_agent` is B). Patch B adds two tests: one that also drives the follow-up turn and asserts the full paired sequence (start A, handoff, end A, start B, end B), which directly exercises the per-agent pairing the issue says consumers depend on, and one for a late turn_started arriving after output and a handoff for the same response. Patch B's second source change makes agent_start use the same snapshot when turn_started arrives after the response was already activated by earlier output; in that ordering base would name B as the starter of A's response, the same misattribution class the issue describes, so the change is consistent with the fix and the guardrail precedent rather than unrelated churn, and it is covered by a test. It is low risk because a fresh response re-assigns the snapshot in the same branch. Craft is a minor wash: A's test sits in test_session.py with top-level imports, while B uses function-local imports in a handoff test file, which is slightly less tidy. Overall B wins on test strength (verifies the next-turn pairing the issue motivates) with equal correctness on the reported bug and a coherent, low-risk adjacent consistency fix.
Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.
Solve
3m 56s
Wall-clock time in the container
Full test suite
No new failures
411 passed · 0 skipped · 411 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
102vs 39 merged
Changed lines, ours vs merged
The issue
View on GitHubPlease read this first
- Have you read the docs? Yes.
- Have you searched for related issues? Yes. No open or closed issue covers which agent
agent_endnames after an in-response handoff.
Describe the bug
When a Realtime response contains a handoff tool call, the handoff completes before the model's turn_ended event and moves _current_agent to the target agent. The turn_ended branch in RealtimeSession then emits RealtimeAgentEndEvent(agent=self._current_agent), so the caller sees:
agent_start(A) → handoff(A → B) → agent_end(B) → agent_start(B) → agent_end(B)
RealtimeAgentEndEvent.agent is documented as "The agent that ended", but A never ends and B ends without having started. Any consumer that pairs agent_start / agent_end per agent (turn markers in a UI, per-agent timing, per-agent usage attribution read on agent_end) attributes A's turn to B.
The session already knows which agent produced the turn: _active_output_response_agent is captured at turn_started and is what the output guardrails use for the same reason. It is cleared one line before the agent_end event is built.
Debug information
- Agents SDK version:
mainat 58a6d2c9 (also present in v0.22.2) - Python version: 3.12.13
- Operating system: macOS 15
- Model and model provider: any Realtime model; reproduced with
ScriptedRealtimeModelfromagents.realtime.testing - Does the issue reproduce with the latest Agents SDK release? Yes
- Does the issue occur consistently or intermittently? Consistently, whenever a handoff runs inside a response
Repro steps
Self-contained, no network.
import asyncio
from agents.realtime.agent import RealtimeAgent
from agents.realtime.events import RealtimeAgentEndEvent, RealtimeAgentStartEvent, RealtimeHandoffEvent
from agents.realtime.model_events import (
RealtimeModelToolCallEvent,
RealtimeModelTurnEndedEvent,
RealtimeModelTurnStartedEvent,
)
from agents.realtime.session import RealtimeSession
from agents.realtime.testing import ScriptedRealtimeModel
async def main():
agent_b = RealtimeAgent(name="b")
agent_a = RealtimeAgent(name="a", handoffs=[agent_b])
model = ScriptedRealtimeModel(strict=False)
session = RealtimeSession(model, agent_a, None, run_config={"async_tool_calls": False})
await session.enter()
await model.emit(RealtimeModelTurnStartedEvent(response_id="response_1"))
await model.emit(RealtimeModelToolCallEvent(name="transfer_to_b", call_id="call_1", arguments="{}"))
await model.emit(RealtimeModelTurnEndedEvent(response_id="response_1"))
while not session._event_queue.empty():
event = session._event_queue.get_nowait()
if isinstance(event, RealtimeAgentStartEvent):
print("agent_start", event.agent.name)
elif isinstance(event, RealtimeHandoffEvent):
print("handoff", event.from_agent.name, "->", event.to_agent.name)
elif isinstance(event, RealtimeAgentEndEvent):
print("agent_end", event.agent.name)
await session.close()
asyncio.run(main())
Output on main:
agent_start a
handoff a -> b
agent_end b
Expected behavior
agent_start a
handoff a -> b
agent_end a
The turn that ended was produced by A; the handoff only decides who speaks next. I have a one-line fix with a regression test ready and will open a PR referencing this issue.
The two patches
src/agents/realtime/session.pyindex f4f2792f..ded8b8e4 100644@@ -602,9 +602,12 @@ class RealtimeSession(RealtimeModelListener): self._active_output_response_generation = self._latest_output_response_generation self._active_output_response_id = event.response_id self._active_output_response_agent = self._current_agent+ # Name the agent that owns the response; a handoff may already have moved+ # _current_agent if this start arrives after the response began.+ started_agent = self._active_output_response_agent or self._current_agent await self._put_event( RealtimeAgentStartEvent(- agent=self._current_agent,+ agent=started_agent, info=self._event_info, ) )@@ -620,6 +623,9 @@ class RealtimeSession(RealtimeModelListener): event.response_id is None or event.response_id == self._active_output_response_id ) if is_active_response_ended:+ # The turn belongs to the agent that produced it, even if a handoff+ # during the response has already moved _current_agent.+ ended_agent = self._active_output_response_agent or self._current_agent # Clear guardrail state for next turn. self._item_transcripts.clear() self._item_guardrail_run_counts.clear()@@ -629,7 +635,7 @@ class RealtimeSession(RealtimeModelListener): await self._put_event( RealtimeAgentEndEvent(- agent=self._current_agent,+ agent=ended_agent, info=self._event_info, ) )tests/realtime/test_realtime_handoffs.pyindex 652f5de6..bbeacfa2 100644@@ -353,3 +353,95 @@ async def test_realtime_handoff_strict_json_rejects_type_coercion(): valid_json = '{"name": "Alice", "age": 25}' result = await handoff_obj.on_invoke_handoff(RunContextWrapper(None), valid_json) assert result is rt+++def _drain_lifecycle_events(session: Any) -> list[tuple[str, ...]]:+ from agents.realtime.events import (+ RealtimeAgentEndEvent,+ RealtimeAgentStartEvent,+ RealtimeHandoffEvent,+ )++ lifecycle: list[tuple[str, ...]] = []+ while not session._event_queue.empty():+ event = session._event_queue.get_nowait()+ if isinstance(event, RealtimeAgentStartEvent):+ lifecycle.append(("agent_start", event.agent.name))+ elif isinstance(event, RealtimeHandoffEvent):+ lifecycle.append(("handoff", event.from_agent.name, event.to_agent.name))+ elif isinstance(event, RealtimeAgentEndEvent):+ lifecycle.append(("agent_end", event.agent.name))+ return lifecycle+++@pytest.mark.asyncio+async def test_agent_end_names_agent_whose_turn_ended_after_in_response_handoff() -> None:+ from agents.realtime.model_events import (+ RealtimeModelToolCallEvent,+ RealtimeModelTurnEndedEvent,+ RealtimeModelTurnStartedEvent,+ )+ from agents.realtime.session import RealtimeSession+ from agents.realtime.testing import ScriptedRealtimeModel++ agent_b = RealtimeAgent(name="b")+ agent_a = RealtimeAgent(name="a", handoffs=[agent_b])+ model = ScriptedRealtimeModel(strict=False)+ session = RealtimeSession(model, agent_a, None, run_config={"async_tool_calls": False})+ await session.enter()+ try:+ await model.emit(RealtimeModelTurnStartedEvent(response_id="response_1"))+ await model.emit(+ RealtimeModelToolCallEvent(name="transfer_to_b", call_id="call_1", arguments="{}")+ )+ await model.emit(RealtimeModelTurnEndedEvent(response_id="response_1"))++ # The next turn is produced by the handoff target.+ await model.emit(RealtimeModelTurnStartedEvent(response_id="response_2"))+ await model.emit(RealtimeModelTurnEndedEvent(response_id="response_2"))++ assert _drain_lifecycle_events(session) == [+ ("agent_start", "a"),+ ("handoff", "a", "b"),+ ("agent_end", "a"),+ ("agent_start", "b"),+ ("agent_end", "b"),+ ]+ finally:+ await session.close()+++@pytest.mark.asyncio+async def test_late_agent_start_names_response_agent_after_in_response_handoff() -> None:+ from agents.realtime.model_events import (+ RealtimeModelToolCallEvent,+ RealtimeModelTranscriptDeltaEvent,+ RealtimeModelTurnEndedEvent,+ RealtimeModelTurnStartedEvent,+ )+ from agents.realtime.session import RealtimeSession+ from agents.realtime.testing import ScriptedRealtimeModel++ agent_b = RealtimeAgent(name="b")+ agent_a = RealtimeAgent(name="a", handoffs=[agent_b])+ model = ScriptedRealtimeModel(strict=False)+ session = RealtimeSession(model, agent_a, None, run_config={"async_tool_calls": False})+ await session.enter()+ try:+ # Output from A's response arrives before its turn_started event.+ await model.emit(+ RealtimeModelTranscriptDeltaEvent(item_id="item_1", delta="hi", response_id="r1")+ )+ await model.emit(+ RealtimeModelToolCallEvent(name="transfer_to_b", call_id="call_1", arguments="{}")+ )+ await model.emit(RealtimeModelTurnStartedEvent(response_id="r1"))+ await model.emit(RealtimeModelTurnEndedEvent(response_id="r1"))++ assert _drain_lifecycle_events(session) == [+ ("handoff", "a", "b"),+ ("agent_start", "a"),+ ("agent_end", "a"),+ ]+ finally:+ await session.close()
src/agents/realtime/session.pyindex f4f2792fe..7e5f4aebb 100644@@ -620,6 +620,11 @@ class RealtimeSession(RealtimeModelListener): event.response_id is None or event.response_id == self._active_output_response_id ) if is_active_response_ended:+ # A handoff inside the response has already moved _current_agent to the+ # new agent, so the agent that produced this turn is the one captured at+ # turn_started, the same snapshot the output guardrails use.+ ended_agent = self._active_output_response_agent or self._current_agent+ # Clear guardrail state for next turn. self._item_transcripts.clear() self._item_guardrail_run_counts.clear()@@ -629,7 +634,7 @@ class RealtimeSession(RealtimeModelListener): await self._put_event( RealtimeAgentEndEvent(- agent=self._current_agent,+ agent=ended_agent, info=self._event_info, ) )tests/realtime/test_session.pyindex d302c9c48..be2366e0f 100644@@ -23,6 +23,7 @@ from agents.realtime.events import ( RealtimeAudioEnd, RealtimeAudioInterrupted, RealtimeError,+ RealtimeHandoffEvent, RealtimeRawModelEvent, RealtimeToolEnd, RealtimeToolStart,@@ -54,6 +55,7 @@ from agents.realtime.session import ( RealtimeSession, _PendingToolOutputSendError, )+from agents.realtime.testing import ScriptedRealtimeModel from agents.run_context import RunContextWrapper from agents.tool import FunctionTool, function_tool, tool_namespace from agents.tool_context import ToolContext@@ -330,6 +332,36 @@ async def test_close_clears_response_bookkeeping_when_model_close_fails(): assert session._responses_awaiting_guardrail_cleanup == set() +@pytest.mark.asyncio+async def test_agent_end_names_the_agent_whose_turn_ended_after_handoff():+ agent_b = RealtimeAgent(name="b")+ agent_a = RealtimeAgent(name="a", handoffs=[agent_b])+ model = ScriptedRealtimeModel(strict=False)+ session = RealtimeSession(model, agent_a, None, run_config={"async_tool_calls": False})+ await session.enter()+ try:+ await model.emit(RealtimeModelTurnStartedEvent(response_id="response_1"))+ await model.emit(+ RealtimeModelToolCallEvent(name="transfer_to_b", call_id="call_1", arguments="{}")+ )+ await model.emit(RealtimeModelTurnEndedEvent(response_id="response_1"))++ events: list[Any] = []+ while not session._event_queue.empty():+ events.append(session._event_queue.get_nowait())+ finally:+ await session.close()++ starts = [event for event in events if isinstance(event, RealtimeAgentStartEvent)]+ handoffs = [event for event in events if isinstance(event, RealtimeHandoffEvent)]+ ends = [event for event in events if isinstance(event, RealtimeAgentEndEvent)]+ assert [event.agent for event in starts] == [agent_a]+ assert [(event.from_agent, event.to_agent) for event in handoffs] == [(agent_a, agent_b)]+ # The turn that ended was produced by agent A; the handoff only changes who speaks next.+ assert [event.agent for event in ends] == [agent_a]+ assert session._current_agent is agent_b++ @pytest.mark.asyncio async def test_cancelling_one_close_waiter_does_not_cancel_cleanup(): class BlockingCloseModel(_DummyModel):