Backtestspydantic-airun-20260918-215936#7527

Issue #7527 Merged fix PR # Run 18 Sep 2026 · run-20260918-215936 pydantic/pydantic-ai

AG-UI emits orphaned TOOL_CALL_START events for tool-only responses

Worse

The merged fix was judged better than our patch.

In this judgment, Patch B = our agent, Patch A = the merged fix.

The reported issue is specifically about **tool-only responses** (a model response containing only tool calls, no TextPart): `before_response()` mints a synthetic message_id via `new_message_id()`, but since `handle_text_start()` is never called for such a response, no `TEXT_MESSAGE_START` is emitted, leaving `TOOL_CALL_START.parentMessageId` pointing at an ID no emitted event references.

Patch A addresses this at the root: it tracks whether a `TEXT_MESSAGE_START` has been emitted for the current message_id and, when the first tool call arrives with none sent, lazily emits `TextMessageStartEvent(message_id=parent_message_id)`. This makes the parent a real, emitted assistant message. Against the issue's own minimal repro, `next(...)` returns the FIRST `TEXT_MESSAGE_START`, which under Patch A is the lazily-emitted one carrying the synthetic ID that equals the tool parent — so the repro's `parent matches an emitted text message` assertion flips to True. Its test asserts exactly the required ordering (`TEXT_MESSAGE_START` before `TOOL_CALL_START` for a tool-only response) via a full snapshot that would fail before the change.

Patch B does NOT fix the reported scenario. It only changes `handle_text_start()` to reuse the response's message_id (with a `_message_started` guard) so that a tool-call-then-text response *in the same response* shares one assistant-message id. But for a genuinely tool-only response there is no TextPart, so `handle_text_start()` is never invoked and NO `TEXT_MESSAGE_START` is ever emitted — the orphaned parent persists. In the issue's repro, the tool call precedes any text (and `_handle_tool_result` further replaces `message_id` before the later text arrives), so the only emitted `TEXT_MESSAGE_START` carries a different id than the tool parent; Patch B's assertion would fail on the repro. Patch B's test exercises a different, adjacent case (tool+text in one response) rather than the tool-only case the issue is titled and reported for, so it is largely tautological with respect to the actual bug.

On scope both are small; Patch B's refactor is arguably cleaner for the tool+text case (Patch A leaves an empty, unclosed `TEXT_MESSAGE_START` with no matching `TEXT_MESSAGE_END`, and would emit a second `TEXT_MESSAGE_START` for a tool-then-text response — a mild protocol wart), but correctness dominates: Patch A resolves the reported issue and Patch B does not touch its core scenario. Winner A.

The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.

Solve

34m 37s4 turns

Wall-clock time in the container

Full test suite

No new failures

11,744 passed · 574 skipped · 12,543 total

Contamination audit

Unknown

See the audit notes for this run

Patch size

45vs 157 merged

Changed lines, ours vs merged

Initial Checks

Description

Description

For a model response containing only tool calls, AGUIEventStream emits a
TOOL_CALL_START whose parentMessageId does not identify any assistant
message emitted in the stream.1.

before_response() creates a fresh internal message ID. Later,
_handle_tool_call_start() uses that ID as parentMessageId, but no
TEXT_MESSAGE_START is emitted when the response has no TextPart.

AG-UI clients cannot reliably reconstruct the assistant message that owns the
tool call. In our frontend, the resulting assistant message is later replayed
with this synthetic ID, while the following text response has a different
TEXT_MESSAGE_START ID. This produces a non-round-trippable history: a server
that persists the text-message ID cannot recognize the replayed assistant
message, and it re-sends the old tool call and tool result to the model.

This reproduces with Pydantic AI 2.31.0. The current source still has the
relevant behavior:

Related issues:

  • #3316 fixed tool calls attaching to a previous response, but deliberately creates this fresh synthetic parent for each new response.
  • #4098 reported the same missing-parent condition, but the current reproduction shows it remains present in the latest release.
  • #7491 concerns message_id mutation after a function-tool result. It is related, but does not create an emitted assistant message for a tool-only response.

Expected Behavior

TOOL_CALL_START.parentMessageId should identify an assistant message that
the stream emitted, so an AG-UI client can build an AssistantMessage with
the tool call and later replay the history without changing its structure.

Actual Behavior

For a tool-only response, TOOL_CALL_START.parentMessageId references a
synthetic ID with no emitted TEXT_MESSAGE_START. The client must infer or
invent the parent message, and its replayed IDs diverge from the later text
message IDs emitted by the adapter.

Minimal, Reproducible Example

import asyncio
import json

from ag_ui.core import RunAgentInput
from pydantic_ai.messages import (
    FunctionToolResultEvent,
    PartEndEvent,
    PartStartEvent,
    TextPart,
    ToolCallPart,
    ToolReturnPart,
)
from pydantic_ai.ui.ag_ui._event_stream import AGUIEventStream


async def source():
    tool_call = ToolCallPart(tool_name="tool", args={}, tool_call_id="call")
    yield PartStartEvent(index=0, part=tool_call, previous_part_kind=None)
    yield PartEndEvent(index=0, part=tool_call, next_part_kind=None)
    yield FunctionToolResultEvent(
        part=ToolReturnPart(
            tool_name="tool",
            content="result",
            tool_call_id="call",
        )
    )
    text = TextPart(content="final answer")
    yield PartStartEvent(index=0, part=text, previous_part_kind=None)
    yield PartEndEvent(index=0, part=text, next_part_kind=None)


async def main() -> None:
    stream = AGUIEventStream(
        run_input=RunAgentInput(
            thread_id="thread",
            run_id="run",
            messages=[],
            tools=[],
            context=[],
            forwarded_props={},
            state={},
        ),
        accept="text/event-stream",
    )

    events = [
        json.loads(event.removeprefix("data: "))
        async for event in stream.encode_stream(stream.transform_stream(source()))
    ]
    tool_start = next(
        event for event in events if event["type"] == "TOOL_CALL_START"
    )
    final_text_start = next(
        event for event in events if event["type"] == "TEXT_MESSAGE_START"
    )
    print(f"tool parent: {tool_start['parentMessageId']}")
    print(f"final text ID: {final_text_start['messageId']}")
    print(
        "parent matches an emitted text message: "
        f"{tool_start['parentMessageId'] == final_text_start['messageId']}"
    )


asyncio.run(main())


Output:


tool parent: 0988a36d-a56e-4143-9aed-aaa13b31aa39
final text ID: 2fd1d781-bf05-4101-96f8-f413257af882
parent matches an emitted text message: False

Logfire Trace

Not applicable. The minimal example reproduces without an agent run or model request.

Python, Pydantic AI & LLM client version

  • Python: 3.14.6
  • Pydantic AI: 2.31.0
  • ag-ui-protocol: 0.1.20
  • LLM provider SDK: not used

The two patches

Our patch+41 −4
pydantic_ai_slim/pydantic_ai/ui/ag_ui/_event_stream.pyindex bb637689..9ffd96e1 100644@@ -145,6 +145,7 @@ class AGUIEventStream(UIEventStream[RunAgentInput, BaseEvent, AgentDepsT, Output     _builtin_tool_call_ids: dict[str, str] = field(default_factory=dict[str, str])     _error: bool = False     _cancelled_run: bool = False+    _message_started: bool = False      def __post_init__(self) -> None:         self._use_reasoning = parse_ag_ui_version(self.ag_ui_version) >= REASONING_VERSION@@ -199,6 +200,7 @@ class AGUIEventStream(UIEventStream[RunAgentInput, BaseEvent, AgentDepsT, Output         # Prevent parts from a subsequent response being tied to parts from an earlier response.         # See https://github.com/pydantic/pydantic-ai/issues/3316         self.new_message_id()+        self._message_started = False         return         yield  # Make this an async generator @@ -262,10 +264,14 @@ class AGUIEventStream(UIEventStream[RunAgentInput, BaseEvent, AgentDepsT, Output         yield      async def handle_text_start(self, part: TextPart, follows_text: bool = False) -> AsyncIterator[BaseEvent]:-        if follows_text:-            message_id = self.message_id-        else:-            message_id = self.new_message_id()+        # Reuse the response's message ID, minted in `before_response`, rather than minting a second one+        # here: tool calls in the same response already reference it as `parent_message_id`, so a+        # tool-call-then-text response is announced as one assistant message instead of two. The+        # `_message_started` guard keeps a single `TEXT_MESSAGE_START` per response even when text parts+        # are split around a tool call or thinking part.+        message_id = self.message_id+        if not follows_text and not self._message_started:+            self._message_started = True             yield TextMessageStartEvent(message_id=message_id)          if part.content:  # pragma: no branchtests/test_ag_ui.pyindex e88a80f1..3bef24c2 100644@@ -4664,6 +4664,37 @@ async def test_event_stream_multiple_responses_with_tool_calls():     assert result_message_id != next_response_id  +async def test_tool_call_then_text_share_parent_message_id():+    """A tool call followed by text in one response is announced under a single assistant-message id.++    Regression for https://github.com/pydantic/pydantic-ai/issues/7527: `TOOL_CALL_START.parentMessageId`+    must identify the assistant message the stream emits for the same response (the one announced by+    `TEXT_MESSAGE_START`), rather than a second, separately-minted id.+    """++    async def event_generator():+        tool_call = ToolCallPart(tool_name='tool', args='{}', tool_call_id='call_1')+        yield PartStartEvent(index=0, part=tool_call, previous_part_kind=None)+        yield PartEndEvent(index=0, part=tool_call, next_part_kind='text')+        text = TextPart(content='final answer')+        yield PartStartEvent(index=1, part=text, previous_part_kind='tool-call')+        yield PartEndEvent(index=1, part=text, next_part_kind=None)++    run_input = create_input(UserMessage(id='msg_1', content='Please call the tool'))+    event_stream = AGUIEventStream(run_input=run_input)+    events = [+        json.loads(event.removeprefix('data: '))+        async for event in event_stream.encode_stream(event_stream.transform_stream(event_generator()))+    ]++    tool_start = next(event for event in events if event['type'] == 'TOOL_CALL_START')+    text_starts = [event for event in events if event['type'] == 'TEXT_MESSAGE_START']+    # Exactly one assistant message is announced for the single response...+    assert len(text_starts) == 1+    # ...and the tool call is parented to that announced message.+    assert tool_start['parentMessageId'] == text_starts[0]['messageId']++ async def test_agent_multiple_responses_use_distinct_parent_message_ids() -> None:     """Each model response gets a fresh AG-UI parent through the supported `Agent` lifecycle."""  
Merged fix · PR #+157 −0
pydantic_ai_slim/pydantic_ai/ui/ag_ui/_event_stream.pyindex bb637689a6..45d8d86e61 100644@@ -145,6 +145,14 @@ class AGUIEventStream(UIEventStream[RunAgentInput, BaseEvent, AgentDepsT, Output     _builtin_tool_call_ids: dict[str, str] = field(default_factory=dict[str, str])     _error: bool = False     _cancelled_run: bool = False+    # Track whether a TextMessageStartEvent has been emitted for the current+    # message_id.  When a response contains only tool calls (no TextPart),+    # before_response() mints a synthetic message_id but handle_text_start()+    # is never called, so _handle_tool_call_start() emits ToolCallStartEvent+    # with a parent_message_id that no emitted event refers to.  The flag lets+    # us emit a TextMessageStartEvent lazily when the first tool call arrives.+    # See https://github.com/pydantic/pydantic-ai/issues/7527+    _text_message_start_sent: bool = field(default=False, init=False)      def __post_init__(self) -> None:         self._use_reasoning = parse_ag_ui_version(self.ag_ui_version) >= REASONING_VERSION@@ -199,6 +207,7 @@ class AGUIEventStream(UIEventStream[RunAgentInput, BaseEvent, AgentDepsT, Output         # Prevent parts from a subsequent response being tied to parts from an earlier response.         # See https://github.com/pydantic/pydantic-ai/issues/3316         self.new_message_id()+        self._text_message_start_sent = False         return         yield  # Make this an async generator @@ -267,6 +276,7 @@ class AGUIEventStream(UIEventStream[RunAgentInput, BaseEvent, AgentDepsT, Output         else:             message_id = self.new_message_id()             yield TextMessageStartEvent(message_id=message_id)+            self._text_message_start_sent = True          if part.content:  # pragma: no branch             yield TextMessageContentEvent(message_id=message_id, delta=part.content)@@ -336,6 +346,14 @@ class AGUIEventStream(UIEventStream[RunAgentInput, BaseEvent, AgentDepsT, Output         tool_call_id = tool_call_id or part.tool_call_id         parent_message_id = self.message_id +        # When a response contains only tool calls (no TextPart), no+        # TextMessageStartEvent has been emitted for the synthetic message_id+        # that before_response() created.  Emit one now so the client can+        # build an assistant message to own the tool call.+        if not self._text_message_start_sent:+            yield TextMessageStartEvent(message_id=parent_message_id)+            self._text_message_start_sent = True+         yield ToolCallStartEvent(             tool_call_id=tool_call_id, tool_call_name=part.tool_name, parent_message_id=parent_message_id         )tests/test_ag_ui.pyindex e88a80f13e..f674908b23 100644@@ -8025,3 +8025,142 @@ async def test_tool_availability_delta_stream_matches_dumped_activity_message()     # The literal is a frontend-facing wire contract: deriving both sides from the shared constant     # would let a rename drift silently.     assert activity.activity_type == 'pydantic_ai_tool_availability_delta'+++async def test_tool_only_response_emits_text_message_start() -> None:+    """Regression test for #7527: tool-only responses must emit TEXT_MESSAGE_START.++    When a response contains only tool calls (no TextPart), before_response()+    mints a synthetic message_id but no TEXT_MESSAGE_START is emitted.+    TOOL_CALL_START then uses that synthetic ID as parentMessageId, which+    refers to no event in the stream. This test verifies a TEXT_MESSAGE_START+    is emitted before the first TOOL_CALL_START for a tool-only response.+    """++    async def stream_function(+        messages: list[ModelMessage], agent_info: AgentInfo+    ) -> AsyncIterator[DeltaToolCalls | str]:+        if len(messages) == 1:+            yield {0: DeltaToolCall(name='get_weather', json_args='{"location":"Paris"}')}+        else:+            yield 'Final result'++    agent = Agent(+        model=FunctionModel(stream_function=stream_function),+        tools=[get_weather()],+    )++    thread_id = uuid_str()+    run_inputs = [+        create_input(+            UserMessage(+                id='msg_1',+                content='Get weather for Paris',+            ),+            tools=[get_weather()],+            thread_id=thread_id,+        ),+        create_input(+            UserMessage(+                id='msg_1',+                content='Get weather for Paris',+            ),+            AssistantMessage(+                id='msg_2',+                tool_calls=[+                    ToolCall(+                        id='pyd_ai_00000000000000000000000000000003',+                        type='function',+                        function=FunctionCall(+                            name='get_weather',+                            arguments='{"location": "Paris"}',+                        ),+                    ),+                ],+            ),+            ToolMessage(+                id='msg_3',+                content='Tool result',+                tool_call_id='pyd_ai_00000000000000000000000000000003',+            ),+            thread_id=thread_id,+        ),+    ]++    events = await run_and_collect_events(agent, *run_inputs)++    assert events == snapshot(+        [+            {+                'type': 'RUN_STARTED',+                'timestamp': IsInt(),+                'threadId': thread_id,+                'runId': (run_id := IsSameStr()),+            },+            {+                # The first response is tool-only: TEXT_MESSAGE_START must be+                # emitted before TOOL_CALL_START so the client has a parent+                # message to attach the tool call to.+                'type': 'TEXT_MESSAGE_START',+                'timestamp': IsInt(),+                'messageId': IsStr(),+                'role': 'assistant',+            },+            {+                'type': 'TOOL_CALL_START',+                'timestamp': IsInt(),+                'toolCallId': (tool_call_id := IsSameStr()),+                'toolCallName': 'get_weather',+                'parentMessageId': IsStr(),+            },+            {+                'type': 'TOOL_CALL_ARGS',+                'timestamp': IsInt(),+                'toolCallId': tool_call_id,+                'delta': '{"location":"Paris"}',+            },+            {'type': 'TOOL_CALL_END', 'timestamp': IsInt(), 'toolCallId': tool_call_id},+            # Tool call result via _handle_tool_result mints its own message_id.+            {+                'type': 'TOOL_CALL_RESULT',+                'timestamp': IsInt(),+                'messageId': IsStr(),+                'type_': 'tool_call_result',+                'role': 'tool',+                'toolCallId': tool_call_id,+                'content': '"Tool result"',+            },+            {+                'type': 'RUN_FINISHED',+                'timestamp': IsInt(),+                'threadId': thread_id,+                'runId': run_id,+            },+            {+                'type': 'RUN_STARTED',+                'timestamp': IsInt(),+                'threadId': thread_id,+                'runId': (run_id := IsSameStr()),+            },+            {+                # Second response has a TextPart — normal TEXT_MESSAGE_START.+                'type': 'TEXT_MESSAGE_START',+                'timestamp': IsInt(),+                'messageId': (message_id := IsSameStr()),+                'role': 'assistant',+            },+            {+                'type': 'TEXT_MESSAGE_CONTENT',+                'timestamp': IsInt(),+                'messageId': message_id,+                'delta': 'Final result',+            },+            {'type': 'TEXT_MESSAGE_END', 'timestamp': IsInt(), 'messageId': message_id},+            {+                'type': 'RUN_FINISHED',+                'timestamp': IsInt(),+                'threadId': thread_id,+                'runId': run_id,+            },+        ]+    ) 

What the agent reported

The full AG-UI suite is running in the background (it exceeds the 2-minute foreground window). I'll report when it completes.