Backtestspydantic-airun-20260918-215936#7527
AG-UI emits orphaned TOOL_CALL_START events for tool-only responses
The merged fix was judged better than our patch.
In this judgment, Patch B = our agent, Patch A = the merged fix.
The reported issue is specifically about **tool-only responses** (a model response containing only tool calls, no TextPart): `before_response()` mints a synthetic message_id via `new_message_id()`, but since `handle_text_start()` is never called for such a response, no `TEXT_MESSAGE_START` is emitted, leaving `TOOL_CALL_START.parentMessageId` pointing at an ID no emitted event references. Patch A addresses this at the root: it tracks whether a `TEXT_MESSAGE_START` has been emitted for the current message_id and, when the first tool call arrives with none sent, lazily emits `TextMessageStartEvent(message_id=parent_message_id)`. This makes the parent a real, emitted assistant message. Against the issue's own minimal repro, `next(...)` returns the FIRST `TEXT_MESSAGE_START`, which under Patch A is the lazily-emitted one carrying the synthetic ID that equals the tool parent — so the repro's `parent matches an emitted text message` assertion flips to True. Its test asserts exactly the required ordering (`TEXT_MESSAGE_START` before `TOOL_CALL_START` for a tool-only response) via a full snapshot that would fail before the change. Patch B does NOT fix the reported scenario. It only changes `handle_text_start()` to reuse the response's message_id (with a `_message_started` guard) so that a tool-call-then-text response *in the same response* shares one assistant-message id. But for a genuinely tool-only response there is no TextPart, so `handle_text_start()` is never invoked and NO `TEXT_MESSAGE_START` is ever emitted — the orphaned parent persists. In the issue's repro, the tool call precedes any text (and `_handle_tool_result` further replaces `message_id` before the later text arrives), so the only emitted `TEXT_MESSAGE_START` carries a different id than the tool parent; Patch B's assertion would fail on the repro. Patch B's test exercises a different, adjacent case (tool+text in one response) rather than the tool-only case the issue is titled and reported for, so it is largely tautological with respect to the actual bug. On scope both are small; Patch B's refactor is arguably cleaner for the tool+text case (Patch A leaves an empty, unclosed `TEXT_MESSAGE_START` with no matching `TEXT_MESSAGE_END`, and would emit a second `TEXT_MESSAGE_START` for a tool-then-text response — a mild protocol wart), but correctness dominates: Patch A resolves the reported issue and Patch B does not touch its core scenario. Winner A.
The judge saw only the two patches, labelled A and B in random order, with issue links and version numbers removed. The reasoning above is shown verbatim.
Solve
34m 37s4 turns
Wall-clock time in the container
Full test suite
No new failures
11,744 passed · 574 skipped · 12,543 total
Contamination audit
Unknown
See the audit notes for this run
Patch size
45vs 157 merged
Changed lines, ours vs merged
The issue
View on GitHubInitial Checks
- I'm using the latest version of Pydantic AI
- I've searched for my issue in the issue tracker before opening this issue
Description
Description
For a model response containing only tool calls, AGUIEventStream emits aTOOL_CALL_START whose parentMessageId does not identify any assistant
message emitted in the stream.1.
before_response() creates a fresh internal message ID. Later,_handle_tool_call_start() uses that ID as parentMessageId, but noTEXT_MESSAGE_START is emitted when the response has no TextPart.
AG-UI clients cannot reliably reconstruct the assistant message that owns the
tool call. In our frontend, the resulting assistant message is later replayed
with this synthetic ID, while the following text response has a differentTEXT_MESSAGE_START ID. This produces a non-round-trippable history: a server
that persists the text-message ID cannot recognize the replayed assistant
message, and it re-sends the old tool call and tool result to the model.
This reproduces with Pydantic AI 2.31.0. The current source still has the
relevant behavior: