Backtestslitellmrun-20261018-182052#42869
[Bug]: streaming /v1/responses emits no reasoning item for signature-only thinking (Claude Fable 5.1 / Opus 5.5 default, Bedrock adaptive), so reasoning cannot be replayed
Our patch was judged better than the merged fix.
In this judgment, Patch A = our agent, Patch B = the merged fix.
Both patches hit the same two root-cause sites named in the issue: `_flush_thinking_block` now keeps a signed block with empty text, and `_ensure_output_item_for_chunk` opens the reasoning item when the delta carries a signed (`signature`) or redacted (`data`) thinking block even with no `reasoning_content`. The independent tests pass identically on both. Patch A goes further on protocol correctness for the exact scenario: when no reasoning text was streamed, it emits the reasoning item with `summary: []` instead of a `summary_text` part with empty text, and it skips `reasoning_summary_text.done` / `reasoning_summary_part.done` events that would otherwise close a summary part that was never opened. Patch B leaves those paths untouched, so a signature-only stream still emits summary-done events for an empty/nonexistent summary part and an item whose summary contains an empty text part; it works but is a slightly malformed Responses stream. Tests: A's parametrized test covers signature-only, text-with-signature, and redacted thinking in both sync and async modes, checks item ordering in both the stream and `response.completed`, verifies the same item id, and pins the empty-summary lifecycle; B adds a single signature-only case plus a chunk-builder test. Both chunk-builder tests are equivalent. Scope: B deletes a 93-line unrelated Vertex Gemma test file and re-adds a version elsewhere that depends on a `_make_gemma_vertex_response` helper not shown in the diff, which is unrelated churn with some risk of a broken test and nothing to do with the issue. A's extra changes are confined to the reasoning-close path the issue implicates. Craft is comparable; A's helper is a documented staticmethod, B's is a terse module-level function, both fine. A is the better fix.
Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.
Solve
37m 4s
Wall-clock time in the container
Full test suite
No new failures
64 passed · 0 skipped · 64 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
206vs 245 merged
Changed lines, ours vs merged
The issue
View on GitHubCheck for existing issues
- I have searched the existing issues and this is not a duplicate (no hits for "signature-only thinking" with streaming
/v1/responses,_ensure_output_item_for_chunk, or a missing reasoning item withencrypted_content).
What happened?
On streaming /v1/responses backed by an Anthropic model (the chat -> Responses bridge, litellm/responses/litellm_completion_transformation/streaming_iterator.py), a thinking block that has a signature but empty thinking text never becomes a reasoning output item. No response.output_item.added for reasoning is streamed, and response.completed has no reasoning item, so there is no encrypted_content to replay on the next turn. The reasoning tokens are still billed (usage.output_tokens_details.reasoning_tokens is non-zero)
That shape is common: Claude Fable 5.1 and Claude Opus 5.5 return {"type": "thinking", "thinking": "", "signature": "..."} by default, and Bedrock adaptive thinking returns it with or without a reasoning effort. Non-streaming /v1/responses is not affected
Two places key on the thinking text:
LiteLLMCompletionStreamingIterator._ensure_output_item_for_chunkopens the reasoning item only whendelta.reasoning_contentis non-empty. A signature-only block arrives asdelta.thinking_blocks=[{"type": "thinking", "thinking": "", "signature": "..."}]with noreasoning_content, so the first output item opened is the messageChunkProcessor.get_combined_thinking_content(_flush_thinking_block) keeps an assembled block only when it collected thinking text, so the signed block is also missing from the assembled response thatresponse.completedis built from
Measured through the proxy against real providers, streaming /v1/responses, is a reasoning item emitted?
| backend | no reasoning effort | reasoning.effort: "high" |
|---|---|---|
| Bedrock (Claude Opus 4.8, Fable 5.1, Opus 5.5) | no | no |
| Vertex AI (Claude Fable 5.1, Opus 5.5) | no | yes |
Vertex AI, deployment sets thinking: {"type": "adaptive", ...} | no | no |
Reasoning tokens were billed in every cell. The "yes" cell is the only one where the provider returns thinking text, which is why the bug can look provider-specific
User Flow
Before a (hypothetical) fix: a client using streaming Responses with a Claude model loses its reasoning between turns
- They send
POST https://litellm-domain/v1/responseswith"stream": trueand"include": ["reasoning.encrypted_content"]to abedrock/us.anthropic.claude-opus-4-8(or Fable 5.1 / Opus 5.5) deployment - The stream opens only a
messageitem, andresponse.completed.response.outputis[message], whileusage.output_tokens_details.reasoning_tokensis non-zero - They replay
outputon the next turn. There is no reasoning item to replay, so the model continues without its earlier reasoning
After a (hypothetical) fix: the same request streams reasoning then message, and response.completed.response.output carries the reasoning item with encrypted_content, ready to replay. Responses whose thinking arrives as text are unchanged
Proof the bug occurs
A litellm --config proxy on main @ a3d791f348, one deployment pointed at a stand-in upstream that streams an Anthropic message with one thinking block, then text:
model_list:
- model_name: anthropic-fable
litellm_params:
model: anthropic/claude-fable-5-1
api_base: http://127.0.0.1:9913
api_key: fake
curl -sN http://localhost:4000/v1/responses \
-H "Authorization: Bearer sk-1234" -H "content-type: application/json" \
-d '{"model":"anthropic-fable","stream":true,"input":"What is 2+2?","max_output_tokens":200,
"include":["reasoning.encrypted_content"]}'
| upstream thinking block | output_item.added streamed | response.completed output | reasoning encrypted_content |
|---|---|---|---|
thinking: "", signature set (default Fable 5.1 shape) | message | [message] | missing |
thinking: "Two plus two is four.", signature set | reasoning, message | [reasoning, message] | set |
The same two unit-level shapes are pinned in the linked PR's test, which fails on unpatched main
What part of LiteLLM is this about?
Proxy
What LiteLLM version are you on ?
v1.101.0-rc.1 (present on main @ a3d791f348)
Twitter / LinkedIn details
_No response_
The two patches
litellm/litellm_core_utils/streaming_chunk_builder_utils.pyindex d975c3551f..a086d8a64b 100644@@ -685,7 +685,9 @@ class ChunkProcessor: def _flush_thinking_block() -> None: nonlocal current_thinking_text_parts, current_signature- if len(current_thinking_text_parts) > 0 and current_signature:+ # A signed block is kept even with empty text: signature-only thinking+ # (e.g. Claude Fable 5.1 / Opus 5.5, Bedrock adaptive) must survive for replay.+ if current_signature: thinking_blocks.append( ChatCompletionThinkingBlock( type="thinking",litellm/responses/litellm_completion_transformation/streaming_iterator.pyindex 5173cd04a8..2887f3045b 100644@@ -839,12 +839,17 @@ class LiteLLMCompletionStreamingIterator(ResponsesAPIStreamingIterator): **{ "id": reasoning_item_id, "type": "reasoning",- "summary": [- {- "type": "summary_text",- "text": reasoning_content,- }- ],+ # Signature-only thinking has no summary text to report+ "summary": (+ [+ {+ "type": "summary_text",+ "text": reasoning_content,+ }+ ]+ if reasoning_content+ else []+ ), } ), )@@ -924,6 +929,25 @@ class LiteLLMCompletionStreamingIterator(ResponsesAPIStreamingIterator): else: raise StopAsyncIteration + @staticmethod+ def _delta_has_reasoning(delta: object) -> bool:+ """+ True when the delta carries reasoning: summary text, or a signed/redacted+ thinking block. Signature-only thinking (empty text, e.g. Claude Fable 5.1 /+ Opus 5.5 default, Bedrock adaptive) has no ``reasoning_content`` but must+ still open a reasoning item so its ``encrypted_content`` can be replayed.+ """+ if getattr(delta, "reasoning_content", None):+ return True+ thinking_blocks: Final[Sequence[object]] = getattr(delta, "thinking_blocks", None) or ()+ for block in thinking_blocks:+ if not isinstance(block, dict):+ continue+ fields = cast(dict[str, object], block) # cast-ok: thinking blocks are str-keyed TypedDicts+ if fields.get("signature") or fields.get("data"):+ return True+ return False+ def _ensure_output_item_for_chunk(self, chunk: ModelResponseStream) -> None: # Change: Never return a value, just enqueue output item events if self.sent_output_item_added_event:@@ -936,7 +960,7 @@ class LiteLLMCompletionStreamingIterator(ResponsesAPIStreamingIterator): self.sent_output_item_added_event = True # Reasoning-first- if hasattr(delta, "reasoning_content") and delta.reasoning_content:+ if self._delta_has_reasoning(delta): self._reasoning_active = True if self._cached_reasoning_item_id is None: self._cached_reasoning_item_id = f"rs_{uuid.uuid4()}"@@ -1026,21 +1050,25 @@ class LiteLLMCompletionStreamingIterator(ResponsesAPIStreamingIterator): ) reasoning_item_id = self._cached_reasoning_item_id - # Create text.done event first with its own sequence number- self._sequence_number += 1- text_done_event = self.create_reasoning_summary_text_done_event(- reasoning_item_id=reasoning_item_id,- reasoning_content=reasoning_content,- sequence_number=self._sequence_number,- )-- # Create part.done event second with its own sequence number- self._sequence_number += 1- part_done_event = self.create_reasoning_summary_part_done_event(- reasoning_item_id=reasoning_item_id,- reasoning_content=reasoning_content,- sequence_number=self._sequence_number,- )+ # Signature-only thinking streamed no summary text, so+ # there is no summary part to close.+ if reasoning_content:+ # Create text.done event first with its own sequence number+ self._sequence_number += 1+ text_done_event = self.create_reasoning_summary_text_done_event(+ reasoning_item_id=reasoning_item_id,+ reasoning_content=reasoning_content,+ sequence_number=self._sequence_number,+ )++ # Create part.done event second with its own sequence number+ self._sequence_number += 1+ part_done_event = self.create_reasoning_summary_part_done_event(+ reasoning_item_id=reasoning_item_id,+ reasoning_content=reasoning_content,+ sequence_number=self._sequence_number,+ )+ self._pending_response_events.extend([text_done_event, part_done_event]) self._sequence_number += 1 reasoning_output_item_done_event = self.create_reasoning_output_item_done_event(@@ -1048,13 +1076,7 @@ class LiteLLMCompletionStreamingIterator(ResponsesAPIStreamingIterator): reasoning_content=reasoning_content, sequence_number=self._sequence_number, )- self._pending_response_events.extend(- [- text_done_event,- part_done_event,- reasoning_output_item_done_event,- ]- )+ self._pending_response_events.append(reasoning_output_item_done_event) self._reasoning_done_emitted = True self._reasoning_active = False tests/unit/litellm_core_utils/test_streaming_chunk_builder_utils.pyindex af763da2d8..000fd5bf84 100644@@ -1795,3 +1795,29 @@ def test_calculate_usage_keeps_a_reported_count_over_a_later_chunks_zero() -> No ) assert (usage.prompt_tokens, usage.completion_tokens, usage.total_tokens) == (5, 17, 22)+++def test_get_combined_thinking_content_keeps_signature_only_block():+ """Claude Fable 5.1 / Opus 5.5 default thinking is signed with empty text; it must survive for replay."""+ chunks = [+ ModelResponseStream(+ id="chatcmpl-123",+ object="chat.completion.chunk",+ created=1234567890,+ model="claude-fable-5-1",+ choices=[+ StreamingChoices(+ index=0,+ delta=Delta(+ reasoning_content="",+ thinking_blocks=[{"type": "thinking", "thinking": "", "signature": "sig_fable"}],+ ),+ finish_reason=None,+ )+ ],+ )+ ]++ result = ChunkProcessor(chunks=chunks).get_combined_thinking_content(chunks)++ assert result == [{"type": "thinking", "thinking": "", "signature": "sig_fable"}]tests/unit/responses/litellm_completion_transformation/test_streaming_iterator_transformation.pyindex 8fbba0dbf8..60c6fbbe09 100644@@ -1131,3 +1131,99 @@ async def test_plain_text_stream_announces_exactly_one_message_item(sync_mode: b ResponsesAPIStreamEvents.OUTPUT_TEXT_DONE, ): assert event.item_id == message_item_adds[0].item.id+++def _thinking_block_chunk(thinking_block: dict, reasoning: str | None = None) -> ModelResponseStream:+ return ModelResponseStream(+ id=CHAT_COMPLETION_ID,+ created=1748575031,+ model="claude-fable-5-1",+ object="chat.completion.chunk",+ choices=[+ StreamingChoices(+ index=0,+ delta=Delta(role="assistant", reasoning_content=reasoning, thinking_blocks=[thinking_block]),+ finish_reason=None,+ )+ ],+ )+++# Chunk shapes the Anthropic / Bedrock stream parsers emit for one thinking block.+_SIGNATURE_ONLY_THINKING: Final = [+ _thinking_block_chunk({"type": "thinking", "thinking": "", "signature": "sig_fable"}, reasoning=""),+]+_TEXT_THINKING: Final = [+ _thinking_block_chunk(+ {"type": "thinking", "thinking": "Two plus two is four.", "signature": None},+ reasoning="Two plus two is four.",+ ),+ _thinking_block_chunk(+ {"type": "thinking", "thinking": "Two plus two is four.", "signature": "sig_fable"}, reasoning=""+ ),+]+_REDACTED_THINKING: Final = [+ _thinking_block_chunk({"type": "redacted_thinking", "data": "sig_fable"}),+]+++@pytest.mark.parametrize("sync_mode", [True, False])+@pytest.mark.parametrize(+ "thinking_chunks",+ [_SIGNATURE_ONLY_THINKING, _TEXT_THINKING, _REDACTED_THINKING],+ ids=["signature_only", "with_text", "redacted"],+)+@pytest.mark.asyncio+async def test_signed_thinking_streams_a_replayable_reasoning_item(thinking_chunks, sync_mode: bool):+ """+ Claude Fable 5.1 / Opus 5.5 (and Bedrock adaptive thinking) return signed thinking+ blocks with empty text. The stream must still open a reasoning item before the+ message, and response.completed must carry it with encrypted_content for replay.+ """+ iterator: Final = _build_iterator(+ [chunk.model_copy(deep=True) for chunk in thinking_chunks]+ + [_chunk("4"), _chunk("", finish_reason="stop")]+ )++ events: Final = await _collect_events(iterator, sync_mode)++ added_items: Final = [+ event.item for event in events if getattr(event, "type", None) == ResponsesAPIStreamEvents.OUTPUT_ITEM_ADDED+ ]+ assert [item.type for item in added_items] == ["reasoning", "message"]++ completed: Final = next(+ event for event in events if getattr(event, "type", None) == ResponsesAPIStreamEvents.RESPONSE_COMPLETED+ )+ output: Final = completed.response.output+ assert [item.type for item in output] == ["reasoning", "message"]+ assert output[0].id == added_items[0].id+ encrypted_blocks: Final = json.loads(output[0].encrypted_content)+ assert [block.get("signature") or block.get("data") for block in encrypted_blocks] == ["sig_fable"]+++@pytest.mark.asyncio+async def test_signature_only_reasoning_item_closes_with_empty_summary_before_message():+ iterator: Final = _build_iterator(+ [chunk.model_copy(deep=True) for chunk in _SIGNATURE_ONLY_THINKING]+ + [_chunk("4"), _chunk("", finish_reason="stop")]+ )++ events: Final = await _collect_events(iterator, sync_mode=False)++ event_types: Final = [getattr(event, "type", None) for event in events]+ assert ResponsesAPIStreamEvents.REASONING_SUMMARY_TEXT_DONE not in event_types+ assert ResponsesAPIStreamEvents.REASONING_SUMMARY_PART_DONE not in event_types+ item_lifecycle: Final = [+ (event.type, event.item.type, getattr(event.item, "summary", None))+ for event in events+ if getattr(event, "type", None)+ in (ResponsesAPIStreamEvents.OUTPUT_ITEM_ADDED, ResponsesAPIStreamEvents.OUTPUT_ITEM_DONE)+ ]+ assert [(event_type, item_type) for event_type, item_type, _ in item_lifecycle] == [+ (ResponsesAPIStreamEvents.OUTPUT_ITEM_ADDED, "reasoning"),+ (ResponsesAPIStreamEvents.OUTPUT_ITEM_DONE, "reasoning"),+ (ResponsesAPIStreamEvents.OUTPUT_ITEM_ADDED, "message"),+ (ResponsesAPIStreamEvents.OUTPUT_ITEM_DONE, "message"),+ ]+ assert item_lifecycle[1][2] == []
litellm/litellm_core_utils/streaming_chunk_builder_utils.pyindex d975c3551f3..67684a230e3 100644@@ -685,7 +685,7 @@ class ChunkProcessor: def _flush_thinking_block() -> None: nonlocal current_thinking_text_parts, current_signature- if len(current_thinking_text_parts) > 0 and current_signature:+ if current_signature: thinking_blocks.append( ChatCompletionThinkingBlock( type="thinking",litellm/responses/litellm_completion_transformation/streaming_iterator.pyindex 5173cd04a89..21a33c17ab8 100644@@ -73,6 +73,11 @@ def _output_items_with_id(items: tuple[Any, ...], item_type: str, item_id: str | ) +def _delta_has_signed_thinking_block(delta: object) -> bool:+ blocks: Final = getattr(delta, "thinking_blocks", None) or ()+ return any(isinstance(b, dict) and (b.get("signature") or b.get("data")) for b in blocks)++ class LiteLLMCompletionStreamingIterator(ResponsesAPIStreamingIterator): """ Async iterator for processing streaming responses from the Responses API.@@ -936,7 +941,7 @@ class LiteLLMCompletionStreamingIterator(ResponsesAPIStreamingIterator): self.sent_output_item_added_event = True # Reasoning-first- if hasattr(delta, "reasoning_content") and delta.reasoning_content:+ if (hasattr(delta, "reasoning_content") and delta.reasoning_content) or _delta_has_signed_thinking_block(delta): self._reasoning_active = True if self._cached_reasoning_item_id is None: self._cached_reasoning_item_id = f"rs_{uuid.uuid4()}"tests/test_litellm/llms/vertex_ai/vertex_gemma_models/test_vertex_gemma_transformation.pydeleted file mode 100644index 294d26b2e58..00000000000@@ -1,93 +0,0 @@-import json-from collections.abc import AsyncIterator-from types import SimpleNamespace-from typing import Any, cast--import httpx-import pytest--import litellm-from litellm.litellm_core_utils.streaming_handler import CustomStreamWrapper-from litellm.main import vertex_gemma_chat_completion-from litellm.types.llms.openai import OutputTextDeltaEvent, ResponseCompletedEvent, ResponsesAPIStreamingResponse--_VERTEX_URL = "https://example.invalid/v1/projects/test/locations/us-central1/endpoints/test:predict"-_MESSAGES = [{"role": "user", "content": "Reply exactly READY"}]-_FAKE_CREDENTIALS = "gemma-test-credentials"---def _vertex_response():- return {- "predictions": {- "id": "chatcmpl-stream-test",- "created": 1759863903,- "model": "google/gemma-3-12b-it",- "object": "chat.completion",- "choices": [{"index": 0, "finish_reason": "stop", "message": {"role": "assistant", "content": "READY"}}],- "usage": {"prompt_tokens": 14, "completion_tokens": 1, "total_tokens": 15},- }- }---@pytest.fixture(autouse=True)-def _cached_access_token():- """Serve a fake token from the handler's credential cache so no auth round-trip runs."""- cache = vertex_gemma_chat_completion._credentials_project_mapping- key = (_FAKE_CREDENTIALS, "test")- cache[key] = (SimpleNamespace(token="fake-token", expired=False), "test")- yield- cache.pop(key, None)---def test_sync_gemma_stream():- captured: dict[str, Any] = {}-- def handle(request: httpx.Request) -> httpx.Response:- captured["body"] = json.loads(request.content)- return httpx.Response(200, json=_vertex_response())-- stream = litellm.completion(- model="vertex_ai/gemma/test-model",- messages=_MESSAGES,- stream=True,- api_base=_VERTEX_URL,- vertex_project="test",- vertex_location="us-central1",- vertex_credentials=_FAKE_CREDENTIALS,- client=httpx.Client(transport=httpx.MockTransport(handle)),- )-- assert isinstance(stream, CustomStreamWrapper)- chunks = list(stream)-- assert "stream" not in captured["body"]["instances"][0]- assert len(chunks) == 2- assert chunks[0].choices[0].delta.content == "READY"- assert chunks[1].choices[0].finish_reason == "stop"---@pytest.mark.asyncio-async def test_async_gemma_responses_stream():- captured: dict[str, Any] = {}-- def handle(request: httpx.Request) -> httpx.Response:- captured["body"] = json.loads(request.content)- return httpx.Response(200, json=_vertex_response())-- response = await litellm.aresponses(- model="vertex_ai/gemma/test-model",- input="Reply exactly READY",- stream=True,- api_base=_VERTEX_URL,- vertex_project="test",- vertex_location="us-central1",- vertex_credentials=_FAKE_CREDENTIALS,- client=httpx.AsyncClient(transport=httpx.MockTransport(handle)),- )- events = [event async for event in cast(AsyncIterator[ResponsesAPIStreamingResponse], response)]-- assert "stream" not in captured["body"]["instances"][0]- assert "READY" in "".join(event.delta for event in events if isinstance(event, OutputTextDeltaEvent))- assert isinstance(events[-1], ResponseCompletedEvent)- assert events[-1].response.usage is not None- assert events[-1].response.usage.total_tokens == 15tests/unit/litellm_core_utils/test_streaming_chunk_builder_utils.pyindex af763da2d87..aaf877df364 100644@@ -236,6 +236,31 @@ def test_get_combined_thinking_content_preserves_interleaved_blocks(): assert result[2]["signature"] == "sig_block2" +def test_get_combined_thinking_content_keeps_signed_block_without_thinking_text():+ chunks: Final = [+ ModelResponseStream(+ id="chatcmpl-123",+ object="chat.completion.chunk",+ created=1234567890,+ model="claude-sonnet-4-20250514",+ choices=[+ StreamingChoices(+ index=0,+ delta=Delta(thinking_blocks=[{"type": "thinking", "thinking": "", "signature": "sig_only"}]),+ finish_reason=None,+ )+ ],+ )+ ]++ result: Final = ChunkProcessor(chunks=chunks).get_combined_thinking_content(chunks)++ assert result is not None+ assert [(block["type"], block["thinking"], block["signature"]) for block in result] == [+ ("thinking", "", "sig_only")+ ]++ def test_cache_read_input_tokens_retained(): chunk1 = ModelResponseStream( id="chatcmpl-95aabb85-c39f-443d-ae96-0370c404d70c",tests/unit/llms/vertex_ai/vertex_gemma_models/test_vertex_gemma_transformation.pyindex 97f4f290958..92e684e42c8 100644@@ -1303,3 +1303,81 @@ class TestVertexGemmaCompletion: mock_async_post.assert_awaited_once() assert mock_async_post.call_args.kwargs["client"] is None assert response.choices[0].message.content == "default async handler fallback"+++_GEMMA_VERTEX_URL = "https://example.invalid/v1/projects/test/locations/us-central1/endpoints/test:predict"+_FAKE_GEMMA_CREDENTIALS = "gemma-test-credentials"+++@pytest.fixture+def _gemma_cached_access_token():+ """Serve a fake token from the handler's credential cache so no auth round-trip runs."""+ from types import SimpleNamespace++ from litellm.main import vertex_gemma_chat_completion++ cache = vertex_gemma_chat_completion._credentials_project_mapping+ key = (_FAKE_GEMMA_CREDENTIALS, "test")+ cache[key] = (SimpleNamespace(token="fake-token", expired=False), "test")+ yield+ cache.pop(key, None)+++def test_sync_gemma_stream(_gemma_cached_access_token):+ import httpx++ from litellm.litellm_core_utils.streaming_handler import CustomStreamWrapper++ captured = {}++ def handle(request):+ captured["body"] = json.loads(request.content)+ return httpx.Response(200, json=_make_gemma_vertex_response(content="READY", total_tokens=15))++ stream = litellm.completion(+ model="vertex_ai/gemma/test-model",+ messages=[{"role": "user", "content": "Reply exactly READY"}],+ stream=True,+ api_base=_GEMMA_VERTEX_URL,+ vertex_project="test",+ vertex_location="us-central1",+ vertex_credentials=_FAKE_GEMMA_CREDENTIALS,+ client=httpx.Client(transport=httpx.MockTransport(handle)),+ )++ assert isinstance(stream, CustomStreamWrapper)+ chunks = list(stream)++ assert "stream" not in captured["body"]["instances"][0]+ assert len(chunks) == 2+ assert chunks[0].choices[0].delta.content == "READY"+ assert chunks[1].choices[0].finish_reason == "stop"+++@pytest.mark.asyncio+async def test_async_gemma_responses_stream(_gemma_cached_access_token):+ import httpx++ captured = {}++ def handle(request):+ captured["body"] = json.loads(request.content)+ return httpx.Response(200, json=_make_gemma_vertex_response(content="READY", total_tokens=15))++ response = await litellm.aresponses(+ model="vertex_ai/gemma/test-model",+ input="Reply exactly READY",+ stream=True,+ api_base=_GEMMA_VERTEX_URL,+ vertex_project="test",+ vertex_location="us-central1",+ vertex_credentials=_FAKE_GEMMA_CREDENTIALS,+ client=httpx.AsyncClient(transport=httpx.MockTransport(handle)),+ )+ events = [event async for event in cast(AsyncIterator[ResponsesAPIStreamingResponse], response)]++ assert "stream" not in captured["body"]["instances"][0]+ assert "READY" in "".join(event.delta for event in events if isinstance(event, OutputTextDeltaEvent))+ assert isinstance(events[-1], ResponseCompletedEvent)+ assert events[-1].response.usage is not None+ assert events[-1].response.usage.total_tokens == 15tests/unit/responses/litellm_completion_transformation/test_streaming_iterator_transformation.pyindex 8fbba0dbf87..041bcf1b6d7 100644@@ -978,6 +978,25 @@ def _reasoning_chunk(reasoning: str, finish_reason: str | None = None) -> ModelR ) +def _signature_only_thinking_chunk(signature: str) -> ModelResponseStream:+ return ModelResponseStream(+ id=CHAT_COMPLETION_ID,+ created=1748575031,+ model="claude-haiku-4-5",+ object="chat.completion.chunk",+ choices=[+ StreamingChoices(+ index=0,+ delta=Delta(+ role="assistant",+ thinking_blocks=[{"type": "thinking", "thinking": "", "signature": signature}],+ ),+ finish_reason=None,+ )+ ],+ )++ async def _collect_events( iterator: LiteLLMCompletionStreamingIterator, sync_mode: bool ) -> list[BaseLiteLLMOpenAIResponseObject]:@@ -1015,6 +1034,27 @@ async def test_tool_only_stream_emits_no_message_item_events(sync_mode: bool): assert any(getattr(event, "type", None) == ResponsesAPIStreamEvents.RESPONSE_COMPLETED for event in events) +@pytest.mark.parametrize("sync_mode", [True, False])+@pytest.mark.asyncio+async def test_signature_only_thinking_streams_a_replayable_reasoning_item(sync_mode: bool):+ iterator: Final = _build_iterator([_signature_only_thinking_chunk("sig_only"), _chunk("4", finish_reason="stop")])++ events: Final = await _collect_events(iterator, sync_mode)++ added_item_types: Final = [+ event.item.type+ for event in events+ if getattr(event, "type", None) == ResponsesAPIStreamEvents.OUTPUT_ITEM_ADDED+ ]+ completed: Final = next(+ event for event in events if getattr(event, "type", None) == ResponsesAPIStreamEvents.RESPONSE_COMPLETED+ )+ reasoning_items: Final = [item for item in completed.response.output if getattr(item, "type", None) == "reasoning"]+ assert added_item_types[0] == "reasoning"+ assert len(reasoning_items) == 1+ assert json.loads(reasoning_items[0].encrypted_content)[0]["signature"] == "sig_only"++ @pytest.mark.parametrize("sync_mode", [True, False]) @pytest.mark.asyncio async def test_reasoning_then_text_announces_message_item_before_text_events(sync_mode: bool):