Backtestslitellmrun-20261111-220245#25322

Issue #25322 Merged fix PR # Run 11 Nov 2026 · run-20261111-220245 BerriAI/litellm

[Bug]: Gemini models degenerate in multi-turn tool-calling via /v1/messages — thoughtSignature not propagated from thought parts

Better

Our patch was judged better than the merged fix.

In this judgment, Patch B = our agent, Patch A = the merged fix.

The issue reports two distinct failures and names two files. Patch A fixes only Failure 1 (thought-part signature carried onto the functionCall in `_transform_parts`) and does nothing in `adapters/transformation.py`, so unsigned thinking blocks are still echoed back to Gemini; the four independent tests covering Failure 2 fail on base and still fail on Patch A, while all eight pass on Patch B. Patch B fixes both root causes: it carries the thought-part signature onto the first functionCall (non-streaming and, additionally, across streaming chunks where the signed thought part arrives before the functionCall), and it drops unsigned thinking blocks when the target is Gemini, gated so DeepSeek/Moonshot reasoning is still replayed. It also prevents the same signature from being sent twice on replay when the response's thinking block and tool call both carry it. Both patches add real regression tests; B's are broader (adapter tests parametrized over gemini/vertex providers, non-gemini preservation, end-to-end transform_response round-trip, and a streaming iterator test). B is larger and the streaming/pending-signature bookkeeping adds some complexity, but it is targeted at the issue's scenarios rather than unrelated churn. A is a clean partial fix; B is a complete one.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

24m 20s

Wall-clock time in the container

Full test suite

No new failures

1,860 passed · 9 skipped · 1,869 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

393vs 86 merged

Changed lines, ours vs merged

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

When using Gemini models through LiteLLM's /v1/messages endpoint for multi-turn tool-calling conversations (e.g. Claude Code), the model degenerates after several turns — producing garbage short responses ("ja"), repetition loops ("Executing. Done. Going. Now."), or stopping mid-task. This happens because Gemini's thoughtSignature is not properly round-tripped between turns.

There are two related failures:


Failure 1: thoughtSignature on thought parts is never captured

Gemini's API requires that the thoughtSignature from the previous assistant turn be echoed back on the next turn for multi-turn thinking coherence. LiteLLM's _transform_parts() in vertex_and_google_ai_studio_gemini.py only reads thoughtSignature from the part that contains functionCall:

# Current code in _transform_parts:
if "functionCall" in part:
    _function_chunk = { ... }
    thought_signature = part.get("thoughtSignature")  # ← only checks THIS part

But Gemini often places thoughtSignature on a separate thought part (thought: true), not on the functionCall part. When the signature lives on the thought part, it's never captured, never embedded in the tool_call_id, and never sent back on the following turn. Without the signature, Gemini loses coherence and degenerates.

Example Gemini response with signature on thought part (not functionCall part):

{
  "candidates": [{
    "content": {
      "parts": [
        {"thought": true, "text": "I need to list files...", "thoughtSignature": "ErcNCrQN..."},
        {"functionCall": {"name": "Bash", "args": {"command": "ls -R"}}}
      ]
    }
  }]
}

The thoughtSignature is on parts[0] (thought part), but _transform_parts only checks parts[1] (the functionCall part) — so the signature is lost.

Fix: Before iterating parts, collect all thoughtSignature values from any part. When a functionCall part has no signature of its own, fall back to the first available collected signature:

# Collect all signatures from any part in the response
_all_signatures = [
    p["thoughtSignature"] for p in parts if p.get("thoughtSignature")
]

for part in parts:
    if "functionCall" in part:
        _function_chunk = { ... }
        # Fall back to signature from thought parts if functionCall has none
        thought_signature = part.get("thoughtSignature") or (
            _all_signatures[0] if _all_signatures else None
        )

Failure 2: Unsigned thinking blocks echoed back cause repetition loops

The very first thinking turn Gemini produces often has no thoughtSignature yet (the signature appears starting from the second thinking turn). When this unsigned thinking block is included in conversation history on subsequent turns, Gemini outputs its reasoning as visible plain text and then enters repetition loops — consuming the entire token budget with patterns like "Executing. Done. Going. Now." before making any tool calls.

This happens in adapters/transformation.py → translate_anthropic_messages_to_openai(), in the request path where conversation history is converted for Gemini. The thinking block handler unconditionally includes all thinking blocks:

elif content.get("type") == "thinking":
    thinking_block = ChatCompletionThinkingBlock(
        type="thinking",
        thinking=content.get("thinking") or "",
        signature=content.get("signature") or "",  # ← empty string for unsigned blocks
    )
    thinking_blocks.append(thinking_block)  # ← always appended, even without signature

Fix: Skip thinking blocks that have an empty or null signature:

elif content.get("type") == "thinking":
    _thinking_sig = content.get("signature") or ""
    if _thinking_sig:  # Only include signed thinking blocks
        thinking_block = ChatCompletionThinkingBlock(
            type="thinking",
            thinking=content.get("thinking") or "",
            signature=_thinking_sig,
        )
        thinking_blocks.append(thinking_block)

Impact

Any user routing Claude Code (or any Anthropic-format client with thinking enabled) through LiteLLM to Gemini models will experience progressive degeneration in multi-turn tool-calling conversations. The model works fine for the first 1-3 turns, then starts producing incoherent or repetitive output. This is particularly visible in agentic coding workflows where 10+ tool-calling turns are common.

Files affected

  • litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py — _transform_parts() (Failure 1)
  • litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py — translate_anthropic_messages_to_openai() (Failure 2)

Steps to Reproduce

  1. Configure LiteLLM with a Gemini model that supports thinking: ```yaml model_list:
  2. model_name: gemini-3-flash-preview litellm_params: model: gemini/gemini-3-flash-preview api_key: os.environ/GEMINI_API_KEY litellm_settings: drop_params: true modify_params: true ```
  1. Point Claude Code at the proxy: bash export ANTHROPIC_BASE_URL="http://localhost:4000" export ANTHROPIC_AUTH_TOKEN="$LITELLM_MASTER_KEY"
  1. Run a multi-step coding task that requires 5+ tool calls: bash claude --model gemini-3-flash-preview -p "create a hello world web app with index.html, styles.css, and script.js"
  1. Observe: the model works for the first few turns, then starts producing:
  2. Very short garbage responses ("ja", single characters)
  3. Repetition loops ("Executing. Done. Going. Now. Executing. Done...")
  4. Visible reasoning text in the output instead of tool calls
  5. Stops making progress and consumes tokens without producing useful output
  1. Inspect the request payloads sent to Gemini — the thoughtSignature from thought parts is missing in the conversation history, and unsigned thinking blocks are being echoed back.

Relevant log output

# Gemini response showing thoughtSignature on thought part (NOT on functionCall part):
parts: [
  {"thought": true, "text": "I need to...", "thoughtSignature": "ErcNCrQN..."},
  {"functionCall": {"name": "Bash", "args": {"command": "ls"}}}
]

# LiteLLM _transform_parts only checks functionCall part for signature:
thought_signature = part.get("thoughtSignature")  # → None (it's on the other part)

# Result: tool_call_id has no embedded signature, next turn sends no signature back
# Gemini loses thinking coherence and degenerates

# Failure 2 — unsigned thinking block in conversation history causes repetition:
# First thinking turn has signature="" (empty), gets echoed back on turn 2+
# Gemini sees unsigned thinking block and outputs reasoning as plain text:
"Executing. Done. Going. Now. Executing. Done. Going. Now..."
# (repeats until token budget exhausted)

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.82.3-stable.patch.2 (also reproduces on v1.81.14-stable and v1.83.4-nightly)

Twitter / LinkedIn details

_No response_

The two patches

Our patch+385 −8
litellm/llms/anthropic/experimental_pass_through/adapters/transformation.pyindex db89066213..adbf44131b 100644@@ -415,8 +415,10 @@ class LiteLLMAnthropicMessagesAdapter:         self,         messages: list[AllAnthropicPassThroughMessageValues],         model: str | None = None,+        custom_llm_provider: str | None = None,     ) -> list:         new_messages: Final[list[AllMessageValues]] = []+        drop_unsigned_thinking: Final = self.is_gemini_target(model, custom_llm_provider)         for m in messages:             user_message: ChatCompletionUserMessage | None = None             tool_message_list: list[ChatCompletionToolMessage] = []@@ -533,12 +535,18 @@ class LiteLLMAnthropicMessagesAdapter:                                 # redacted_thinking blocks, and anthropic_messages_pt replays                                 # these verbatim at content[0], so carrying one here (or                                 # inventing an empty one) is a guaranteed 400 on the way back.-                                thinking_block = ChatCompletionThinkingBlock(-                                    type="thinking",-                                    thinking=content.get("thinking") or "",-                                    signature=content.get("signature") or "",-                                )-                                thinking_blocks.append(thinking_block)+                                # Gemini degenerates (leaks its reasoning as text and loops) when+                                # an unsigned thought is replayed, so unsigned thinking blocks are+                                # dropped for Gemini targets. Other reasoning providers (DeepSeek,+                                # Moonshot) never sign and still need the text as reasoning_content.+                                thinking_signature = content.get("signature") or ""+                                if thinking_signature or not drop_unsigned_thinking:+                                    thinking_block = ChatCompletionThinkingBlock(+                                        type="thinking",+                                        thinking=content.get("thinking") or "",+                                        signature=thinking_signature,+                                    )+                                    thinking_blocks.append(thinking_block)                             elif content.get("type") == "redacted_thinking":                                 redacted_thinking_block = ChatCompletionRedactedThinkingBlock(                                     type="redacted_thinking",@@ -620,6 +628,15 @@ class LiteLLMAnthropicMessagesAdapter:         model_lower: Final = model.lower()         return "anthropic" in model_lower or "claude" in model_lower +    @staticmethod+    def is_gemini_target(model: str | None, custom_llm_provider: str | None) -> bool:+        """+        Check if the request is routed to a Gemini model (Google AI Studio, or Gemini on Vertex AI).+        """+        if custom_llm_provider == "gemini":+            return True+        return model is not None and "gemini" in model.lower()+     @staticmethod     def is_bedrock_arn_model(model: str) -> bool:         """@@ -1171,6 +1188,7 @@ class LiteLLMAnthropicMessagesAdapter:         new_messages = self.translate_anthropic_messages_to_openai(             messages=messages_list,             model=anthropic_message_request.get("model"),+            custom_llm_provider=custom_llm_provider,         )         ## ADD SYSTEM MESSAGE TO MESSAGES         self._add_system_message_to_messages(new_messages, anthropic_message_request)litellm/llms/vertex_ai/gemini/transformation.pyindex 13e2238fdf..780199bc7a 100644@@ -885,10 +885,16 @@ def _gemini_convert_messages_with_history(                 if reasoning_content is not None:                     assistant_content.append(PartType(thought=True, text=reasoning_content))                 if thinking_blocks is not None:+                    # A thought part's signature that was moved onto this message's+                    # first tool call is replayed there; a second copy on the thought+                    # part would double-bill (see _collect_tool_call_thought_signatures)+                    thinking_tool_call_signatures = _collect_tool_call_thought_signatures(assistant_msg)                     for block in thinking_blocks:                         if block["type"] == "thinking":                             block_thinking_str = block.get("thinking")                             block_signature = block.get("signature")+                            if block_signature in thinking_tool_call_signatures:+                                continue                             if block_thinking_str is not None and block_signature is not None:                                 try:                                     assistant_content.append(litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.pyindex 69fe5678de..265b4ef849 100644@@ -1549,21 +1549,38 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):         parts: list[HttpxPartType],         cumulative_tool_call_idx: int,         is_function_call: bool | None,+        fallback_thought_signature: str | None = None,     ) -> tuple[         ChatCompletionToolCallFunctionChunk | None,         list[ChatCompletionToolCallChunk] | None,         int,     ]:+        """+        Gemini may put the step's `thoughtSignature` on a separate `thought: true`+        part instead of on the `functionCall` part. When no functionCall part+        carries its own signature, the first signature found on a thought part (or+        `fallback_thought_signature`, carried over from an earlier streaming chunk)+        is attached to the first functionCall, so it is echoed back on the next+        turn. Signatures on plain text parts stay on the text part.+        """         function: ChatCompletionToolCallFunctionChunk | None = None         _tools: Final[list[ChatCompletionToolCallChunk]] = []+        pending_signature: str | None = None+        if not any("functionCall" in part and part.get("thoughtSignature") for part in parts):+            pending_signature = next(+                (signature for part in parts if part.get("thought") and (signature := part.get("thoughtSignature"))),+                fallback_thought_signature,+            )         for part in parts:             if "functionCall" in part:                 _function_chunk: ChatCompletionToolCallFunctionChunk = {                     "name": part["functionCall"]["name"],                     "arguments": json.dumps(part["functionCall"]["args"], ensure_ascii=False),                 }-                # Extract thought signature if present-                thought_signature = part.get("thoughtSignature")+                # Extract thought signature if present, else use the step's signature+                # from a sibling part (only for the first functionCall)+                thought_signature = part.get("thoughtSignature") or pending_signature+                pending_signature = None                 # Gemini 3.5+ returns a stable `id` per function call to enable                 # strict response matching. Preserve it as the OpenAI                 # tool_call_id so it can be echoed back unchanged.@@ -2190,10 +2207,14 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):         model_response: Union[ModelResponse, "ModelResponseStream"],         standard_optional_params: dict,         cumulative_tool_call_index: int = 0,+        fallback_thought_signature: str | None = None,     ) -> tuple[list[dict], list[dict], list, list, int]:         """         Helper method to process candidates and extract metadata +        `fallback_thought_signature` is a signature seen on an earlier streaming+        chunk that has not yet been attached to a function call.+         Returns:             grounding_metadata: List[dict]             url_context_metadata: List[dict]@@ -2291,6 +2312,7 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):                     parts=candidate["content"]["parts"],                     cumulative_tool_call_idx=cumulative_tool_call_index,                     is_function_call=is_function_call(standard_optional_params),+                    fallback_thought_signature=fallback_thought_signature,                 )              if "logprobsResult" in candidate:@@ -3095,6 +3117,9 @@ class ModelResponseIterator:         self.is_function_call = check_is_function_call(logging_obj)         self.cumulative_tool_call_index: int = 0         self.has_seen_tool_calls: bool = False+        # Signature streamed on a `thought: true` part before the step's first+        # functionCall arrived in a later chunk.+        self.pending_thought_signature: str | None = None      @property     def accumulated_json(self) -> str:@@ -3134,6 +3159,9 @@ class ModelResponseIterator:         _candidates: list[Candidates],         model_response: "ModelResponseStream",     ) -> tuple[list[dict], list[dict], list[dict], list[dict]]:+        chunk_parts: Final[list[HttpxPartType]] = [+            part for candidate in _candidates for part in (candidate.get("content") or {}).get("parts") or []+        ]         (             grounding_metadata,             url_context_metadata,@@ -3145,7 +3173,21 @@ class ModelResponseIterator:             model_response,             self.logging_obj.optional_params,             cumulative_tool_call_index=self.cumulative_tool_call_index,+            fallback_thought_signature=self.pending_thought_signature,         )+        if any("functionCall" in part for part in chunk_parts):+            # The step's first functionCall has been emitted (with its own or the+            # carried-over signature), so the carried-over signature is consumed.+            self.pending_thought_signature = None+        elif self.pending_thought_signature is None:+            self.pending_thought_signature = next(+                (+                    signature+                    for part in chunk_parts+                    if part.get("thought") and (signature := part.get("thoughtSignature"))+                ),+                None,+            )          # Track whether tool_calls have been seen across streaming chunks.         # Gemini sends tool_calls and finishReason in separate chunks,tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.pyindex c59ec70b01..9767f2edf0 100644@@ -437,6 +437,102 @@ def test_translate_anthropic_messages_to_openai_sets_reasoning_content():     assert result[1]["content"] == "Denver."  +def _assistant_turns_with_unsigned_first_thinking():+    return [+        AnthropicMessagesUserMessageParam(+            role="user",+            content=[{"type": "text", "text": "Create a hello world app."}],+        ),+        AnthopicMessagesAssistantMessageParam(+            role="assistant",+            content=[+                {"type": "thinking", "thinking": "First turn, Gemini sent no signature yet.", "signature": ""},+                {"type": "tool_use", "id": "toolu_1", "name": "Bash", "input": {"command": "ls"}},+            ],+        ),+        AnthropicMessagesUserMessageParam(+            role="user",+            content=[{"type": "tool_result", "tool_use_id": "toolu_1", "content": "README.md"}],+        ),+        AnthopicMessagesAssistantMessageParam(+            role="assistant",+            content=[+                {"type": "thinking", "thinking": "Second turn is signed.", "signature": "sig_turn_2"},+                {"type": "tool_use", "id": "toolu_2", "name": "Bash", "input": {"command": "pwd"}},+            ],+        ),+    ]+++@pytest.mark.parametrize(+    "model, custom_llm_provider",+    [+        ("gemini/gemini-3-flash-preview", None),+        ("gemini-3-flash-preview", "gemini"),+        ("vertex_ai/gemini-3-pro-preview", "vertex_ai"),+    ],+)+def test_translate_anthropic_messages_to_openai_drops_unsigned_thinking_for_gemini(model, custom_llm_provider):+    """Replaying Gemini's unsigned first thinking turn makes it leak its reasoning as text and+    loop, so it must not reach Gemini; signed thinking must still round-trip."""+    result = LiteLLMAnthropicMessagesAdapter().translate_anthropic_messages_to_openai(+        messages=_assistant_turns_with_unsigned_first_thinking(),+        model=model,+        custom_llm_provider=custom_llm_provider,+    )++    first_assistant = result[1]+    assert first_assistant["role"] == "assistant"+    assert first_assistant.get("thinking_blocks") is None+    assert "reasoning_content" not in first_assistant+    assert first_assistant["tool_calls"][0]["id"] == "toolu_1"++    second_assistant = result[3]+    assert second_assistant["role"] == "assistant"+    assert second_assistant["thinking_blocks"] == [+        {"type": "thinking", "thinking": "Second turn is signed.", "signature": "sig_turn_2"}+    ]+    assert second_assistant["reasoning_content"] == "Second turn is signed."+++def test_translate_anthropic_to_openai_drops_unsigned_thinking_for_gemini_provider():+    """End-to-end /v1/messages translation: the provider resolved by the router is used even+    when the request model is a bare alias."""+    from litellm.types.llms.anthropic import AnthropicMessagesRequest++    request = AnthropicMessagesRequest(+        model="gemini-3-flash-preview",+        messages=_assistant_turns_with_unsigned_first_thinking(),+        max_tokens=1024,+    )++    openai_request, _ = LiteLLMAnthropicMessagesAdapter().translate_anthropic_to_openai(+        anthropic_message_request=request,+        custom_llm_provider="gemini",+    )++    assistant_messages = [m for m in openai_request["messages"] if m["role"] == "assistant"]+    assert assistant_messages[0].get("thinking_blocks") is None+    assert [b["signature"] for b in assistant_messages[1]["thinking_blocks"]] == ["sig_turn_2"]+++@pytest.mark.parametrize(+    "model, custom_llm_provider",+    [("deepseek/deepseek-reasoner", None), ("kimi-k2-thinking", "moonshot")],+)+def test_translate_anthropic_messages_to_openai_keeps_unsigned_thinking_for_non_gemini(model, custom_llm_provider):+    """DeepSeek / Moonshot reasoning never carries a signature but must still be replayed as+    reasoning_content."""+    result = LiteLLMAnthropicMessagesAdapter().translate_anthropic_messages_to_openai(+        messages=_assistant_turns_with_unsigned_first_thinking(),+        model=model,+        custom_llm_provider=custom_llm_provider,+    )++    assert result[1]["reasoning_content"] == "First turn, Gemini sent no signature yet."+    assert result[1]["thinking_blocks"][0]["signature"] == ""++ def test_translate_anthropic_messages_to_openai_sets_no_reasoning_content_without_thinking():     anthropic_messages = [         AnthopicMessagesAssistantMessageParam(tests/test_litellm/llms/vertex_ai/gemini/test_thought_signature_in_tool_call_id.pyindex 208cba519f..1e14680966 100644@@ -285,3 +285,218 @@ def test_parallel_tool_calls_with_signatures(enable_preview_features):     assert THOUGHT_SIGNATURE_SEPARATOR not in tools[1]["id"]     sig2 = _get_thought_signature_from_tool({"id": tools[1]["id"], "type": "function"})     assert sig2 is None+++def test_signature_on_thought_part_is_attached_to_function_call():+    """Gemini may put thoughtSignature on a separate thought part instead of the+    functionCall part; it must still be attached to the tool call (and its id)."""+    signature = "ErcNCrQN_signature_on_thought_part"+    gemini_parts = [+        HttpxPartType(thought=True, text="I need to list files...", thoughtSignature=signature),+        HttpxPartType(functionCall={"name": "Bash", "args": {"command": "ls -R"}}),+    ]++    _, tools, _ = VertexGeminiConfig._transform_parts(+        parts=gemini_parts,+        cumulative_tool_call_idx=0,+        is_function_call=False,+    )++    assert tools is not None+    assert len(tools) == 1+    assert tools[0].get("provider_specific_fields", {}).get("thought_signature") == signature+    assert _get_thought_signature_from_tool({"id": tools[0]["id"], "type": "function"}) == signature+++def test_signature_on_thought_part_only_attached_to_first_parallel_call():+    """Gemini expects the step signature only on the first functionCall of a step."""+    signature = "signature_on_thought_part"+    gemini_parts = [+        HttpxPartType(thought=True, text="thinking", thoughtSignature=signature),+        HttpxPartType(functionCall={"name": "get_temperature", "args": {"location": "Paris"}}),+        HttpxPartType(functionCall={"name": "get_temperature", "args": {"location": "London"}}),+    ]++    _, tools, _ = VertexGeminiConfig._transform_parts(+        parts=gemini_parts,+        cumulative_tool_call_idx=0,+        is_function_call=False,+    )++    assert tools is not None+    assert len(tools) == 2+    assert _get_thought_signature_from_tool({"id": tools[0]["id"], "type": "function"}) == signature+    assert THOUGHT_SIGNATURE_SEPARATOR not in tools[1]["id"]+    assert "provider_specific_fields" not in tools[1]+++def test_function_call_own_signature_wins_over_thought_part_signature():+    thought_signature = "signature_on_thought_part"+    call_signature = "signature_on_function_call"+    gemini_parts = [+        HttpxPartType(thought=True, text="thinking", thoughtSignature=thought_signature),+        HttpxPartType(+            functionCall={"name": "get_temperature", "args": {"location": "Paris"}},+            thoughtSignature=call_signature,+        ),+    ]++    _, tools, _ = VertexGeminiConfig._transform_parts(+        parts=gemini_parts,+        cumulative_tool_call_idx=0,+        is_function_call=False,+    )++    assert tools is not None+    assert tools[0].get("provider_specific_fields", {}).get("thought_signature") == call_signature+++def test_signature_on_thought_part_with_legacy_function_call_mode():+    signature = "signature_on_thought_part"+    gemini_parts = [+        HttpxPartType(thought=True, text="thinking", thoughtSignature=signature),+        HttpxPartType(functionCall={"name": "get_temperature", "args": {"location": "Paris"}}),+    ]++    function, _, _ = VertexGeminiConfig._transform_parts(+        parts=gemini_parts,+        cumulative_tool_call_idx=0,+        is_function_call=True,+    )++    assert function is not None+    assert function.get("provider_specific_fields", {}).get("thought_signature") == signature+++def test_signature_on_thought_part_round_trips_in_completion_response():+    """Full non-streaming response transform: the signature from the thought part+    must end up on the OpenAI tool call so it is echoed back on the next turn."""+    import httpx+    from unittest.mock import MagicMock++    from litellm.types.utils import ModelResponse++    signature = "ErcNCrQN_signature_on_thought_part"+    raw_response = httpx.Response(+        status_code=200,+        json={+            "candidates": [+                {+                    "content": {+                        "role": "model",+                        "parts": [+                            {"thought": True, "text": "I need to list files...", "thoughtSignature": signature},+                            {"functionCall": {"name": "Bash", "args": {"command": "ls -R"}}},+                        ],+                    },+                    "finishReason": "STOP",+                    "index": 0,+                }+            ],+            "usageMetadata": {"promptTokenCount": 10, "candidatesTokenCount": 5, "totalTokenCount": 15},+        },+        request=httpx.Request("POST", "https://generativelanguage.googleapis.com"),+    )+    logging_obj = MagicMock()++    result = VertexGeminiConfig().transform_response(+        model="gemini-3-flash-preview",+        raw_response=raw_response,+        model_response=ModelResponse(),+        logging_obj=logging_obj,+        request_data={},+        messages=[{"role": "user", "content": "list files"}],+        optional_params={},+        litellm_params={},+        encoding=None,+    )++    tool_calls = result.choices[0].message.tool_calls+    assert tool_calls is not None and len(tool_calls) == 1+    assert _get_thought_signature_from_tool(tool_calls[0].model_dump()) == signature++    # Echoing the tool call back sends the signature on the functionCall part+    gemini_part = convert_to_gemini_tool_call_invoke(+        {"role": "assistant", "content": None, "tool_calls": [tool_calls[0].model_dump()]}+    )+    assert gemini_part[0]["thoughtSignature"] == signature++    # Replaying the whole assistant message (thinking blocks + tool calls) must+    # send the signature exactly once, on the functionCall part+    from litellm.llms.vertex_ai.gemini.transformation import (+        _gemini_convert_messages_with_history,+    )++    assistant_message = result.choices[0].message.model_dump()+    assert assistant_message["thinking_blocks"][0]["signature"] == signature+    contents = _gemini_convert_messages_with_history(+        messages=[+            {"role": "user", "content": "list files"},+            assistant_message,+            {"role": "tool", "tool_call_id": tool_calls[0].id, "content": "a.txt"},+        ],+        model="gemini-3-flash-preview",+    )+    model_parts = next(c["parts"] for c in contents if c["role"] == "model")+    signed_parts = [p for p in model_parts if p.get("thoughtSignature") == signature]+    assert len(signed_parts) == 1+    assert "function_call" in signed_parts[0] or "functionCall" in signed_parts[0]+++def test_streaming_signature_on_earlier_thought_chunk_is_attached_to_function_call():+    """When streaming, the signed thought part can arrive in an earlier chunk than+    the functionCall; the signature must still be attached to that tool call."""+    from unittest.mock import MagicMock++    from litellm.llms.vertex_ai.gemini.vertex_and_google_ai_studio_gemini import (+        ModelResponseIterator,+    )++    signature = "ErcNCrQN_streamed_signature"+    logging_obj = MagicMock()+    logging_obj.optional_params = {}+    iterator = ModelResponseIterator(streaming_response=iter([]), sync_stream=True, logging_obj=logging_obj)++    thought_chunk = {+        "candidates": [+            {+                "content": {+                    "role": "model",+                    "parts": [{"thought": True, "text": "I need to list files...", "thoughtSignature": signature}],+                },+                "index": 0,+            }+        ]+    }+    first_call_chunk = {+        "candidates": [+            {+                "content": {+                    "role": "model",+                    "parts": [{"functionCall": {"name": "Bash", "args": {"command": "ls"}}}],+                },+                "index": 0,+            }+        ]+    }+    second_call_chunk = {+        "candidates": [+            {+                "content": {+                    "role": "model",+                    "parts": [{"functionCall": {"name": "Bash", "args": {"command": "pwd"}}}],+                },+                "index": 0,+            }+        ]+    }++    iterator.chunk_parser(thought_chunk)+    first = iterator.chunk_parser(first_call_chunk)+    second = iterator.chunk_parser(second_call_chunk)++    first_tool_call = first.choices[0].delta.tool_calls[0]+    assert _get_thought_signature_from_tool(first_tool_call.model_dump()) == signature+    # The step signature is consumed by the first functionCall only+    second_tool_call = second.choices[0].delta.tool_calls[0]+    assert THOUGHT_SIGNATURE_SEPARATOR not in second_tool_call.id 
Merged fix · PR #+85 −1
litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.pyindex 69fe5678de9..7b202b55823 100644@@ -1556,14 +1556,25 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):     ]:         function: ChatCompletionToolCallFunctionChunk | None = None         _tools: Final[list[ChatCompletionToolCallChunk]] = []+        # Gemini 3+ may emit the thought signature on a preceding thought part+        # instead of on the functionCall part itself. Carry it forward so it is+        # not dropped; consume it on the first function call that follows, since+        # in parallel calls only the first one carries a signature.+        pending_thought_signature: str | None = None         for part in parts:+            if part.get("thought") and part.get("thoughtSignature"):+                pending_thought_signature = part.get("thoughtSignature")+                continue             if "functionCall" in part:                 _function_chunk: ChatCompletionToolCallFunctionChunk = {                     "name": part["functionCall"]["name"],                     "arguments": json.dumps(part["functionCall"]["args"], ensure_ascii=False),                 }                 # Extract thought signature if present-                thought_signature = part.get("thoughtSignature")+                # Extract thought signature if present. Fall back to the one+                # carried by a preceding thought part (Gemini 3+ places it there).+                thought_signature = part.get("thoughtSignature") or pending_thought_signature+                pending_thought_signature = None                 # Gemini 3.5+ returns a stable `id` per function call to enable                 # strict response matching. Preserve it as the OpenAI                 # tool_call_id so it can be echoed back unchanged.tests/test_litellm/llms/vertex_ai/gemini/test_thought_signature_in_tool_call_id.pyindex 208cba519f3..6b3fd5ec952 100644@@ -285,3 +285,76 @@ def test_parallel_tool_calls_with_signatures(enable_preview_features):     assert THOUGHT_SIGNATURE_SEPARATOR not in tools[1]["id"]     sig2 = _get_thought_signature_from_tool({"id": tools[1]["id"], "type": "function"})     assert sig2 is None+++def test_thought_signature_on_separate_thought_part_is_preserved():+    """Gemini 3+ places the signature on a thought part, not on the functionCall part.++    Regression for the case where `_transform_parts` only looked at the+    functionCall part and silently dropped the signature, so it was never+    echoed back on the next turn.+    """+    test_signature = "CvsBAdHtim9thoughtPartSignature"+    parts = [+        HttpxPartType(thought=True, thoughtSignature=test_signature),+        HttpxPartType(functionCall={"name": "get_weather", "args": {"city": "Recife"}}),+    ]++    _, tools, _ = VertexGeminiConfig._transform_parts(+        parts=parts,+        cumulative_tool_call_idx=0,+        is_function_call=False,+    )++    assert tools is not None and len(tools) == 1+    assert (+        tools[0].get("provider_specific_fields", {}).get("thought_signature")+        == test_signature+    )+++def test_thought_signature_on_function_call_part_still_wins():+    """A signature on the functionCall part keeps precedence over a pending one."""+    own_signature = "CvsBAdHtim9onTheCallPart"+    parts = [+        HttpxPartType(thought=True, thoughtSignature="CvsBAdHtim9onTheThoughtPart"),+        HttpxPartType(+            functionCall={"name": "get_weather", "args": {"city": "Recife"}},+            thoughtSignature=own_signature,+        ),+    ]++    _, tools, _ = VertexGeminiConfig._transform_parts(+        parts=parts,+        cumulative_tool_call_idx=0,+        is_function_call=False,+    )++    assert tools is not None and len(tools) == 1+    assert (+        tools[0].get("provider_specific_fields", {}).get("thought_signature")+        == own_signature+    )+++def test_pending_thought_signature_is_consumed_once():+    """In parallel calls only the first function call carries a signature."""+    test_signature = "CvsBAdHtim9onlyForTheFirstCall"+    parts = [+        HttpxPartType(thought=True, thoughtSignature=test_signature),+        HttpxPartType(functionCall={"name": "get_weather", "args": {"city": "Recife"}}),+        HttpxPartType(functionCall={"name": "get_time", "args": {"city": "Recife"}}),+    ]++    _, tools, _ = VertexGeminiConfig._transform_parts(+        parts=parts,+        cumulative_tool_call_idx=0,+        is_function_call=False,+    )++    assert tools is not None and len(tools) == 2+    assert (+        tools[0].get("provider_specific_fields", {}).get("thought_signature")+        == test_signature+    )+    assert tools[1].get("provider_specific_fields") is None