Backtestslitellmrun-20261019-033420#40477

Issue #40477 Merged fix PR #41892 Run 19 Oct 2026 · run-20261019-033420 BerriAI/litellm

[Bug]: Gemini candidate with a finishReason but no content is dropped, so the reason never reaches the client

Worse

The merged fix was judged better than our patch.

In this judgment, Patch A = our agent, Patch B = the merged fix.

Both patches take the same overall approach: stop dropping content-less candidates that carry a finishReason, map NO_IMAGE/IMAGE_RECITATION/IMAGE_OTHER to content_filter, add content_filter -> refusal in the Anthropic adapter, and derive Responses API incomplete_details from the chat finish_reason. The decisive differences are in applicability, completeness, and craft.

Patch A commits two binary .mypy_cache files. That is unrelated churn that made the patch fail to apply at all in the independent run (CRASH on every test), so its correctness could not be confirmed by the harness, and it would be rejected in review as-is. Beyond that, A's non-stream fix special-cases content-less candidates inside the loop while leaving the shared per-loop message/tool state intact (it works around the leak by building a fresh dict only for the content-less branch), and for streaming it relies on the pre-existing empty-delta fallback rather than handling the candidate through the normal path. It only sets native_finish_reason on the content-less branch, so the raw reason is inconsistent across choices.

Patch B applies cleanly and passes all 10 independent tests derived from the issue (chat, /v1/messages non-stream and stream, /v1/responses, including the MAX_TOKENS thinking-budget case). It handles content-less candidates uniformly for both ModelResponse and ModelResponseStream by relaxing the skip condition to require both content and finishReason absent, and guards the parts access. It also moves per-candidate state inside the loop, which fixes the multi-candidate state leak at the root rather than papering over it, and verifies that with a multi-candidate test. It sets native_finish_reason on every choice, consolidates the finish-reason key set to derive from the GeminiFinishReason Literal (with a test asserting every documented reason has a mapping), and adds focused unit tests at each layer (core_helpers map, Anthropic adapter, Gemini transform, Responses transform, streaming chunk). Minor risks in B (tools default changed from [] to None; provider_specific_fields now populated on all Gemini choices; extra ESCALATION/UNEXPECTED_TOOL_CALL/MISSING_THOUGHT_SIGNATURE mappings) are small and covered by its tests.

B is the better fix on correctness, tests, scope hygiene, and craft.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

1h 47m

Wall-clock time in the container

Full test suite

No new failures

4,357 passed · 9 skipped · 4,368 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

223vs 462 merged

Changed lines, ours vs merged

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

When Gemini answers with a candidate that has a finishReason but no content, the gateway drops the candidate. gemini-2.5-flash-image served with modalities: ["image"] and asked a text-only question is a deterministic way to get one: Gemini returns finishReason: NO_IMAGE and no parts. Since PR #40294 the gateway no longer answers HTTP 500 on that shape, but what it returns instead carries no trace of why the model produced nothing: /v1/chat/completions comes back with "choices": [] and no finish_reason, /v1/messages comes back with "content": [] and "stop_reason": "end_turn", and /v1/responses comes back with an empty output and "status": "completed". A client cannot tell a declined request from an empty answer, and the same drop hides MALFORMED_FUNCTION_CALL and OTHER. A thinking model that spends its whole budget on thoughts (finishReason: MAX_TOKENS, no content) hits the same branch and arrives with nonzero output_tokens: that case is #36881, with fix PRs #36870, #38301 and #40252 open, so a fix here should build on one of those rather than duplicate it

Expected: the completion carries one choice with an empty assistant message and a finish_reason mapped from Gemini's finishReason (raw value kept in the provider-specific fields), /v1/messages maps it to the matching Anthropic stop_reason, and /v1/responses carries the matching incomplete_details. Which OpenAI and Anthropic values NO_IMAGE and the other non-STOP reasons map to is a decision for the fix PR to record with its alternatives. Gemini safety blocks are a different path and already come back as a content_filter choice

User Flow

Before a (hypothetical) fix: a developer asking an image model a text-only question gets an empty 200 with no reason attached, so their app shows a blank answer and logs nothing to explain it

  1. They send POST https://litellm-domain/v1/chat/completions with "model": "gemini-2.5-flash-image" and a text-only user message
  2. They get HTTP 200 with "choices": [] and "completion_tokens": 0, no finish_reason anywhere
  3. They send the same question as POST https://litellm-domain/v1/messages and get HTTP 200 with "content": [] and "stop_reason": "end_turn", which reads as a normal, complete answer
  4. They send it as POST https://litellm-domain/v1/responses and get HTTP 200 with "output": [] and "status": "completed"

After a (hypothetical) fix: the same requests still return 200, and each one says why the model produced nothing

  1. They send POST https://litellm-domain/v1/chat/completions with "model": "gemini-2.5-flash-image" and a text-only user message
  2. They get HTTP 200 with one choice, an empty assistant message, a finish_reason mapped from NO_IMAGE, and the raw NO_IMAGE value in the provider-specific fields
  3. POST https://litellm-domain/v1/messages returns HTTP 200 with "content": [] and the matching stop_reason
  4. POST https://litellm-domain/v1/responses returns HTTP 200 with the matching incomplete_details instead of a bare completed

Proof the bug occurs

Config the proxy ran with (GEMINI_API_KEY and LITELLM_MASTER_KEY set in the environment, Redis on 127.0.0.1:21556):

model_list:
  - model_name: gemini-2.5-flash-image
    litellm_params:
      model: gemini/gemini-2.5-flash-image
      api_key: os.environ/GEMINI_API_KEY
      modalities: ["image"]
litellm_settings:
  cache: true
  cache_params:
    type: redis
    host: 127.0.0.1
    port: 21556
    ttl: 900
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

Commit: 29af8b7349 on PR #40294 (the tip of litellm_internal_staging plus that PR). Before #40294 the same requests failed with HTTP 500, so the missing reason was hidden behind the error

H=(-H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json")

POST /v1/chat/completions (non-stream, first send: cache miss)

$ curl -s -w '
HTTP %{http_code}
' http://localhost:4000/v1/chat/completions ${H[@]} -d '{"model":"gemini-2.5-flash-image","messages":[{"role":"user","content":"What is 2+2? Reply in text only. Do not generate any image."}]}'
{"id":"EcuhaqKDKbPSjMcP_f3XkAM","created":1788988177,"model":"gemini-2.5-flash-image","object":"chat.completion","choices":[],"usage":{"completion_tokens":0,"prompt_tokens":19,"total_tokens":19,"prompt_tokens_details":{"text_tokens":19}},"vertex_ai_grounding_metadata":[],"vertex_ai_url_context_metadata":[],"vertex_ai_safety_results":[],"vertex_ai_citation_metadata":[],"service_tier":"default"}
HTTP 200

POST /v1/messages (non-stream, first send: cache miss)

$ curl -s -w '
HTTP %{http_code}
' http://localhost:4000/v1/messages ${H[@]} -d '{"model":"gemini-2.5-flash-image","max_tokens":256,"messages":[{"role":"user","content":"What is 2+2? Reply in text only. Do not generate any image."}]}'
{"id":"H8uhap7KDqi-jMcPkLuJkQM","type":"message","role":"assistant","model":"gemini-2.5-flash-image","stop_sequence":null,"usage":{"input_tokens":19,"output_tokens":0},"content":[],"stop_reason":"end_turn","stop_details":null}
HTTP 200

POST /v1/messages (stream=true, first send)

$ curl -sN -w '
HTTP %{http_code}
' http://localhost:4000/v1/messages ${H[@]} -d '{"model":"gemini-2.5-flash-image","max_tokens":256,"stream":true,"messages":[{"role":"user","content":"What is 2+2? Reply in text only. Do not generate any image."}]}'
event: message_start
data: {"type": "message_start", "message": {"id": "msg_026c9335-6ffe-400f-980f-41917bc6bde2", "type": "message", "role": "assistant", "content": [], "model": "gemini-2.5-flash-image", "stop_reason": null, "stop_sequence": null, "usage": {"input_tokens": 0, "output_tokens": 0, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0}}}

event: content_block_start
data: {"type": "content_block_start", "index": 0, "content_block": {"type": "text", "text": ""}}

event: content_block_stop
data: {"type": "content_block_stop", "index": 0}

event: message_delta
data: {"type": "message_delta", "delta": {"stop_reason": "end_turn"}, "usage": {"input_tokens": 19, "output_tokens": 0}}

event: message_stop
data: {"type": "message_stop"}


HTTP 200

POST /v1/responses (non-stream)

$ curl -s -w '
HTTP %{http_code}
' http://localhost:4000/v1/responses ${H[@]} -d '{"model":"gemini-2.5-flash-image","input":"What is 2+2? Reply in text only. Do not generate any image."}'
{"id":"resp_QFs0vCMBFWvLper1WOZ0MVATVtn5aNkDiHz6mlFd9wImxDFiCOk0glkwRnxnzuDSxU0svkSD0n0ha9dbPHUbbbHwqhsBLfp4BGRSHkHcVicIz1XdyBM5STWBmawE3rI1IzZ2-5uCfwmv6Xs2Bc-JCLlLDRmfk9YVt1IsLkpDvi22PkQHo4BSUAD3Gtj7vQdvAisfw3WxqDK2Ud9D0dvZi-OFDY8pobLc3wuLWQTJs1JSpUhR8sPhWqBjCG7a3v4i8yLyjRMuY5bA5TpNL43LVke6As5zccjdhZ7hgQS0c_hVjv1rYNIblKMG7jeEb7ahrLeHb8tGHtL7rGm8CDIJsusQS1vhY8aInl0RgQh2bUgf6_vFemALz1URTfAihipyfIs7GEzoK3-iUh1VAmCTRtNYDgERHw==","created_at":1788988204,"error":null,"incomplete_details":null,"instructions":null,"metadata":{},"model":"gemini-2.5-flash-image","object":"response","output":[],"parallel_tool_calls":false,"temperature":0.0,"tool_choice":"auto","tools":[],"top_p":null,"max_output_tokens":null,"previous_response_id":null,"reasoning":null,"status":"completed","text":{},"truncation":null,"usage":{"input_tokens":19,"input_tokens_details":{"audio_tokens":null,"cached_tokens":0,"text_tok
HTTP 200

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

29af8b7349 (PR #40294 on litellm_internal_staging)

Twitter / LinkedIn details

_No response_

The two patches

Our patch+220 −3
.mypy_cache/3.12/cache.dbnew file mode 100644index 0000000000..43aabbece2Binary files /dev/null and b/.mypy_cache/3.12/cache.db differ.mypy_cache/3.12/cache.db-journalnew file mode 100644index 0000000000..aad541a184Binary files /dev/null and b/.mypy_cache/3.12/cache.db-journal differlitellm/litellm_core_utils/core_helpers.pyindex 15380bc5d5..e80c1b8f8b 100644@@ -222,6 +222,13 @@ _FINISH_REASON_MAP: Final[dict[str, OpenAIChatCompletionFinishReason]] = {     "SPII": "content_filter",     "IMAGE_SAFETY": "content_filter",     "IMAGE_PROHIBITED_CONTENT": "content_filter",+    # The model declined to produce the requested image output. Mapped to+    # content_filter (not "stop") so a declined request is distinguishable from+    # an empty answer: /v1/messages surfaces it as "refusal" and /v1/responses+    # as status "incomplete". "length" was rejected as it implies truncation.+    "NO_IMAGE": "content_filter",+    "IMAGE_RECITATION": "content_filter",+    "IMAGE_OTHER": "content_filter",     "TOO_MANY_TOOL_CALLS": "stop",     "MALFORMED_RESPONSE": "stop",     # Zhipu GLMlitellm/llms/anthropic/experimental_pass_through/adapters/transformation.pyindex b213b5e30b..16eb5dc6bf 100644@@ -1410,6 +1410,8 @@ class LiteLLMAnthropicMessagesAdapter:             return "max_tokens"         elif openai_finish_reason == "tool_calls":             return "tool_use"+        elif openai_finish_reason == "content_filter":+            return "refusal"         return "end_turn"      @staticmethodlitellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.pyindex e8b316b590..6eb403f148 100644@@ -1345,6 +1345,9 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):             "SPII",             "IMAGE_SAFETY",             "IMAGE_PROHIBITED_CONTENT",+            "NO_IMAGE",+            "IMAGE_RECITATION",+            "IMAGE_OTHER",             "TOO_MANY_TOOL_CALLS",             "MALFORMED_RESPONSE",         }@@ -2245,8 +2248,15 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):         server_side_tool_invocations: list[dict[str, object]] | None = None          for idx, candidate in enumerate(_candidates):-            if "content" not in candidate:+            # A content-less candidate that still carries a finishReason (e.g.+            # NO_IMAGE, MALFORMED_FUNCTION_CALL, MAX_TOKENS spent on thoughts)+            # must still produce a choice, otherwise the reason never reaches the+            # client. Streaming handles it in ModelResponseIterator._apply_stream_candidates.+            if "content" not in candidate and (+                isinstance(model_response, ModelResponseStream) or candidate.get("finishReason") is None+            ):                 continue+            candidate_content = candidate.get("content")              # Extract metadata using helper function             (@@ -2261,7 +2271,29 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):             safety_ratings.extend(candidate_safety_ratings)             citation_metadata.extend(candidate_citation_metadata) -            if "parts" in candidate["content"]:+            if candidate_content is None:+                # Build the choice from fresh state: the message/tool state above is+                # shared across candidates and would leak a previous candidate's output.+                raw_finish_reason = candidate.get("finishReason")+                empty_message: ChatCompletionResponseMessage = {"role": "assistant"}+                model_response.choices.append(+                    litellm.Choices(+                        finish_reason=VertexGeminiConfig._check_finish_reason(empty_message, raw_finish_reason),+                        index=candidate.get("index", idx),+                        message=empty_message,+                        logprobs=(+                            VertexGeminiConfig._transform_logprobs(logprobs_result=candidate["logprobsResult"])+                            if "logprobsResult" in candidate+                            else None+                        ),+                        enhancements=None,+                        # Keep Gemini's raw reason so clients can tell e.g. NO_IMAGE apart.+                        provider_specific_fields={"native_finish_reason": raw_finish_reason},+                    )+                )+                continue++            if "parts" in candidate_content:                 (                     content,                     reasoning_content,litellm/responses/litellm_completion_transformation/transformation.pyindex 0d1297f63e..49b6fd4aec 100644@@ -26,6 +26,7 @@ from openai.types.chat.chat_completion_named_tool_choice_param import (     Function as NamedToolChoiceFunction, ) from openai.types.responses import ResponseFunctionToolCall, ResponseFunctionWebSearch+from openai.types.responses.response import IncompleteDetails from openai.types.responses.response_create_params import ResponseInputParam from openai.types.responses.tool_choice_custom_param import ToolChoiceCustomParam from openai.types.responses.tool_choice_function_param import ToolChoiceFunctionParam@@ -2299,6 +2300,20 @@ class LiteLLMCompletionResponsesConfig:             # Default to completed for unknown finish reasons             return "completed" +    @staticmethod+    def _map_chat_completion_finish_reason_to_incomplete_details(+        finish_reason: str | None,+    ) -> IncompleteDetails | None:+        """+        Map an "incomplete" chat completion finish_reason to the Responses API+        incomplete_details, so clients see why the response is incomplete.+        """+        if finish_reason == "length":+            return IncompleteDetails(reason="max_output_tokens")+        if finish_reason in ("content_filter", "refusal"):+            return IncompleteDetails(reason="content_filter")+        return None+     @staticmethod     def _tool_call_id_from_responses_item(item_id: str | None, call_id: str | None) -> str:         """Bedrock Mantle returns a non-unique, index-based ``call_id`` (``call_0``,@@ -2421,7 +2436,8 @@ class LiteLLMCompletionResponsesConfig:             model=chat_completion_response.model,             object="response",             error=getattr(chat_completion_response, "error", None),-            incomplete_details=getattr(chat_completion_response, "incomplete_details", None),+            incomplete_details=getattr(chat_completion_response, "incomplete_details", None)+            or LiteLLMCompletionResponsesConfig._map_chat_completion_finish_reason_to_incomplete_details(finish_reason),             instructions=getattr(chat_completion_response, "instructions", None),             metadata=getattr(chat_completion_response, "metadata", {}),             output=LiteLLMCompletionResponsesConfig._transform_chat_completion_choices_to_responses_output(litellm/types/llms/vertex_ai.pyindex 3b95b78663..ab3406b472 100644@@ -440,6 +440,9 @@ class Candidates(TypedDict, total=False):         "SPII",         "MALFORMED_FUNCTION_CALL",         "IMAGE_SAFETY",+        "NO_IMAGE",+        "IMAGE_RECITATION",+        "IMAGE_OTHER",     ]     safetyRatings: list[SafetyRatings]     citationMetadata: CitationMetadatatests/test_litellm/llms/vertex_ai/gemini/test_gemini_contentless_candidate_finish_reason.pynew file mode 100644index 0000000000..8c5154c0b8@@ -0,0 +1,157 @@+"""+Regression tests: a Gemini candidate with a finishReason but no content.++``gemini-2.5-flash-image`` with ``modalities: ["image"]`` asked a text-only+question answers ``finishReason: NO_IMAGE`` with no ``content``. The candidate+used to be dropped, so /v1/chat/completions returned ``choices: []``,+/v1/messages returned ``stop_reason: end_turn`` and /v1/responses returned a+bare ``status: completed`` — no trace of why the model produced nothing.+"""++import json+from unittest.mock import MagicMock, patch++import httpx+import pytest++import litellm+from litellm.llms.anthropic.experimental_pass_through.adapters.streaming_iterator import (+    AnthropicStreamWrapper,+)+from litellm.llms.custom_httpx.http_handler import HTTPHandler+from litellm.llms.vertex_ai.gemini.vertex_and_google_ai_studio_gemini import (+    ModelResponseIterator,+)++MODEL = "gemini/gemini-2.5-flash-image"+MESSAGES = [{"role": "user", "content": "What is 2+2? Reply in text only."}]+++def _contentless_body(finish_reason: str) -> dict:+    return {+        "candidates": [{"finishReason": finish_reason, "index": 0}],+        "usageMetadata": {"promptTokenCount": 19, "totalTokenCount": 19},+        "modelVersion": "gemini-2.5-flash-image",+        "responseId": "resp-contentless",+    }+++def _mock_gemini_post(finish_reason: str):+    def _post(*args, **kwargs):+        return httpx.Response(+            status_code=200,+            content=json.dumps(_contentless_body(finish_reason)).encode(),+            headers={"content-type": "application/json"},+            request=httpx.Request("POST", "https://generativelanguage.googleapis.com"),+        )++    return patch.object(HTTPHandler, "post", side_effect=_post)+++@pytest.mark.parametrize(+    "gemini_reason, openai_reason",+    [+        ("NO_IMAGE", "content_filter"),+        ("MALFORMED_FUNCTION_CALL", "stop"),+        ("OTHER", "content_filter"),+        ("MAX_TOKENS", "length"),+    ],+)+def test_chat_completion_keeps_contentless_candidate(gemini_reason, openai_reason):+    with _mock_gemini_post(gemini_reason):+        response = litellm.completion(model=MODEL, messages=MESSAGES, api_key="fake-key", client=HTTPHandler())++    assert len(response.choices) == 1+    choice = response.choices[0]+    assert choice.finish_reason == openai_reason+    assert choice.message.role == "assistant"+    assert not choice.message.content+    assert choice.provider_specific_fields["native_finish_reason"] == gemini_reason+++def test_contentless_candidate_does_not_inherit_previous_candidate_output():+    body = {+        "candidates": [+            {+                "index": 0,+                "content": {+                    "role": "model",+                    "parts": [+                        {"text": "calling"},+                        {"functionCall": {"name": "get_weather", "args": {"city": "Paris"}}},+                    ],+                },+                "finishReason": "STOP",+            },+            {"index": 1, "finishReason": "NO_IMAGE"},+        ],+        "usageMetadata": {"promptTokenCount": 19, "totalTokenCount": 19},+    }++    def _post(*args, **kwargs):+        return httpx.Response(+            status_code=200,+            content=json.dumps(body).encode(),+            headers={"content-type": "application/json"},+            request=httpx.Request("POST", "https://generativelanguage.googleapis.com"),+        )++    with patch.object(HTTPHandler, "post", side_effect=_post):+        response = litellm.completion(model=MODEL, messages=MESSAGES, api_key="fake-key", client=HTTPHandler())++    assert len(response.choices) == 2+    assert response.choices[0].message.tool_calls+    contentless = response.choices[1]+    assert contentless.finish_reason == "content_filter"+    assert not contentless.message.content+    assert not contentless.message.tool_calls+    assert contentless.provider_specific_fields["native_finish_reason"] == "NO_IMAGE"+++@pytest.mark.parametrize(+    "gemini_reason, anthropic_reason",+    [("NO_IMAGE", "refusal"), ("MAX_TOKENS", "max_tokens"), ("MALFORMED_FUNCTION_CALL", "end_turn")],+)+def test_anthropic_messages_maps_contentless_candidate_stop_reason(gemini_reason, anthropic_reason):+    with _mock_gemini_post(gemini_reason):+        response = litellm.anthropic.messages.create(+            model=MODEL,+            messages=MESSAGES,+            max_tokens=256,+            api_key="fake-key",+            client=HTTPHandler(),+        )++    assert response["content"] == []+    assert response["stop_reason"] == anthropic_reason+++@pytest.mark.parametrize(+    "gemini_reason, incomplete_reason",+    [("NO_IMAGE", "content_filter"), ("MAX_TOKENS", "max_output_tokens")],+)+def test_responses_api_reports_incomplete_details_for_contentless_candidate(gemini_reason, incomplete_reason):+    with _mock_gemini_post(gemini_reason):+        response = litellm.responses(+            model=MODEL, input=MESSAGES[0]["content"], api_key="fake-key", client=HTTPHandler()+        )++    assert response.status == "incomplete"+    assert response.incomplete_details is not None+    assert response.incomplete_details.reason == incomplete_reason+++def test_streaming_no_image_maps_to_content_filter_and_anthropic_refusal():+    logging_obj = MagicMock()+    logging_obj.optional_params = {}+    iterator = ModelResponseIterator(streaming_response=iter([]), sync_stream=True, logging_obj=logging_obj)++    chunk = iterator.chunk_parser(_contentless_body("NO_IMAGE"))++    assert chunk is not None+    assert len(chunk.choices) == 1+    assert chunk.choices[0].finish_reason == "content_filter"++    events = list(AnthropicStreamWrapper(completion_stream=iter([chunk]), model="gemini-2.5-flash-image"))+    message_delta = next(event for event in events if event.get("type") == "message_delta")+    assert message_delta["delta"]["stop_reason"] == "refusal" 
Merged fix · PR #41892+408 −54
litellm/litellm_core_utils/core_helpers.pyindex 15380bc5d57..d29b1fc74ef 100644@@ -224,6 +224,12 @@ _FINISH_REASON_MAP: Final[dict[str, OpenAIChatCompletionFinishReason]] = {     "IMAGE_PROHIBITED_CONTENT": "content_filter",     "TOO_MANY_TOOL_CALLS": "stop",     "MALFORMED_RESPONSE": "stop",+    "NO_IMAGE": "content_filter",+    "IMAGE_RECITATION": "content_filter",+    "IMAGE_OTHER": "content_filter",+    "ESCALATION": "content_filter",+    "UNEXPECTED_TOOL_CALL": "stop",+    "MISSING_THOUGHT_SIGNATURE": "stop",     # Zhipu GLM     "network_error": "stop",     "sensitive": "content_filter",litellm/llms/anthropic/experimental_pass_through/adapters/transformation.pyindex b213b5e30ba..e9235bc80a7 100644@@ -1410,6 +1410,8 @@ class LiteLLMAnthropicMessagesAdapter:             return "max_tokens"         elif openai_finish_reason == "tool_calls":             return "tool_use"+        elif openai_finish_reason in ["content_filter", "refusal"]:+            return "refusal"         return "end_turn"      @staticmethodlitellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.pyindex e8b316b5902..46f1b948026 100644@@ -6,7 +6,7 @@ import time from collections.abc import Callable, Mapping, Sequence from copy import deepcopy from functools import partial-from typing import TYPE_CHECKING, Any, Final, Literal, Optional, Union, cast+from typing import TYPE_CHECKING, Any, Final, Literal, Optional, Union, cast, get_args  import httpx @@ -57,6 +57,7 @@ from litellm.types.llms.vertex_ai import (     ContentType,     FunctionCallingConfig,     FunctionDeclaration,+    GeminiFinishReason,     GeminiThinkingConfig,     GenerateContentResponseBody,     HttpxPartType,@@ -1330,25 +1331,7 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):             "IMAGE_PROHIBITED_CONTENT": "The token generation was stopped as the response was flagged for prohibited image content.",         } -    _GEMINI_FINISH_REASON_KEYS = frozenset(-        {-            "STOP",-            "MAX_TOKENS",-            "SAFETY",-            "RECITATION",-            "FINISH_REASON_UNSPECIFIED",-            "MALFORMED_FUNCTION_CALL",-            "LANGUAGE",-            "OTHER",-            "BLOCKLIST",-            "PROHIBITED_CONTENT",-            "SPII",-            "IMAGE_SAFETY",-            "IMAGE_PROHIBITED_CONTENT",-            "TOO_MANY_TOOL_CALLS",-            "MALFORMED_RESPONSE",-        }-    )+    _GEMINI_FINISH_REASON_KEYS: Final[frozenset[str]] = frozenset(get_args(GeminiFinishReason))      @staticmethod     def get_finish_reason_mapping() -> dict[str, OpenAIChatCompletionFinishReason]:@@ -2232,22 +2215,23 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):          grounding_metadata: Final[list[dict]] = []         url_context_metadata: Final[list[dict]] = []-        image_response: list[ImageURLListItem] | None = None         safety_ratings: Final[list] = []         citation_metadata: Final[list] = []-        chat_completion_message: Final[ChatCompletionResponseMessage] = {"role": "assistant"}-        chat_completion_logprobs: ChoiceLogprobs | None = None-        tools: list[ChatCompletionToolCallChunk] | None = []-        functions: ChatCompletionToolCallFunctionChunk | None = None-        thinking_blocks: list[ChatCompletionThinkingBlock] | None = None-        reasoning_content: str | None = None-        thought_signatures: Sequence[str] | None = None-        server_side_tool_invocations: list[dict[str, object]] | None = None          for idx, candidate in enumerate(_candidates):-            if "content" not in candidate:+            if "content" not in candidate and "finishReason" not in candidate:                 continue +            image_response: list[ImageURLListItem] | None = None+            chat_completion_message: ChatCompletionResponseMessage = {"role": "assistant"}+            chat_completion_logprobs: ChoiceLogprobs | None = None+            tools: list[ChatCompletionToolCallChunk] | None = None+            functions: ChatCompletionToolCallFunctionChunk | None = None+            thinking_blocks: list[ChatCompletionThinkingBlock] | None = None+            reasoning_content: str | None = None+            thought_signatures: Sequence[str] | None = None+            server_side_tool_invocations: list[dict[str, object]] | None = None+             # Extract metadata using helper function             (                 candidate_grounding_metadata,@@ -2261,7 +2245,7 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):             safety_ratings.extend(candidate_safety_ratings)             citation_metadata.extend(candidate_citation_metadata) -            if "parts" in candidate["content"]:+            if "content" in candidate and candidate["content"] and "parts" in candidate["content"]:                 (                     content,                     reasoning_content,@@ -2368,14 +2352,18 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):                 )                 model_response.choices.append(choice)             elif isinstance(model_response, ModelResponse):+                native_finish_reason = candidate.get("finishReason")                 choice = litellm.Choices(                     finish_reason=VertexGeminiConfig._check_finish_reason(-                        chat_completion_message, candidate.get("finishReason")+                        chat_completion_message, native_finish_reason                     ),                     index=candidate.get("index", idx),                     message=chat_completion_message,                     logprobs=chat_completion_logprobs,                     enhancements=None,+                    provider_specific_fields=(+                        {"native_finish_reason": native_finish_reason} if native_finish_reason is not None else None+                    ),                 )                 model_response.choices.append(choice) @@ -3173,12 +3161,10 @@ class ModelResponseIterator:                     self.has_seen_tool_calls = True                     break -        # _process_candidates skips candidates without a "content" part, so a-        # content-less chunk leaves choices empty and the downstream streaming-        # handler hits IndexError on choices[0]. This covers the final chunk-        # (finishReason, no content) and mid-stream metadata-only chunks-        # (grounding/web-search/thought, no content and no finishReason — seen-        # with web_search + reasoning) by emitting an empty-delta choice.+        # _process_candidates skips candidates with neither "content" nor+        # "finishReason", so a metadata-only chunk (grounding/web-search/thought,+        # seen with web_search + reasoning) leaves choices empty and the downstream+        # streaming handler hits IndexError on choices[0]. Emit an empty-delta choice.         if not model_response.choices and _candidates:             from litellm.types.utils import Delta, StreamingChoices litellm/responses/litellm_completion_transformation/transformation.pyindex 0d1297f63e2..044596676dd 100644@@ -61,6 +61,7 @@ from litellm.types.llms.openai import (     ChatCompletionToolParamFunctionChunk,     ChatCompletionUserMessage,     GenericChatCompletionMessage,+    IncompleteDetails,     InputTokensDetails,     OpenAIChatCompletionTextObject,     OpenAIMcpServerTool,@@ -111,6 +112,9 @@ ResponseTools: TypeAlias = Sequence[Mapping[str, object]] | None ChatToolParam: TypeAlias = ChatCompletionToolParam | OpenAIMcpServerTool NAMESPACE_DESCRIPTION_SEPARATOR: Final = "\n\n" NAMESPACE_MEMBER_TYPES_WITH_CHAT_TOOLS: Final = frozenset({"function", "custom"})+_INCOMPLETE_REASON_BY_FINISH_REASON: Final[Mapping[str, Literal["max_output_tokens", "content_filter"]]] = (+    MappingProxyType({"length": "max_output_tokens", "content_filter": "content_filter", "refusal": "content_filter"})+)   @dataclass(frozen=True, slots=True)@@ -2299,6 +2303,18 @@ class LiteLLMCompletionResponsesConfig:             # Default to completed for unknown finish reasons             return "completed" +    @staticmethod+    def _incomplete_details_for_finish_reason(+        finish_reason: str | None,+        existing: IncompleteDetails | None,+    ) -> IncompleteDetails | None:+        if existing is not None:+            return existing+        if finish_reason is None:+            return None+        reason: Final = _INCOMPLETE_REASON_BY_FINISH_REASON.get(finish_reason)+        return IncompleteDetails(reason=reason) if reason is not None else None+     @staticmethod     def _tool_call_id_from_responses_item(item_id: str | None, call_id: str | None) -> str:         """Bedrock Mantle returns a non-unique, index-based ``call_id`` (``call_0``,@@ -2415,13 +2431,18 @@ class LiteLLMCompletionResponsesConfig:         if choices and len(choices) > 0:             finish_reason = choices[0].finish_reason +        incomplete_details: Final = LiteLLMCompletionResponsesConfig._incomplete_details_for_finish_reason(+            finish_reason=finish_reason,+            existing=getattr(chat_completion_response, "incomplete_details", None),+        )+         responses_api_response: Final[ResponsesAPIResponse] = ResponsesAPIResponse(             id=chat_completion_response.id,             created_at=chat_completion_response.created,             model=chat_completion_response.model,             object="response",             error=getattr(chat_completion_response, "error", None),-            incomplete_details=getattr(chat_completion_response, "incomplete_details", None),+            incomplete_details=incomplete_details,             instructions=getattr(chat_completion_response, "instructions", None),             metadata=getattr(chat_completion_response, "metadata", {}),             output=LiteLLMCompletionResponsesConfig._transform_chat_completion_choices_to_responses_output(litellm/types/llms/vertex_ai.pyindex 3b95b786631..ce51e46ef15 100644@@ -425,22 +425,35 @@ class UrlContextMetadata(TypedDict, total=False):     urlMetadata: list[UrlMetadata]  +GeminiFinishReason = Literal[+    "FINISH_REASON_UNSPECIFIED",+    "STOP",+    "MAX_TOKENS",+    "SAFETY",+    "RECITATION",+    "LANGUAGE",+    "OTHER",+    "BLOCKLIST",+    "PROHIBITED_CONTENT",+    "SPII",+    "MALFORMED_FUNCTION_CALL",+    "IMAGE_SAFETY",+    "IMAGE_PROHIBITED_CONTENT",+    "TOO_MANY_TOOL_CALLS",+    "MALFORMED_RESPONSE",+    "NO_IMAGE",+    "IMAGE_RECITATION",+    "IMAGE_OTHER",+    "ESCALATION",+    "UNEXPECTED_TOOL_CALL",+    "MISSING_THOUGHT_SIGNATURE",+]++ class Candidates(TypedDict, total=False):     index: int     content: HttpxContentType-    finishReason: Literal[-        "FINISH_REASON_UNSPECIFIED",-        "STOP",-        "MAX_TOKENS",-        "SAFETY",-        "RECITATION",-        "OTHER",-        "BLOCKLIST",-        "PROHIBITED_CONTENT",-        "SPII",-        "MALFORMED_FUNCTION_CALL",-        "IMAGE_SAFETY",-    ]+    finishReason: GeminiFinishReason     safetyRatings: list[SafetyRatings]     citationMetadata: CitationMetadata     groundingMetadata: GroundingMetadatatests/test_litellm/litellm_core_utils/test_core_helpers.pyindex e937be47441..b2ad13c205e 100644@@ -151,6 +151,12 @@ class TestMapFinishReasonGemini:             ("IMAGE_PROHIBITED_CONTENT", "content_filter"),             ("TOO_MANY_TOOL_CALLS", "stop"),             ("MALFORMED_RESPONSE", "stop"),+            ("NO_IMAGE", "content_filter"),+            ("IMAGE_RECITATION", "content_filter"),+            ("IMAGE_OTHER", "content_filter"),+            ("ESCALATION", "content_filter"),+            ("UNEXPECTED_TOOL_CALL", "stop"),+            ("MISSING_THOUGHT_SIGNATURE", "stop"),         ],     )     def test_gemini_finish_reasons(self, gemini_reason, expected):tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.pyindex 2318ceb937a..a5fb19b236f 100644@@ -105,6 +105,46 @@ def test_translate_chat_length_takes_precedence_over_refusal():     assert result.get("stop_details") is None  +def test_translate_chat_content_filter_to_anthropic_response():+    response = ModelResponse(+        id="chatcmpl-content-filter",+        model="openai-model",+        choices=[+            Choices(+                index=0,+                finish_reason="content_filter",+                message=Message(content=None, role="assistant"),+            )+        ],+        usage=Usage(prompt_tokens=1, completion_tokens=0, total_tokens=1),+    )++    result = LiteLLMAnthropicMessagesAdapter().translate_openai_response_to_anthropic(response)++    assert result["content"] == []+    assert result["stop_reason"] == "refusal"+++def test_translate_chat_refusal_finish_reason_to_anthropic_response():+    response = ModelResponse(+        id="chatcmpl-refusal-reason",+        model="openai-model",+        choices=[+            Choices(+                index=0,+                finish_reason="refusal",+                message=Message(content=None, role="assistant"),+            )+        ],+        usage=Usage(prompt_tokens=1, completion_tokens=0, total_tokens=1),+    )++    result = LiteLLMAnthropicMessagesAdapter().translate_openai_response_to_anthropic(response)++    assert result["content"] == []+    assert result["stop_reason"] == "refusal"++ def test_translate_streaming_openai_chunk_to_anthropic_content_block():     choices = [         StreamingChoices(tests/test_litellm/llms/vertex_ai/gemini/test_vertex_and_google_ai_studio_gemini.pyindex 84295666fbf..6c818016c87 100644@@ -2,7 +2,7 @@ import asyncio import json import re from copy import deepcopy-from typing import Final, List, cast+from typing import Final, List, cast, get_args from unittest.mock import MagicMock, patch  import httpx@@ -18,7 +18,7 @@ from litellm.llms.vertex_ai.common_utils import VertexAIError from litellm.llms.vertex_ai.gemini.vertex_and_google_ai_studio_gemini import (     VertexGeminiConfig, )-from litellm.types.llms.vertex_ai import UsageMetadata+from litellm.types.llms.vertex_ai import GeminiFinishReason, UsageMetadata from litellm.types.utils import ChoiceLogprobs, Usage from litellm.utils import CustomStreamWrapper @@ -940,6 +940,11 @@ def test_check_finish_reason():         )  +def test_every_documented_gemini_finish_reason_has_an_explicit_mapping():+    documented: Final = frozenset(get_args(GeminiFinishReason))+    assert set(VertexGeminiConfig.get_finish_reason_mapping()) == documented++ def test_finish_reason_unspecified_and_malformed_function_call():     """     Test that FINISH_REASON_UNSPECIFIED and MALFORMED_FUNCTION_CALL@@ -968,6 +973,12 @@ def test_finish_reason_unspecified_and_malformed_function_call():     # Test new Gemini finish reasons     assert finish_reason_mappings["TOO_MANY_TOOL_CALLS"] == "stop"     assert finish_reason_mappings["MALFORMED_RESPONSE"] == "stop"+    assert finish_reason_mappings["NO_IMAGE"] == "content_filter"+    assert finish_reason_mappings["IMAGE_RECITATION"] == "content_filter"+    assert finish_reason_mappings["IMAGE_OTHER"] == "content_filter"+    assert finish_reason_mappings["ESCALATION"] == "content_filter"+    assert finish_reason_mappings["UNEXPECTED_TOOL_CALL"] == "stop"+    assert finish_reason_mappings["MISSING_THOUGHT_SIGNATURE"] == "stop"   def test_vertex_ai_usage_metadata_response_token_count():@@ -6074,3 +6085,210 @@ def test_prompt_blocked_chunk_keeps_served_model_version():      assert streaming_chunk.model == "gemini-3.8-flash-001"     assert streaming_chunk.choices[0].finish_reason == "content_filter"+++def test_gemini_candidate_with_finish_reason_no_content_chat_completion():+    config = VertexGeminiConfig()+    completion_response = {+        "candidates": [+            {+                "finishReason": "NO_IMAGE",+                "index": 0,+            }+        ],+        "usageMetadata": {+            "promptTokenCount": 19,+            "candidatesTokenCount": 0,+            "totalTokenCount": 19,+        },+    }+    model_response = ModelResponse()+    logging_obj = MagicMock()+    raw_response = MagicMock()+    raw_response.headers = {}++    resp = config._transform_google_generate_content_to_openai_model_response(+        completion_response=completion_response,+        model_response=model_response,+        model="gemini-2.5-flash-image",+        logging_obj=logging_obj,+        raw_response=raw_response,+    )+    assert len(resp.choices) == 1+    assert resp.choices[0].finish_reason == "content_filter"+    assert resp.choices[0].message.content is None+    assert resp.choices[0].provider_specific_fields["native_finish_reason"] == "NO_IMAGE"+++def test_gemini_candidate_with_finish_reason_no_content_anthropic_messages():+    from litellm.llms.anthropic.experimental_pass_through.adapters.transformation import (+        LiteLLMAnthropicMessagesAdapter,+    )++    config = VertexGeminiConfig()+    completion_response = {+        "candidates": [+            {+                "finishReason": "NO_IMAGE",+                "index": 0,+            }+        ],+        "usageMetadata": {+            "promptTokenCount": 19,+            "candidatesTokenCount": 0,+            "totalTokenCount": 19,+        },+    }+    resp = config._transform_google_generate_content_to_openai_model_response(+        completion_response=completion_response,+        model_response=ModelResponse(),+        model="gemini-2.5-flash-image",+        logging_obj=MagicMock(),+        raw_response=MagicMock(headers={}),+    )++    adapter = LiteLLMAnthropicMessagesAdapter()+    anthropic_resp = adapter.translate_openai_response_to_anthropic(+        response=resp,+        tool_name_mapping={},+    )+    assert anthropic_resp["stop_reason"] == "refusal"+    assert anthropic_resp["content"] == []+++def test_gemini_candidate_with_finish_reason_no_content_responses_api():+    from litellm.responses.litellm_completion_transformation.transformation import (+        LiteLLMCompletionResponsesConfig,+    )++    config = VertexGeminiConfig()+    completion_response = {+        "candidates": [+            {+                "finishReason": "NO_IMAGE",+                "index": 0,+            }+        ],+        "usageMetadata": {+            "promptTokenCount": 19,+            "candidatesTokenCount": 0,+            "totalTokenCount": 19,+        },+    }+    resp = config._transform_google_generate_content_to_openai_model_response(+        completion_response=completion_response,+        model_response=ModelResponse(),+        model="gemini-2.5-flash-image",+        logging_obj=MagicMock(),+        raw_response=MagicMock(headers={}),+    )++    responses_resp = LiteLLMCompletionResponsesConfig.transform_chat_completion_response_to_responses_api_response(+        request_input="Generate picture",+        responses_api_request={},+        chat_completion_response=resp,+    )+    assert responses_resp.status == "incomplete"+    assert responses_resp.incomplete_details is not None+    assert responses_resp.incomplete_details.reason == "content_filter"+++def test_gemini_candidate_other_finish_reasons_no_content():+    from litellm.llms.anthropic.experimental_pass_through.adapters.transformation import (+        LiteLLMAnthropicMessagesAdapter,+    )+    from litellm.responses.litellm_completion_transformation.transformation import (+        LiteLLMCompletionResponsesConfig,+    )++    config = VertexGeminiConfig()+    max_tokens_response = {+        "candidates": [{"finishReason": "MAX_TOKENS", "index": 0}],+        "usageMetadata": {"promptTokenCount": 10, "candidatesTokenCount": 50, "totalTokenCount": 60},+    }+    resp_length = config._transform_google_generate_content_to_openai_model_response(+        completion_response=max_tokens_response,+        model_response=ModelResponse(),+        model="gemini-2.5-flash",+        logging_obj=MagicMock(),+        raw_response=MagicMock(headers={}),+    )+    assert len(resp_length.choices) == 1+    assert resp_length.choices[0].finish_reason == "length"+    assert resp_length.choices[0].provider_specific_fields["native_finish_reason"] == "MAX_TOKENS"++    anthropic_length = LiteLLMAnthropicMessagesAdapter().translate_openai_response_to_anthropic(+        response=resp_length,+        tool_name_mapping={},+    )+    assert anthropic_length["stop_reason"] == "max_tokens"++    responses_length = LiteLLMCompletionResponsesConfig.transform_chat_completion_response_to_responses_api_response(+        request_input="thinking request",+        responses_api_request={},+        chat_completion_response=resp_length,+    )+    assert responses_length.status == "incomplete"+    assert responses_length.incomplete_details.reason == "max_output_tokens"+++def test_gemini_candidate_with_finish_reason_no_content_streaming_chunk():+    from litellm.llms.vertex_ai.gemini.vertex_and_google_ai_studio_gemini import (+        ModelResponseIterator,+    )++    chunk: Final = {+        "candidates": [{"finishReason": "NO_IMAGE", "index": 0}],+        "usageMetadata": {"promptTokenCount": 19, "candidatesTokenCount": 0, "totalTokenCount": 19},+    }+    iterator: Final = ModelResponseIterator(streaming_response=[], sync_stream=True, logging_obj=MagicMock())++    streaming_chunk: Final = iterator.chunk_parser(chunk)++    assert len(streaming_chunk.choices) == 1+    assert streaming_chunk.choices[0].finish_reason == "content_filter"+    assert streaming_chunk.choices[0].delta.content is None+    assert streaming_chunk.choices[0].delta.tool_calls is None+++def test_gemini_multi_candidate_messages_do_not_share_state():+    config: Final = VertexGeminiConfig()+    completion_response: Final = {+        "candidates": [+            {+                "content": {+                    "role": "model",+                    "parts": [+                        {"text": "Let me check the weather.", "thought": True},+                        {"functionCall": {"name": "get_weather", "args": {"city": "Paris"}}},+                    ],+                },+                "finishReason": "STOP",+                "index": 0,+            },+            {+                "content": {"role": "model", "parts": [{"text": "It is sunny in Paris."}]},+                "finishReason": "STOP",+                "index": 1,+            },+        ],+        "usageMetadata": {"promptTokenCount": 10, "candidatesTokenCount": 20, "totalTokenCount": 30},+    }++    resp: Final = config._transform_google_generate_content_to_openai_model_response(+        completion_response=completion_response,+        model_response=ModelResponse(),+        model="gemini-2.5-flash",+        logging_obj=MagicMock(),+        raw_response=MagicMock(headers={}),+    )++    assert len(resp.choices) == 2+    assert resp.choices[0].finish_reason == "tool_calls"+    assert resp.choices[0].message.tool_calls[0].function.name == "get_weather"+    assert resp.choices[0].message.reasoning_content == "Let me check the weather."+    assert resp.choices[1].finish_reason == "stop"+    assert resp.choices[1].message.content == "It is sunny in Paris."+    assert resp.choices[1].message.tool_calls is None+    assert getattr(resp.choices[1].message, "reasoning_content", None) is None+    assert resp.choices[1].provider_specific_fields["native_finish_reason"] == "STOP"tests/test_litellm/responses/litellm_completion_transformation/test_litellm_completion_responses.pyindex f1924b87a20..c66133ed5f1 100644@@ -4938,3 +4938,65 @@ class TestStreamingSnapshotItemIds:         reasoning_items = _bridged_output_items(completed_event.response, "reasoning")         assert len(reasoning_items) == 1         assert reasoning_items[0].id == streamed_event.item_id+++def test_transform_chat_completion_response_incomplete_details():+    from litellm.types.llms.openai import IncompleteDetails++    resp_length = ModelResponse(+        id="resp-length",+        choices=[Choices(index=0, finish_reason="length", message=Message(content="cutoff", role="assistant"))],+        model="gpt-4o",+    )+    result_length = LiteLLMCompletionResponsesConfig.transform_chat_completion_response_to_responses_api_response(+        request_input="test prompt",+        responses_api_request={},+        chat_completion_response=resp_length,+    )+    assert result_length.status == "incomplete"+    assert result_length.incomplete_details is not None+    assert result_length.incomplete_details.reason == "max_output_tokens"++    resp_filter = ModelResponse(+        id="resp-filter",+        choices=[Choices(index=0, finish_reason="content_filter", message=Message(content=None, role="assistant"))],+        model="gpt-4o",+    )+    result_filter = LiteLLMCompletionResponsesConfig.transform_chat_completion_response_to_responses_api_response(+        request_input="test prompt",+        responses_api_request={},+        chat_completion_response=resp_filter,+    )+    assert result_filter.status == "incomplete"+    assert result_filter.incomplete_details is not None+    assert result_filter.incomplete_details.reason == "content_filter"++    resp_refusal = ModelResponse(+        id="resp-refusal",+        choices=[Choices(index=0, finish_reason="refusal", message=Message(content=None, role="assistant"))],+        model="gpt-4o",+    )+    result_refusal = LiteLLMCompletionResponsesConfig.transform_chat_completion_response_to_responses_api_response(+        request_input="test prompt",+        responses_api_request={},+        chat_completion_response=resp_refusal,+    )+    assert result_refusal.status == "incomplete"+    assert result_refusal.incomplete_details is not None+    assert result_refusal.incomplete_details.reason == "content_filter"++    existing_details = IncompleteDetails(reason="content_filter")+    resp_existing = ModelResponse(+        id="resp-existing",+        choices=[Choices(index=0, finish_reason="length", message=Message(content="cutoff", role="assistant"))],+        model="gpt-4o",+    )+    resp_existing.incomplete_details = existing_details+    result_existing = LiteLLMCompletionResponsesConfig.transform_chat_completion_response_to_responses_api_response(+        request_input="test prompt",+        responses_api_request={},+        chat_completion_response=resp_existing,+    )+    assert result_existing.status == "incomplete"+    assert result_existing.incomplete_details == existing_details+