Backtestslitellmrun-20261019-033420#41834

Issue #41834 Merged fix PR #46111 Run 19 Oct 2026 · run-20261019-033420 BerriAI/litellm

[Bug]: Databricks service_tier is dropped in request translation, streaming, and pricing

Better

Our patch was judged better than the merged fix.

In this judgment, Patch B = our agent, Patch A = the merged fix.

Patch A only adds `service_tier` to the Databricks supported-params list, fixes the Responses-to-Chat request read, and preserves the tier in `transform_response`. It does nothing for streaming (the tier arriving only on the terminal usage-only chunk is still lost in `CustomStreamWrapper`), nor for Chat-to-Responses or Chat-to-Messages response conversion. The independent tests confirm this: A fails the terminal-usage-chunk aggregation test, the Responses and Messages returned-tier preservation tests, and the streamed-default-overrides-requested-priority pricing test, which the issue explicitly calls out as the reason the defects must be fixed together. Patch B covers the same request-side fixes (with a safe `or kwargs.get` fallback) and additionally stamps the served tier from usage-only chunks in the stream wrapper, propagates it into the Responses API response, adds a `served_service_tier` helper that never manufactures a tier, threads it through the Anthropic Messages adapter for both non-streaming responses and the streaming `message_delta` usage (sync and async), and extends the TypedDicts accordingly. B passes all 13 independent tests. B's own test file is far more thorough, covering pricing precedence, legacy model resolution, drop_params mapping, no-manufactured-tier cases, stream aggregation via the real Databricks iterator and `CustomStreamWrapper`, and the Messages bridge end-to-end to cost. A's tests only cover the three narrow paths it fixed. B's extra scope is directly demanded by the issue, not churn, and is implemented consistently with surrounding conventions (Final annotations, setattr with noqa for pydantic extras).

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

59m 46s

Wall-clock time in the container

Full test suite

Not run

The harness did not reach the test stage

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

396vs 56 merged

Changed lines, ours vs merged

Summary

In LiteLLM 1.100.0, Databricks service_tier="priority" can be dropped during request translation and parameter filtering. Databricks cost calculation also ignores the resolved tier, so configured priority input/output/cache-read rates are not used. Returned tiers are lost in response conversion and streaming aggregation.

These defects need to be handled together: fixing pricing alone can apply requested priority rates even when Databricks reports service_tier="default" after fallback.

Reproduction and environment

Reproduced with a clean installation of litellm[proxy]==1.100.0 on Python 3.11 in an isolated virtual environment. The portable pytest file will be attached in a follow-up comment. It uses synthetic model prices and response objects; no API credentials, provider calls, proxy, or downstream repository files are required.

Save the attached file as test_databricks_service_tier_upstream.py, outside any existing checkout, and run:

python3 -m venv /tmp/databricks-tier-repro
/tmp/databricks-tier-repro/bin/pip install 'litellm[proxy]==1.100.0' pytest
LITELLM_LOCAL_MODEL_COST_MAP=True /tmp/databricks-tier-repro/bin/python -m pytest -q test_databricks_service_tier_upstream.py

Observed results: 6 failed, 7 passed.

CaseExpectedActual on 1.100.0
Priority pricing from explicit argument, request params, or response$0.000568$0.000284 in all three cases
Responses-to-Chat request translationservice_tier="priority" preservedField absent (KeyError)
Databricks parameter mapping with drop_params=Trueservice_tier="priority" preservedField absent (KeyError)
Stream with tier on terminal usage-only chunkAggregated response has service_tier="default"None

The synthetic pricing case has 100 input tokens (20 cached) and 50 output tokens. Standard rates are $1/$4 per million input/output tokens and $0.20 per million cached input tokens; priority rates are twice those values.

Default pricing, legacy Databricks model resolution, and returned-default fallback controls pass on stock 1.100.0. The fallback controls pass because stock Databricks pricing always uses standard rates; they must continue to pass after priority pricing is enabled. All 13 tests pass with a local fix.

Root cause

Source locations refer to LiteLLM 1.100.0:

  • litellm/llms/databricks/chat/transformation.py, DatabricksConfig.get_supported_openai_params (~line 259): service_tier is absent from the supported parameter list.
  • litellm/responses/litellm_completion_transformation/transformation.py, transform_responses_api_request_to_chat_completion_request (~line 340): reads kwargs.get("service_tier"), but the tier is in responses_api_request. Adding only the Databricks allowlist entry does not repair this earlier loss.
  • litellm/cost_calculator.py (~line 623) and litellm/llms/databricks/cost_calculator.py (~line 34): the Databricks dispatch/helper do not pass service_tier through to generic_cost_per_token.
  • litellm/litellm_core_utils/streaming_chunk_builder_utils.py: stream aggregation does not carry the returned tier, including when it arrives only on the final usage chunk. Databricks response parsing, stream-wrapper reconstruction, and Chat/Responses/Messages conversion also need to preserve it for end-to-end accounting.
  • completion_cost currently gives a requested tier precedence over a returned tier. For Databricks, a concrete returned default tier should override requested priority.

Additional Router/HTTP-interception tests exposed a related Messages-stream accounting issue: litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py calls stream_chunk_builder without its existing request logging object. Standalone cost calculation can then miss Router pricing and cache a zero cost. Passing the logging object preserves the configured pricing context. This path is covered by the broader local regression suite, not the 13-test portable file.

Expected behavior

  • Forward supported Databricks default/priority tiers through Chat, Responses, and Messages bridges, including streaming.
  • Preserve provider-returned tiers through response parsing, adapters, stream reconstruction/aggregation, and accounting.
  • Apply configured tier-specific input, output, and cache-read prices.
  • Use a concrete returned Databricks tier over the requested tier; retain standard pricing and legacy model-name resolution.
  • If the provider omits a returned tier, do not manufacture one. Requested-tier pricing is an estimate, not evidence of the tier delivered or invoiced.

Databricks documents priority fallback and model/region availability here: https://docs.databricks.com/aws/en/machine-learning/foundation-model-apis/priority-mode

I searched open and closed issues for Databricks priority/service-tier symptoms and the affected function names and found no matching report. PR #35674 concerns cache pricing and does not address these tier-propagation gaps.

The two patches

Our patch+392 −4
litellm/litellm_core_utils/streaming_handler.pyindex a78e6f17cc..61e48bc824 100644@@ -1508,6 +1508,10 @@ class CustomStreamWrapper:                         except Exception:                             model_response.choices[0].delta = Delta()                 else:+                    # Providers (e.g. Databricks) may report the served tier only on the usage-only chunk.+                    usage_chunk_service_tier: Final = original_chunk.model_dump().get("service_tier")+                    if isinstance(usage_chunk_service_tier, str) and usage_chunk_service_tier:+                        setattr(model_response, "service_tier", usage_chunk_service_tier)  # noqa: B010  # pydantic extra                     if self.send_stream_usage is True:                         model_response.choices = []                         return model_responselitellm/llms/anthropic/pass_through/adapters/streaming_iterator.pyindex 34c18b4f0b..1056ff4def 100644@@ -21,6 +21,7 @@ from litellm._logging import verbose_logger from litellm._uuid import uuid from litellm.exceptions import ContextWindowExceededError, MidStreamFallbackError from litellm.litellm_core_utils.hidden_params import set_hidden_params+from litellm.llms.anthropic.pass_through.utils import served_service_tier from litellm.llms.base_llm.chat.transformation import BaseLLMException from litellm.types.llms.anthropic import (     AppliedEdit,@@ -351,6 +352,8 @@ class AnthropicStreamWrapper(AdapterCompletionStreamWrapper):         self.compaction_block = compaction_block         self.iterations_usage = iterations_usage         self._refusal_text: str = ""+        # Tier the provider reported serving on; it may arrive only on the final usage chunk.+        self.served_service_tier: str | None = None         self.sent_compaction_block: bool = False         # Per-phase flags so the compaction block's start/delta/stop events         # are emitted (and the public state machine is advanced) in@@ -445,14 +448,26 @@ class AnthropicStreamWrapper(AdapterCompletionStreamWrapper):         augmented["context_management"] = ContextManagementResponse(applied_edits=list(self.applied_edits))         return augmented +    def _observe_service_tier(self, chunk: object) -> None:+        tier: Final = served_service_tier(chunk)+        if tier is not None:+            self.served_service_tier = tier++    def _attach_service_tier(self, message_delta_chunk: MessageBlockDelta) -> MessageBlockDelta:+        """Report the provider-served tier on the final ``message_delta`` usage, as Anthropic does."""+        usage: Final = message_delta_chunk.get("usage")+        if self.served_service_tier is None or not isinstance(usage, dict) or "service_tier" in usage:+            return message_delta_chunk+        return {**message_delta_chunk, "usage": {**usage, "service_tier": self.served_service_tier}}+     def _augment_message_delta_usage(self, message_delta_chunk: MessageBlockDelta) -> MessageBlockDelta:         """Attach polyfill compaction iteration usage to the final message_delta.          Also defensively re-attaches ``context_management`` so the direct         held-chunk flush path stays in sync with the merge path's guarantee-        when ``self.applied_edits`` is non-empty.+        when ``self.applied_edits`` is non-empty, and stamps the served tier.         """-        message_delta_chunk = self._ensure_context_management_attached(message_delta_chunk)+        message_delta_chunk = self._attach_service_tier(self._ensure_context_management_attached(message_delta_chunk))         if self.iterations_usage is None:             return message_delta_chunk         usage: Final = message_delta_chunk.get("usage")@@ -590,6 +605,7 @@ class AnthropicStreamWrapper(AdapterCompletionStreamWrapper):                 if chunk == "None" or chunk is None:                     raise Exception +                self._observe_service_tier(chunk)                 if not getattr(chunk, "choices", None):                     if self._handle_choiceless_chunk(chunk):                         return self.chunk_queue.popleft()@@ -825,6 +841,7 @@ class AnthropicStreamWrapper(AdapterCompletionStreamWrapper):                 if chunk == "None" or chunk is None:                     raise Exception +                self._observe_service_tier(chunk)                 if not getattr(chunk, "choices", None):                     if self._handle_choiceless_chunk(chunk):                         return self.chunk_queue.popleft()litellm/llms/anthropic/pass_through/adapters/transformation.pyindex 2ee2385289..dd4613ea14 100644@@ -12,6 +12,7 @@ from litellm.litellm_core_utils.hidden_params import get_hidden_params from litellm.llms.anthropic.pass_through.utils import (     is_reasoning_auto_summary_enabled,     prompt_cache_key_from_user_id,+    served_service_tier, )  # OpenAI has a 64-character limit for function/tool names@@ -1601,6 +1602,12 @@ class LiteLLMAnthropicMessagesAdapter:             if polyfill_iterations is not None             else message_usage         )+        response_service_tier: Final = served_service_tier(response)+        reported_usage: Final[AnthropicUsage] = (+            {**anthropic_usage, "service_tier": response_service_tier}+            if response_service_tier is not None+            else anthropic_usage+        )          translated_obj: Final = AnthropicMessagesResponse(             id=response.id,@@ -1608,7 +1615,7 @@ class LiteLLMAnthropicMessagesAdapter:             role="assistant",             model=response.model or "unknown-model",             stop_sequence=None,-            usage=anthropic_usage,+            usage=reported_usage,             content=anthropic_content,             stop_reason=anthropic_finish_reason,             stop_details=(refusal_stop_details(refusal_text) if anthropic_finish_reason == "refusal" else None),litellm/llms/anthropic/pass_through/utils.pyindex 95baecbfbf..ea6c59f3f0 100644@@ -47,6 +47,20 @@ def prompt_cache_key_from_user_id(user_id: object) -> str | None:         return None  +def served_service_tier(response: object) -> str | None:+    """The tier the provider reported serving, read off an OpenAI-shaped response or chunk, then its usage.++    Never falls back to the requested tier: a tier the provider did not report is unknown, not assumed.+    """+    response_tier: Final = getattr(response, "service_tier", None)+    if isinstance(response_tier, str) and response_tier:+        return response_tier+    usage_tier: Final = getattr(getattr(response, "usage", None), "service_tier", None)+    if isinstance(usage_tier, str) and usage_tier:+        return usage_tier+    return None++ def litellm_logging_obj_from_kwargs(kwargs: Mapping[str, object]) -> "LiteLLMLoggingObject | None":     """The logging object the bridged call logs through, when the caller supplied one."""     from litellm.litellm_core_utils.litellm_logging import Logginglitellm/llms/databricks/chat/transformation.pyindex 50d3b82a98..9fdaae754a 100644@@ -285,6 +285,7 @@ class DatabricksConfig(DatabricksBase, OpenAILikeChatConfig, AnthropicConfig):             "tool_choice",             "reasoning_effort",             "thinking",+            "service_tier",         ]      @staticmethod@@ -697,6 +698,9 @@ class DatabricksConfig(DatabricksBase, OpenAILikeChatConfig, AnthropicConfig):         model_response.id = completion_response["id"]         model_response.created = completion_response["created"]         setattr(model_response, "usage", Usage(**completion_response["usage"]))+        served_service_tier: Final = completion_response.get("service_tier")+        if isinstance(served_service_tier, str) and served_service_tier:+            setattr(model_response, "service_tier", served_service_tier)  # noqa: B010  # pydantic extra          model_response.choices = self._transform_dbrx_choices(             choices=completion_response["choices"],litellm/responses/litellm_completion_transformation/transformation.pyindex 4f99c57055..66473ceaf2 100644@@ -500,7 +500,7 @@ class LiteLLMCompletionResponsesConfig:             "max_tokens": responses_api_request.get("max_output_tokens"),             "stream": stream,             "metadata": kwargs.get("metadata"),-            "service_tier": kwargs.get("service_tier"),+            "service_tier": responses_api_request.get("service_tier") or kwargs.get("service_tier"),             "safety_identifier": responses_api_request.get("safety_identifier"),             "web_search_options": web_search_options,             "response_format": response_format,@@ -2504,6 +2504,9 @@ class LiteLLMCompletionResponsesConfig:             user=echoed.get("user"),             store=echoed.get("store"),         )+        served_service_tier: Final = getattr(chat_completion_response, "service_tier", None)+        if isinstance(served_service_tier, str) and served_service_tier:+            setattr(responses_api_response, "service_tier", served_service_tier)  # noqa: B010  # pydantic extra         chat_completion_hidden_params: Final = get_hidden_params_storage(chat_completion_response)         set_hidden_params(             responses_api_response,litellm/types/llms/anthropic.pyindex f0b5a1bb84..efb1d41d1c 100644@@ -562,6 +562,7 @@ class UsageDelta(TypedDict, total=False):     cache_creation_input_tokens: int     cache_read_input_tokens: int     server_tool_use: ReadOnly[ServerToolUsage]+    service_tier: str   class AppliedEdit(TypedDict, total=False):litellm/types/llms/anthropic_messages/anthropic_response.pyindex 5cf988bd19..18b274da72 100644@@ -83,6 +83,11 @@ class AnthropicUsage(TypedDict, total=False):     """     server_tool_use: NotRequired[ReadOnly[ServerToolUsage]] +    """+    Tier the provider reported serving the request on+    """+    service_tier: str+  class AnthropicMessagesResponse(TypedDict, total=False):     """litellm/types/llms/databricks.pyindex a9c027bd2d..0b68e8429a 100644@@ -2,6 +2,7 @@ from typing import Any, Literal  from pydantic import BaseModel from typing_extensions import (+    NotRequired,     ReadOnly,     Required,     TypedDict,@@ -82,3 +83,4 @@ class DatabricksResponse(TypedDict):     model: str     choices: list[DatabricksChoice]     usage: ChatCompletionUsageBlock+    service_tier: NotRequired[str]tests/unit/llms/databricks/test_databricks_service_tier.pynew file mode 100644index 0000000000..a2a3b45ba3@@ -0,0 +1,331 @@+"""Databricks ``service_tier`` survives request translation, response conversion, streaming, and pricing."""++import asyncio+import json+from collections.abc import AsyncIterator+from typing import Final+from unittest.mock import MagicMock++import httpx+import pytest++import litellm+from litellm.litellm_core_utils.streaming_handler import CustomStreamWrapper+from litellm.llms.anthropic.pass_through.adapters.streaming_iterator import AnthropicStreamWrapper+from litellm.llms.anthropic.pass_through.adapters.transformation import LiteLLMAnthropicMessagesAdapter+from litellm.llms.databricks.chat.transformation import DatabricksChatResponseIterator, DatabricksConfig+from litellm.llms.databricks.cost_calculator import cost_per_token as databricks_cost_per_token+from litellm.responses.litellm_completion_transformation.transformation import LiteLLMCompletionResponsesConfig+from litellm.types.llms.anthropic import AnthropicResponse+from litellm.types.utils import (+    Choices,+    Delta,+    Message,+    ModelResponse,+    ModelResponseStream,+    PromptTokensDetailsWrapper,+    StreamingChoices,+    Usage,+)++MODEL: Final = "databricks/databricks-tier-test-model"+STANDARD_COST: Final = 0.000284+PRIORITY_COST: Final = 0.000568+++@pytest.fixture+def tier_priced_model(monkeypatch: pytest.MonkeyPatch) -> None:+    monkeypatch.setenv("LITELLM_LOCAL_MODEL_COST_MAP", "True")+    monkeypatch.setattr(litellm, "model_cost", litellm.get_model_cost_map(url=""))+    litellm.register_model(+        model_cost={+            MODEL: {+                "litellm_provider": "databricks",+                "mode": "chat",+                "input_cost_per_token": 1e-6,+                "output_cost_per_token": 4e-6,+                "cache_read_input_token_cost": 2e-7,+                "input_cost_per_token_priority": 2e-6,+                "output_cost_per_token_priority": 8e-6,+                "cache_read_input_token_cost_priority": 4e-7,+            }+        }+    )+++def _usage() -> Usage:+    return Usage(+        prompt_tokens=100,+        completion_tokens=50,+        total_tokens=150,+        prompt_tokens_details=PromptTokensDetailsWrapper(cached_tokens=20),+    )+++def _response(service_tier: str | None = None) -> ModelResponse:+    response: Final = ModelResponse(+        model=MODEL,+        choices=[Choices(message=Message(content="ok"), finish_reason="stop")],+        usage=_usage(),+    )+    if service_tier is not None:+        setattr(response, "service_tier", service_tier)  # noqa: B010  # pydantic extra+    return response+++def _cost(response: ModelResponse, **kwargs: object) -> float:+    return litellm.completion_cost(+        completion_response=response, model=MODEL, custom_llm_provider="databricks", **kwargs+    )+++def test_default_pricing_without_tier(tier_priced_model: None) -> None:+    assert _cost(_response()) == pytest.approx(STANDARD_COST)+++def test_priority_pricing_from_explicit_argument(tier_priced_model: None) -> None:+    assert _cost(_response(), service_tier="priority") == pytest.approx(PRIORITY_COST)+++def test_priority_pricing_from_request_params(tier_priced_model: None) -> None:+    assert _cost(_response(), optional_params={"service_tier": "priority"}) == pytest.approx(PRIORITY_COST)+++def test_priority_pricing_from_returned_tier(tier_priced_model: None) -> None:+    assert _cost(_response("priority")) == pytest.approx(PRIORITY_COST)+++def test_returned_default_tier_overrides_requested_priority(tier_priced_model: None) -> None:+    cost: Final = _cost(_response("default"), optional_params={"service_tier": "priority"})+    assert cost == pytest.approx(STANDARD_COST)+++@pytest.mark.parametrize(("service_tier", "expected"), [(None, STANDARD_COST), ("priority", PRIORITY_COST)])+def test_legacy_endpoint_name_resolves_with_tier_pricing(+    monkeypatch: pytest.MonkeyPatch, service_tier: str | None, expected: float+) -> None:+    monkeypatch.setenv("LITELLM_LOCAL_MODEL_COST_MAP", "True")+    monkeypatch.setattr(litellm, "model_cost", litellm.get_model_cost_map(url=""))+    litellm.register_model(+        model_cost={+            "databricks/databricks-dbrx-instruct": {+                "litellm_provider": "databricks",+                "mode": "chat",+                "input_cost_per_token": 1e-6,+                "output_cost_per_token": 4e-6,+                "cache_read_input_token_cost": 2e-7,+                "input_cost_per_token_priority": 2e-6,+                "output_cost_per_token_priority": 8e-6,+                "cache_read_input_token_cost_priority": 4e-7,+            }+        }+    )+    prompt_cost, completion_cost = databricks_cost_per_token(+        model="dbrx-instruct-071224", usage=_usage(), service_tier=service_tier+    )+    assert prompt_cost + completion_cost == pytest.approx(expected)+++def test_responses_to_chat_request_keeps_service_tier() -> None:+    request: Final = LiteLLMCompletionResponsesConfig.transform_responses_api_request_to_chat_completion_request(+        model="databricks-claude-sonnet-4",+        input="hi",+        responses_api_request={"service_tier": "priority"},+        custom_llm_provider="databricks",+    )+    assert request["service_tier"] == "priority"+++def test_databricks_maps_service_tier_with_drop_params() -> None:+    config: Final = DatabricksConfig()+    assert "service_tier" in config.get_supported_openai_params(model="databricks-claude-sonnet-4")+    optional_params: Final = litellm.utils.get_optional_params(+        model="databricks-claude-sonnet-4",+        custom_llm_provider="databricks",+        service_tier="priority",+        drop_params=True,+    )+    assert optional_params["service_tier"] == "priority"+++def _databricks_body(service_tier: str | None) -> dict:+    body: dict = {+        "id": "chatcmpl-1",+        "object": "chat.completion",+        "created": 1700000000,+        "model": "databricks-claude-sonnet-4",+        "choices": [{"index": 0, "message": {"role": "assistant", "content": "ok"}, "finish_reason": "stop"}],+        "usage": {"prompt_tokens": 10, "completion_tokens": 2, "total_tokens": 12},+    }+    if service_tier is not None:+        body["service_tier"] = service_tier+    return body+++def _transform(body: dict) -> ModelResponse:+    return DatabricksConfig().transform_response(+        model="databricks-claude-sonnet-4",+        raw_response=httpx.Response(200, json=body),+        model_response=ModelResponse(),+        logging_obj=MagicMock(),+        request_data={},+        messages=[{"role": "user", "content": "hi"}],+        optional_params={},+        litellm_params={},+        encoding=None,+    )+++def test_databricks_response_preserves_returned_tier() -> None:+    assert _transform(_databricks_body("default")).model_dump().get("service_tier") == "default"+++def test_databricks_response_does_not_manufacture_tier() -> None:+    assert _transform(_databricks_body(None)).model_dump().get("service_tier") is None+++def test_stream_tier_on_terminal_usage_chunk_is_aggregated() -> None:+    parser: Final = DatabricksChatResponseIterator(streaming_response=iter([]), sync_stream=True)+    base: Final = {"id": "chatcmpl-1", "object": "chat.completion.chunk", "created": 1700000000, "model": "m"}+    raw_chunks: Final = (+        {**base, "choices": [{"index": 0, "delta": {"role": "assistant", "content": "ok"}}]},+        {**base, "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}]},+        {+            **base,+            "choices": [],+            "usage": {"prompt_tokens": 10, "completion_tokens": 2, "total_tokens": 12},+            "service_tier": "default",+        },+    )+    chunks: Final = [parser.chunk_parser(chunk) for chunk in raw_chunks]+    assert chunks[-1].model_dump().get("service_tier") == "default"++    rebuilt: Final = litellm.stream_chunk_builder(chunks=chunks)+    assert rebuilt is not None+    assert rebuilt.model_dump().get("service_tier") == "default"++++@pytest.mark.parametrize("include_usage", [True, False])+def test_stream_wrapper_keeps_tier_from_terminal_usage_chunk(include_usage: bool) -> None:+    base: Final = {"id": "chatcmpl-1", "object": "chat.completion.chunk", "created": 1700000000, "model": "m"}+    raw_chunks: Final = (+        {**base, "choices": [{"index": 0, "delta": {"role": "assistant", "content": "ok"}}]},+        {**base, "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}]},+        {+            **base,+            "choices": [],+            "usage": {"prompt_tokens": 10, "completion_tokens": 2, "total_tokens": 12},+            "service_tier": "default",+        },+    )+    lines: Final = [f"data: {json.dumps(chunk)}" for chunk in raw_chunks] + ["data: [DONE]"]+    wrapper: Final = CustomStreamWrapper(+        completion_stream=DatabricksChatResponseIterator(streaming_response=iter(lines), sync_stream=True),+        model="m",+        custom_llm_provider="databricks",+        logging_obj=MagicMock(),+        stream_options={"include_usage": True} if include_usage else None,+    )+    list(wrapper)++    rebuilt: Final = litellm.stream_chunk_builder(chunks=wrapper.chunks)+    assert rebuilt is not None+    assert rebuilt.model_dump().get("service_tier") == "default"++def test_chat_to_responses_response_preserves_returned_tier() -> None:+    converted: Final = LiteLLMCompletionResponsesConfig.transform_chat_completion_response_to_responses_api_response(+        request_input="hi",+        responses_api_request={"service_tier": "priority"},+        chat_completion_response=_response("default"),+    )+    assert converted.model_dump().get("service_tier") == "default"+++def test_chat_to_responses_response_does_not_manufacture_tier_from_request() -> None:+    converted: Final = LiteLLMCompletionResponsesConfig.transform_chat_completion_response_to_responses_api_response(+        request_input="hi",+        responses_api_request={"service_tier": "priority"},+        chat_completion_response=_response(),+    )+    assert converted.model_dump().get("service_tier") is None+++def test_messages_to_chat_request_keeps_service_tier() -> None:+    request, _ = LiteLLMAnthropicMessagesAdapter().translate_anthropic_to_openai(+        anthropic_message_request={+            "model": "databricks-claude-sonnet-4",+            "max_tokens": 10,+            "messages": [{"role": "user", "content": "hi"}],+            "service_tier": "priority",+        },+        custom_llm_provider="databricks",+    )+    assert request["service_tier"] == "priority"+++def test_chat_to_messages_response_preserves_returned_tier() -> None:+    translated: Final = LiteLLMAnthropicMessagesAdapter().translate_openai_response_to_anthropic(+        response=_response("priority")+    )+    assert translated["usage"]["service_tier"] == "priority"+++def test_chat_to_messages_response_does_not_manufacture_tier() -> None:+    translated: Final = LiteLLMAnthropicMessagesAdapter().translate_openai_response_to_anthropic(response=_response())+    assert "service_tier" not in translated["usage"]+++def _messages_stream_chunks(service_tier: str | None) -> list[ModelResponseStream]:+    usage_chunk: Final = ModelResponseStream(+        choices=[], usage=Usage(prompt_tokens=10, completion_tokens=2, total_tokens=12)+    )+    if service_tier is not None:+        setattr(usage_chunk, "service_tier", service_tier)  # noqa: B010  # pydantic extra+    return [+        ModelResponseStream(choices=[StreamingChoices(index=0, delta=Delta(content="ok"), finish_reason=None)]),+        ModelResponseStream(choices=[StreamingChoices(index=0, delta=Delta(), finish_reason="stop")]),+        usage_chunk,+    ]+++def _message_delta_usage(events: list) -> dict:+    return next(event for event in events if event.get("type") == "message_delta")["usage"]+++def test_messages_stream_reports_tier_from_terminal_usage_chunk() -> None:+    wrapper: Final = AnthropicStreamWrapper(completion_stream=iter(_messages_stream_chunks("default")), model="m")+    usage: Final = _message_delta_usage(list(wrapper))+    assert usage["service_tier"] == "default"+    assert usage["input_tokens"] == 10+++def test_async_messages_stream_reports_tier_from_terminal_usage_chunk() -> None:+    async def _aiter() -> AsyncIterator[ModelResponseStream]:+        for chunk in _messages_stream_chunks("priority"):+            yield chunk++    async def _collect() -> list:+        wrapper: Final = AnthropicStreamWrapper(completion_stream=_aiter(), model="m")+        return [event async for event in wrapper]++    assert _message_delta_usage(asyncio.run(_collect()))["service_tier"] == "priority"+++def test_messages_stream_does_not_manufacture_tier() -> None:+    wrapper: Final = AnthropicStreamWrapper(completion_stream=iter(_messages_stream_chunks(None)), model="m")+    assert "service_tier" not in _message_delta_usage(list(wrapper))+++def test_messages_bridge_logged_response_prices_returned_tier(tier_priced_model: None) -> None:+    """The /v1/messages spend row is rebuilt from the Anthropic-shaped response; the served tier must survive."""+    translated: Final = LiteLLMAnthropicMessagesAdapter().translate_openai_response_to_anthropic(+        response=_response("priority")+    )+    logged: Final = litellm.AnthropicConfig().transform_parsed_response(+        completion_response=AnthropicResponse.model_validate(translated).model_dump(),+        raw_response=httpx.Response(status_code=200, headers={}),+        model_response=ModelResponse(),+        json_mode=None,+    )+    assert _cost(logged) == pytest.approx(PRIORITY_COST) 
Merged fix · PR #46111+55 −1
litellm/llms/databricks/chat/transformation.pyindex 50d3b82a989..dceec658824 100644@@ -285,6 +285,7 @@ class DatabricksConfig(DatabricksBase, OpenAILikeChatConfig, AnthropicConfig):             "tool_choice",             "reasoning_effort",             "thinking",+            "service_tier",         ]      @staticmethod@@ -697,6 +698,9 @@ class DatabricksConfig(DatabricksBase, OpenAILikeChatConfig, AnthropicConfig):         model_response.id = completion_response["id"]         model_response.created = completion_response["created"]         setattr(model_response, "usage", Usage(**completion_response["usage"]))+        served_tier: Final = completion_response.get("service_tier")+        if isinstance(served_tier, str) and served_tier:+            setattr(model_response, "service_tier", served_tier)  # noqa: B010  # no declared field          model_response.choices = self._transform_dbrx_choices(             choices=completion_response["choices"],litellm/responses/litellm_completion_transformation/transformation.pyindex 4f99c570551..c2cc876970c 100644@@ -500,7 +500,7 @@ class LiteLLMCompletionResponsesConfig:             "max_tokens": responses_api_request.get("max_output_tokens"),             "stream": stream,             "metadata": kwargs.get("metadata"),-            "service_tier": kwargs.get("service_tier"),+            "service_tier": responses_api_request.get("service_tier"),             "safety_identifier": responses_api_request.get("safety_identifier"),             "web_search_options": web_search_options,             "response_format": response_format,tests/unit/llms/databricks/chat/test_databricks_chat_transformation.pyindex c6e0ecd2b41..fe76a667f3e 100644@@ -2216,3 +2216,42 @@ def test_completion_with_prompt_caching_anthropic_model(     assert response.usage.total_tokens == 1666     assert response.usage.cache_read_input_tokens == 0     assert response.usage.cache_creation_input_tokens == 1545+++def test_get_optional_params_keeps_service_tier_for_databricks() -> None:+    optional_params: Final = litellm.utils.get_optional_params(+        model="databricks-claude-opus-5",+        custom_llm_provider="databricks",+        service_tier="priority",+        drop_params=True,+    )++    assert optional_params["service_tier"] == "priority"+++@pytest.mark.parametrize("served_tier", ["priority", "default", None])+def test_transform_response_carries_served_service_tier(served_tier: str | None) -> None:+    raw_response: Final = MagicMock()+    raw_response.json.return_value = {+        "id": "chatcmpl-1",+        "object": "chat.completion",+        "created": 0,+        "model": "dbrx",+        **({"service_tier": served_tier} if served_tier is not None else {}),+        "choices": [{"index": 0, "message": {"role": "assistant", "content": "ok"}, "finish_reason": "stop"}],+        "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2},+    }++    result: Final = DatabricksConfig().transform_response(+        model="databricks/dbrx",+        raw_response=raw_response,+        model_response=litellm.ModelResponse(),+        logging_obj=MagicMock(),+        request_data={},+        messages=[{"role": "user", "content": "hi"}],+        optional_params={},+        litellm_params={},+        encoding=None,+    )++    assert result.to_dict().get("service_tier") == served_tiertests/unit/responses/litellm_completion_transformation/test_litellm_completion_responses.pyindex c7f20a1d959..7006feb22b2 100644@@ -5180,3 +5180,14 @@ def test_convert_response_function_tool_call_keeps_the_index_it_is_given(index:     )      assert tool_call["index"] == index+++def test_transform_request_reads_service_tier_from_responses_api_request() -> None:+    result: Final = LiteLLMCompletionResponsesConfig.transform_responses_api_request_to_chat_completion_request(+        model="databricks/dbrx",+        input="hello",+        responses_api_request={"service_tier": "priority"},+        custom_llm_provider="databricks",+    )++    assert result["service_tier"] == "priority"