Backtestslitellmrun-20261111-220245#37727

Issue #37727 Merged fix PR #40147 Run 11 Nov 2026 · run-20261111-220245 BerriAI/litellm

[Feature]: azure_ai image generation and OCR should honour the Entra ID token fallback (enable_azure_ad_token_refresh) like azure_ai chat does

Better

Our patch was judged better than the merged fix.

In this judgment, Patch A = our agent, Patch B = the merged fix.

Each patch fixes only one half of what the issue asks, so the question is which half matters more and which patch is the cleaner fix.

Patch A changes only the OCR dispatch layer: when the global `enable_azure_ad_token_refresh` is on and the provider is `azure_ai`, it forwards that flag into the native OCR route's kwargs so the shared Azure AD token resolution can run. The independent tests confirm this is the real gap: all five key-less OCR scenarios (sync, async, default route, native route, and Router row) fail on base with the exact 'Missing Azure AI API Key' error the issue quotes and pass with A. A also adds end-to-end tests that stand up a fake managed-identity endpoint and Foundry server, parametrize over both OCR routes, assert the route actually served the request (so the native case can't silently fall back), and cover Router usage. Its own MAI image tests exercise the issue's named image route without any image code change, which suggests that path already honours the token on this base; A does not break it.

Patch B never touches OCR. The issue's hard-failure case (`litellm.ocr` refusing before the call) remains broken under B, as all five OCR tests show. B instead reworks header/auth handling in `AzureChatCompletion.image_generation` (the `openai/deployments` path), which makes a key-less `azure_ai/gpt-image-1` deployment send a Bearer token. That is a legitimate adjacent gap under the issue's 'any other azure_ai non-chat' clause, and B's tests are solid, but B also pulls in extra surface: a new `lru_cache`d token-provider keyed only by scope, generic auth-header injection, and log redaction. Those are reasonable hardening but add risk in a hot path and obscure the fix, and they do not address the OCR failure the reporter measured.

Weighing correctness first: A resolves the issue's explicitly reported, still-failing OCR case at the point the credential is dropped, with tests that fail before and pass after. B fixes an image variant the issue only mentions generically while leaving the quoted OCR failure intact. A is the better fix.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

58m 48s

Wall-clock time in the container

Full test suite

No new failures

774 passed · 32 skipped · 806 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

417vs 374 merged

Changed lines, ours vs merged

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

The Feature

Make the azure_ai/ provider's non-chat paths honour the same Entra ID token fallback that the azure/ provider and the azure_ai/ chat path already have, so a Foundry deployment can be called key-less with a managed identity.

Today (v1.97.0), with litellm_settings.enable_azure_ad_token_refresh: true and AZURE_CLIENT_ID set (user-assigned managed identity), a model row that resolves no api_key behaves three different ways depending on the route:

routekey-less result
azure/* chat, embeddings (via initialize_azure_sdk_client / get_azure_ad_token)✅ token from get_azure_ad_token_provider → 200
azure_ai/* chat (AzureAIStudioConfig.validate_environment → BaseAzureLLM._base_validate_azure_environment)✅ Authorization: Bearer <token> → 200 (tested with a MaaS deployment on *.services.ai.azure.com)
azure_ai/* image generation (llms/azure_ai/image_generation, MAI /mai/v1/images/generations)❌ request goes out with no credential at all; provider answers 401 Access denied due to invalid subscription key
azure_ai/* OCR (/v1/ocr, llms/azure_ai/ocr)❌ litellm refuses before calling: Missing Azure AI API Key - A call is being made to Azure AI but no key is set either in the environment variables or via params

The ask: route the image-generation and OCR (and any other azure_ai non-chat) validate_environment through the shared Azure AD token resolution (get_azure_ad_token / get_azure_ad_token_provider, scope https://cognitiveservices.azure.com/.default, which *.services.ai.azure.com accepts) when api_key is None, instead of hard-requiring a key. The chat path already shows the pattern.

User Flow

Before: an operator running the proxy on Azure with a managed identity (keyless everywhere) adds Foundry image/OCR deployments.

  1. The operator sets enable_azure_ad_token_refresh: true, attaches a UAMI with Cognitive Services User / Cognitive Services OpenAI User on the Foundry accounts, and creates rows with api_base only.
  2. azure/gpt-* and azure_ai/<MaaS chat model> rows serve 200 with a token.
  3. azure_ai/MAI-Image-2.5* rows return 401 from the provider; azure_ai/mistral-document-ai-* / mistral-ocr-* rows fail with "Missing Azure AI API Key".
  4. The operator has to keep account keys in the DB for exactly those rows — a split fleet (some rows keyless, some keyed) on the same Foundry account.

After: the same rows pick up the Entra token like the chat route does, and the Foundry account key can be retired.

How far you got

Measured on the proxy, ghcr.io/berriai/litellm:1.97.0, Azure Container Apps, user-assigned managed identity with Cognitive Services OpenAI User + Cognitive Services User on the resource group holding the Foundry accounts, AZURE_CLIENT_ID set, enable_azure_ad_token_refresh: true:

  • azure/gpt-5-nano row, api_base only → 200, correct x-litellm-response-cost.
  • azure_ai/DeepSeek-V4-Pro chat row, api_base: https://<account>.services.ai.azure.com, no key → 200.
  • azure_ai/MAI-Image-2.5-Flash image row, same account shape, no key → litellm.AuthenticationError: Azure_aiException - {"error":{"code":"401","message":"Access denied due to invalid subscription key or wrong API endpoint. ..."}}.
  • azure_ai/mistral-document-ai-2512 OCR row, no key → litellm.APIConnectionError: Azure_aiException - Missing Azure AI API Key - A call is being made to Azure AI but no key is set either in the environment variables or via params.
  • Re-adding the key to the same image/OCR rows → 200, so the endpoints and RBAC are fine; only the credential resolution differs per route.

Happy to test a branch against the same deployments.

What part of LiteLLM is this about?

litellm (core), providers: azure_ai

Twitter / LinkedIn details

_No response_

The two patches

Our patch+414 −3
litellm/ocr/dispatch.pyindex 80c93273d1..316206d17d 100644@@ -3,11 +3,12 @@ from typing import Final, cast  # noqa: TID251  # native binding selects a sync  import httpx +import litellm from litellm.llms.base_llm.ocr.transformation import OCRResponse from litellm.ocr import main from litellm.ocr.main import convert_file_document_to_url_document, get_mime_type from litellm.rust_bridge.catalog import Context, Route-from litellm.rust_bridge.dispatch import PublicDispatch, call_hook+from litellm.rust_bridge.dispatch import NativeHook, PublicDispatch, ResultT, call_hook from litellm.rust_bridge.ocr.entrypoints import NATIVE_AOCR, NATIVE_OCR, LiteLLMOcrRequest  __all__ = ("aocr", "convert_file_document_to_url_document", "get_mime_type", "ocr")@@ -56,6 +57,31 @@ def _context(request: LiteLLMOcrRequest) -> Context:     return Context(Route.OCR, provider=request.custom_llm_provider, model=request.model)  +AZURE_AD_TOKEN_REFRESH_PARAM: Final = "enable_azure_ad_token_refresh"+++def _native_kwargs(request: LiteLLMOcrRequest, kwargs: Mapping[str, object]) -> Mapping[str, object]:+    """+    The native route only sees per-call options, so carry `litellm.enable_azure_ad_token_refresh`+    over for `azure_ai` calls; the Python route reads the global itself.+    """+    provider: Final = request.custom_llm_provider or request.model.partition("/")[0]+    if provider != "azure_ai" or kwargs.get(AZURE_AD_TOKEN_REFRESH_PARAM) is not None:+        return kwargs+    if litellm.enable_azure_ad_token_refresh is not True:+        return kwargs+    return {**kwargs, AZURE_AD_TOKEN_REFRESH_PARAM: True}+++def _call_native(+    hook: NativeHook[LiteLLMOcrRequest, ResultT],+    request: LiteLLMOcrRequest,+    args: tuple[object, ...],+    kwargs: Mapping[str, object],+) -> ResultT:+    return call_hook(hook, request, args, _native_kwargs(request, kwargs))++ _DISPATCH: Final = PublicDispatch(     route=Route.OCR,     request=lambda args, kwargs: _public_request("ocr", args, kwargs),@@ -79,7 +105,7 @@ def ocr(         kwargs,         python=_PYTHON_OCR,         binding=NATIVE_OCR,-        native=call_hook,+        native=_call_native,     )  @@ -89,5 +115,5 @@ async def aocr(*args: object, **kwargs: object) -> OCRResponse:  # kwargs-ok: pr         kwargs,         python=_PYTHON_AOCR,         binding=NATIVE_AOCR,-        native=call_hook,+        native=_call_native,     )tests/test_litellm/llms/azure_ai/conftest.pynew file mode 100644index 0000000000..7151351289@@ -0,0 +1,111 @@+"""+A local stand-in for an Azure Container Apps / App Service managed identity and an Azure AI Foundry+account, so key-less `azure_ai` routes can be exercised end to end.+"""++import json+import threading+import time+from collections.abc import Generator+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer+from urllib.parse import parse_qs, urlparse++import pytest++import litellm+from tests.test_litellm.llms.azure_ai.managed_identity_server import (+    IDENTITY_HEADER,+    MANAGED_IDENTITY_CLIENT_ID,+    MANAGED_IDENTITY_TOKEN,+    FoundryRequest,+    FoundryServer,+)+++@pytest.fixture+def foundry_server() -> Generator[FoundryServer, None, None]:+    server_state: FoundryServer | None = None++    class Handler(BaseHTTPRequestHandler):+        def log_message(self, format: str, *args: object) -> None:+            return++        def _reply(self, status: int, body: dict[str, object]) -> None:+            payload = json.dumps(body).encode()+            self.send_response(status)+            self.send_header("Content-Type", "application/json")+            self.send_header("Content-Length", str(len(payload)))+            self.end_headers()+            self.wfile.write(payload)++        def do_GET(self) -> None:+            assert server_state is not None+            parsed = urlparse(self.path)+            if parsed.path != "/msi/token" or self.headers.get("X-IDENTITY-HEADER") != IDENTITY_HEADER:+                self._reply(400, {"error": "unexpected identity request"})+                return+            query = parse_qs(parsed.query)+            server_state.token_requests.append(query)+            self._reply(+                200,+                {+                    "access_token": MANAGED_IDENTITY_TOKEN,+                    "expires_on": str(int(time.time()) + 3600),+                    "resource": query.get("resource", [""])[0],+                    "token_type": "Bearer",+                    "client_id": query.get("client_id", [""])[0],+                },+            )++        def do_POST(self) -> None:+            assert server_state is not None+            self.rfile.read(int(self.headers.get("Content-Length", "0")))+            authorization = self.headers.get("Authorization")+            server_state.requests.append(+                FoundryRequest(path=self.path, authorization=authorization, api_key=self.headers.get("api-key"))+            )+            if authorization != f"Bearer {MANAGED_IDENTITY_TOKEN}":+                self._reply(401, {"error": {"code": "401", "message": "Access denied due to invalid subscription key"}})+                return+            self._reply(200, server_state.post_response)++    httpd = ThreadingHTTPServer(("127.0.0.1", 0), Handler)+    server_state = FoundryServer(base_url=f"http://127.0.0.1:{httpd.server_address[1]}")+    thread = threading.Thread(target=httpd.serve_forever, daemon=True)+    thread.start()+    try:+        yield server_state+    finally:+        httpd.shutdown()+        httpd.server_close()+++@pytest.fixture+def managed_identity_env(monkeypatch: pytest.MonkeyPatch, foundry_server: FoundryServer) -> FoundryServer:+    """`enable_azure_ad_token_refresh: true` + a user-assigned identity, and no Azure AI key anywhere."""+    for env_var in (+        "AZURE_AI_API_KEY",+        "AZURE_AI_API_BASE",+        "AZURE_API_KEY",+        "AZURE_OPENAI_API_KEY",+        "AZURE_AD_TOKEN",+        "AZURE_TENANT_ID",+        "AZURE_CLIENT_SECRET",+        "AZURE_USERNAME",+        "AZURE_PASSWORD",+        "AZURE_CREDENTIAL",+        "AZURE_SCOPE",+        "AZURE_FEDERATED_TOKEN_FILE",+        "IDENTITY_SERVER_THUMBPRINT",+        "IMDS_ENDPOINT",+        "MSI_ENDPOINT",+        "MSI_SECRET",+    ):+        monkeypatch.delenv(env_var, raising=False)+    monkeypatch.setenv("AZURE_CLIENT_ID", MANAGED_IDENTITY_CLIENT_ID)+    monkeypatch.setenv("IDENTITY_ENDPOINT", f"{foundry_server.base_url}/msi/token")+    monkeypatch.setenv("IDENTITY_HEADER", IDENTITY_HEADER)+    monkeypatch.setattr(litellm, "api_key", None)+    monkeypatch.setattr(litellm, "azure_key", None)+    monkeypatch.setattr(litellm, "enable_azure_ad_token_refresh", True)+    return foundry_servertests/test_litellm/llms/azure_ai/image_generation/test_azure_ai_image_generation_managed_identity.pynew file mode 100644index 0000000000..dc26b46349@@ -0,0 +1,65 @@+"""+Key-less Azure AI image generation with a user-assigned managed identity.++With `litellm.enable_azure_ad_token_refresh = True`, `AZURE_CLIENT_ID` set and no API key, an+`azure_ai` MAI image deployment must send the managed identity's Entra token to+`/mai/v1/images/generations` instead of a request with no credential.+"""++import base64+from typing import Final++import pytest++import litellm+from tests.test_litellm.llms.azure_ai.managed_identity_server import FoundryServer, assert_sent_managed_identity_token++MAI_MODEL: Final = "azure_ai/MAI-Image-2.5-Flash"+IMAGE_B64: Final = base64.b64encode(b"managed identity image").decode()+++@pytest.fixture+def managed_identity(managed_identity_env: FoundryServer) -> FoundryServer:+    managed_identity_env.post_response = {"created": 1, "data": [{"b64_json": IMAGE_B64}]}+    return managed_identity_env+++def assert_mai_image_request(server: FoundryServer, response: litellm.ImageResponse) -> None:+    assert response.data is not None+    assert response.data[0].b64_json == IMAGE_B64+    assert_sent_managed_identity_token(server)+    assert server.requests[0].path.startswith("/mai/v1/images/generations")+++def test_image_generation_without_api_key_uses_managed_identity_token(managed_identity: FoundryServer) -> None:+    response = litellm.image_generation(model=MAI_MODEL, prompt="a red circle", api_base=managed_identity.base_url)++    assert_mai_image_request(managed_identity, response)+++@pytest.mark.asyncio+async def test_aimage_generation_without_api_key_uses_managed_identity_token(managed_identity: FoundryServer) -> None:+    response = await litellm.aimage_generation(+        model=MAI_MODEL, prompt="a red circle", api_base=managed_identity.base_url+    )++    assert_mai_image_request(managed_identity, response)+++@pytest.mark.asyncio+async def test_router_image_deployment_without_api_key_uses_managed_identity_token(+    managed_identity: FoundryServer,+) -> None:+    router = litellm.Router(+        model_list=[+            {+                "model_name": "mai-image",+                "litellm_params": {"model": MAI_MODEL, "api_base": managed_identity.base_url},+            }+        ],+        num_retries=0,+    )++    response = await router.aimage_generation(model="mai-image", prompt="a red circle")++    assert_mai_image_request(managed_identity, response)tests/test_litellm/llms/azure_ai/managed_identity_server.pynew file mode 100644index 0000000000..eab75aa074@@ -0,0 +1,33 @@+"""Shared helpers for the local managed identity + Azure AI Foundry stand-in served by `conftest.py`."""++from dataclasses import dataclass, field+from typing import Final++MANAGED_IDENTITY_CLIENT_ID: Final = "uami-client-id"+MANAGED_IDENTITY_TOKEN: Final = "managed-identity-token"+IDENTITY_HEADER: Final = "identity-header-secret"+++@dataclass+class FoundryRequest:+    path: str+    authorization: str | None+    api_key: str | None+++@dataclass+class FoundryServer:+    """Serves the managed identity endpoint (GET /msi/token) and answers every Foundry POST."""++    base_url: str+    post_response: dict[str, object] = field(default_factory=dict)+    token_requests: list[dict[str, list[str]]] = field(default_factory=list)+    requests: list[FoundryRequest] = field(default_factory=list)+++def assert_sent_managed_identity_token(server: FoundryServer) -> None:+    assert [(request.authorization, request.api_key) for request in server.requests] == [+        (f"Bearer {MANAGED_IDENTITY_TOKEN}", None)+    ]+    for token_request in server.token_requests:+        assert token_request.get("client_id") == [MANAGED_IDENTITY_CLIENT_ID]tests/test_litellm/llms/azure_ai/ocr/test_azure_ai_ocr_managed_identity.pynew file mode 100644index 0000000000..2226c01d07@@ -0,0 +1,83 @@+"""+Key-less Azure AI OCR with a user-assigned managed identity.++With `litellm.enable_azure_ad_token_refresh = True`, `AZURE_CLIENT_ID` set and no API key, an+`azure_ai` OCR deployment must authenticate with the managed identity's Entra token, the same+way `azure_ai` chat does, on both the native and the Python OCR route.+"""++from typing import Final++import pytest++import litellm+from litellm.llms.base_llm.ocr.transformation import OCRResponse+from litellm.router_utils.add_retry_fallback_headers import get_hidden_params_dict+from tests.test_litellm.llms.azure_ai.managed_identity_server import FoundryServer, assert_sent_managed_identity_token++OCR_MODEL: Final = "azure_ai/mistral-document-ai-2512"+OCR_DOCUMENT: Final = {"type": "document_url", "document_url": "data:application/pdf;base64,JVBERi0xLjQK"}+OCR_RESPONSE: Final = {+    "pages": [{"index": 0, "markdown": "managed identity OCR", "images": [], "dimensions": None}],+    "model": "mistral-document-ai-2512",+    "usage_info": {"pages_processed": 1},+}+++@pytest.fixture(params=["0", "1"], ids=["python-route", "native-route"])+def ocr_route(request: pytest.FixtureRequest, monkeypatch: pytest.MonkeyPatch) -> str:+    monkeypatch.setenv("LITELLM_RUST", request.param)+    return request.param+++@pytest.fixture+def managed_identity(ocr_route: str, managed_identity_env: FoundryServer) -> FoundryServer:+    managed_identity_env.post_response = OCR_RESPONSE+    return managed_identity_env+++def assert_served_by_route(response: OCRResponse, ocr_route: str) -> None:+    """The native-route cases must not pass by silently falling back to the Python route."""+    additional_headers = get_hidden_params_dict(response).get("additional_headers") or {}+    assert isinstance(additional_headers, dict)+    assert (additional_headers.get("x-litellm-rust") == "true") is (ocr_route == "1")+++def test_ocr_without_api_key_uses_managed_identity_token(managed_identity: FoundryServer, ocr_route: str) -> None:+    response = litellm.ocr(model=OCR_MODEL, document=OCR_DOCUMENT, api_base=managed_identity.base_url)++    assert response.pages[0].markdown == "managed identity OCR"+    assert_served_by_route(response, ocr_route)+    assert_sent_managed_identity_token(managed_identity)+++@pytest.mark.asyncio+async def test_aocr_without_api_key_uses_managed_identity_token(+    managed_identity: FoundryServer, ocr_route: str+) -> None:+    response = await litellm.aocr(model=OCR_MODEL, document=OCR_DOCUMENT, api_base=managed_identity.base_url)++    assert response.pages[0].markdown == "managed identity OCR"+    assert_served_by_route(response, ocr_route)+    assert_sent_managed_identity_token(managed_identity)+++@pytest.mark.asyncio+async def test_router_ocr_deployment_without_api_key_uses_managed_identity_token(+    managed_identity: FoundryServer, ocr_route: str+) -> None:+    router = litellm.Router(+        model_list=[+            {+                "model_name": "document-ai",+                "litellm_params": {"model": OCR_MODEL, "api_base": managed_identity.base_url},+            }+        ],+        num_retries=0,+    )++    response = await router.aocr(model="document-ai", document=OCR_DOCUMENT)++    assert response.pages[0].markdown == "managed identity OCR"+    assert_served_by_route(response, ocr_route)+    assert_sent_managed_identity_token(managed_identity)tests/test_litellm/ocr/test_dispatch.pyindex 14d3368f86..6247a75d3d 100644@@ -387,3 +387,96 @@ async def test_public_aocr_routes_through_dispatch(monkeypatch: pytest.MonkeyPat         NATIVE_AOCR.reset()     assert result is expected     assert [request.model for request in captured] == ["mistral/mistral-ocr-latest"]+++AZURE_DOCUMENT: Final[Mapping[str, object]] = {+    "type": "document_url",+    "document_url": "data:application/pdf;base64,JVBERi0xLjQK",+}+++@pytest.mark.parametrize(+    ("call_kwargs", "global_refresh", "expected_refresh"),+    [+        pytest.param({"model": "azure_ai/mistral-document-ai-2512"}, True, True, id="azure-ai-prefix-inherits-global"),+        pytest.param(+            {"model": "mistral-document-ai-2512", "custom_llm_provider": "azure_ai"},+            True,+            True,+            id="azure-ai-provider-inherits-global",+        ),+        pytest.param(+            {"model": "azure_ai/mistral-document-ai-2512", "enable_azure_ad_token_refresh": False},+            True,+            False,+            id="per-call-value-wins",+        ),+        pytest.param(+            {"model": "azure_ai/mistral-document-ai-2512", "enable_azure_ad_token_refresh": None},+            True,+            True,+            id="per-call-none-inherits-global",+        ),+        pytest.param({"model": "azure_ai/mistral-document-ai-2512"}, False, None, id="global-off"),+        pytest.param({"model": "azure_ai/mistral-document-ai-2512"}, None, None, id="global-unset"),+        pytest.param({"model": "mistral/mistral-ocr-latest"}, True, None, id="non-azure-provider-untouched"),+    ],+)+def test_public_ocr_forwards_azure_ad_token_refresh_to_native(+    monkeypatch: pytest.MonkeyPatch,+    call_kwargs: Mapping[str, object],+    global_refresh: bool | None,+    expected_refresh: bool | None,+) -> None:+    captured: Final[list[Mapping[str, object]]] = []+    expected: Final = response()++    def native(+        request: LiteLLMOcrRequest,+        args: tuple[object, ...],+        kwargs: Mapping[str, object],+    ) -> OCRResponse:+        captured.append(kwargs)+        return expected++    NATIVE_OCR.override(native)+    monkeypatch.setattr(catalog, "RULES", RUST_RULES)+    monkeypatch.setattr(litellm, "enable_azure_ad_token_refresh", global_refresh)+    public_ocr: Final = cast(Callable[..., OCRResponse], litellm.ocr)+    try:+        result: Final = public_ocr(document=AZURE_DOCUMENT, **call_kwargs)+    finally:+        NATIVE_OCR.reset()++    assert result is expected+    if expected_refresh is None:+        assert "enable_azure_ad_token_refresh" not in captured[0]+    else:+        assert captured[0]["enable_azure_ad_token_refresh"] is expected_refresh+++@pytest.mark.asyncio+async def test_public_aocr_forwards_azure_ad_token_refresh_to_native(monkeypatch: pytest.MonkeyPatch) -> None:+    captured: Final[list[Mapping[str, object]]] = []+    expected: Final = response()++    async def native(+        request: LiteLLMOcrRequest,+        args: tuple[object, ...],+        kwargs: Mapping[str, object],+    ) -> OCRResponse:+        captured.append(kwargs)+        return expected++    NATIVE_AOCR.override(native)+    monkeypatch.setattr(catalog, "RULES", RUST_RULES)+    monkeypatch.setattr(litellm, "enable_azure_ad_token_refresh", True)+    public_aocr: Final = cast(Callable[..., Awaitable[OCRResponse]], litellm.aocr)+    try:+        result: Final = await public_aocr(model="azure_ai/mistral-document-ai-2512", document=AZURE_DOCUMENT)+    finally:+        NATIVE_AOCR.reset()++    assert result is expected+    assert captured[0]["enable_azure_ad_token_refresh"] is True+    assert captured[0]["document"] is AZURE_DOCUMENT 
Merged fix · PR #40147+357 −17
litellm/llms/azure/azure.pyindex 3cb17259b93..449319c1b95 100644@@ -46,7 +46,9 @@ from .common_utils import (     AzureOpenAIError,     BaseAzureLLM,     get_azure_ad_token_from_oidc,+    get_azure_request_auth_headers,     process_azure_headers,+    redact_azure_auth_headers,     select_azure_base_url_or_endpoint, ) from .image_generation import (@@ -1145,7 +1147,7 @@ class AzureChatCompletion(BaseAzureLLM, BaseLLM):         api_key: str,         input: list,         logging_obj: LiteLLMLoggingObj,-        headers: dict,+        headers: dict[str, str],         client=None,         timeout=None,         model: str | None = None,@@ -1170,7 +1172,7 @@ class AzureChatCompletion(BaseAzureLLM, BaseLLM):                 additional_args={                     "complete_input_dict": data,                     "api_base": img_gen_api_base,-                    "headers": headers,+                    "headers": redact_azure_auth_headers(headers),                 },             )             httpx_response: Final[httpx.Response] = await self.make_async_azure_httpx_request(@@ -1229,7 +1231,7 @@ class AzureChatCompletion(BaseAzureLLM, BaseLLM):         timeout: float,         optional_params: dict,         logging_obj: LiteLLMLoggingObj,-        headers: dict,+        headers: dict[str, str],         model: str | None = None,         api_key: str | None = None,         api_base: str | None = None,@@ -1264,21 +1266,22 @@ class AzureChatCompletion(BaseAzureLLM, BaseLLM):             if not isinstance(max_retries, int):                 raise AzureOpenAIError(status_code=422, message="max retries must be an int") -            if api_key is None and azure_ad_token_provider is not None:-                azure_ad_token = azure_ad_token_provider()-                if azure_ad_token:-                    headers.pop("api-key", None)-                    headers["Authorization"] = f"Bearer {azure_ad_token}"--            # init AzureOpenAI Client+            auth_params: Final[dict[str, object]] = {**(litellm_params or {})}  # mutable-ok: SDK init takes a dict+            if azure_ad_token is not None:+                auth_params["azure_ad_token"] = azure_ad_token+            if azure_ad_token_provider is not None:+                auth_params["azure_ad_token_provider"] = azure_ad_token_provider             azure_client_params: Final[dict[str, object]] = self.initialize_azure_sdk_client(-                litellm_params=litellm_params or {},+                litellm_params=auth_params,                 api_key=api_key,                 model_name=model or "",                 api_version=api_version,                 api_base=api_base,                 is_async=False,             )+            request_headers: Final = dict(  # mutable-ok: the httpx request helpers take a dict+                get_azure_request_auth_headers(headers=headers, azure_client_params=azure_client_params)+            )             if aimg_generation is True:                 return self.aimage_generation(                     data=data,@@ -1289,7 +1292,7 @@ class AzureChatCompletion(BaseAzureLLM, BaseLLM):                     client=client,                     azure_client_params=azure_client_params,                     timeout=timeout,-                    headers=headers,+                    headers=request_headers,                     model=model,                 ) @@ -1306,7 +1309,7 @@ class AzureChatCompletion(BaseAzureLLM, BaseLLM):                 additional_args={                     "complete_input_dict": data,                     "api_base": img_gen_api_base,-                    "headers": headers,+                    "headers": redact_azure_auth_headers(request_headers),                 },             )             httpx_response: Final[httpx.Response] = self.make_sync_azure_httpx_request(@@ -1316,7 +1319,7 @@ class AzureChatCompletion(BaseAzureLLM, BaseLLM):                 api_version=api_version or "",                 api_key=api_key or "",                 data=data,-                headers=headers,+                headers=request_headers,                 deployment_name=model,             )             provider_config: Final = get_azure_image_generation_config(data.get("model", "dall-e-2"))litellm/llms/azure/common_utils.pyindex e6b3eb1f2bb..c8a146be5cd 100644@@ -96,6 +96,11 @@ def _cached_entra_id_token_provider(     return get_bearer_token_provider(ClientSecretCredential(tenant_id, client_id, client_secret), scope)  +@lru_cache(maxsize=128)+def _cached_azure_ad_token_refresh_provider(scope: str) -> Callable[[], str]:+    return get_azure_ad_token_provider(azure_scope=scope)++ def get_azure_ad_token_from_entra_id(     tenant_id: str,     client_id: str,@@ -406,6 +411,41 @@ def get_azure_ad_token(     return azure_ad_token  +_AZURE_AUTH_HEADER_NAMES: Final = frozenset(("api-key", "authorization"))+_REDACTED_AZURE_HEADER_VALUE: Final = "***REDACTED***"+++def _resolve_azure_ad_token(azure_client_params: Mapping[str, object]) -> str | None:+    azure_ad_token: Final = azure_client_params.get("azure_ad_token")+    if isinstance(azure_ad_token, str) and azure_ad_token:+        return azure_ad_token+    token_provider: Final = azure_client_params.get("azure_ad_token_provider")+    provided_token: Final = token_provider() if callable(token_provider) else None+    return provided_token if isinstance(provided_token, str) and provided_token else None+++def get_azure_request_auth_headers(+    headers: Mapping[str, str],+    azure_client_params: Mapping[str, object],+) -> Mapping[str, str]:+    if any(name.lower() in _AZURE_AUTH_HEADER_NAMES for name in headers):+        return headers+    azure_ad_token: Final = _resolve_azure_ad_token(azure_client_params)+    if azure_ad_token is not None:+        return MappingProxyType({**headers, "Authorization": f"Bearer {azure_ad_token}"})+    api_key: Final = azure_client_params.get("api_key")+    if isinstance(api_key, str) and api_key:+        return MappingProxyType({**headers, "api-key": api_key})+    return headers+++def redact_azure_auth_headers(headers: Mapping[str, str]) -> Mapping[str, str]:+    return {  # mutable-ok: logging callbacks JSON-serialize this copy+        name: (_REDACTED_AZURE_HEADER_VALUE if name.lower() in _AZURE_AUTH_HEADER_NAMES else value)+        for name, value in headers.items()+    }++ class BaseAzureLLM(BaseOpenAILLM):     @staticmethod     def _try_get_default_azure_credential_provider(@@ -616,9 +656,7 @@ class BaseAzureLLM(BaseOpenAILLM):                 "Using Azure AD token provider based on Service Principal with Secret workflow for Azure Auth"             )             try:-                azure_ad_token_provider = get_azure_ad_token_provider(-                    azure_scope=scope,-                )+                azure_ad_token_provider = _cached_azure_ad_token_refresh_provider(scope)             except ValueError:                 verbose_logger.debug("Azure AD Token Provider could not be used.")         if api_version is None:tests/test_litellm/llms/azure/image_generation/test_azure_image_generation_init.pyindex 70b5eab5c37..cfde1760389 100644@@ -10,6 +10,12 @@ import respx import litellm from litellm.caching.llm_caching_handler import LLMClientCache from litellm.llms.azure.azure import AzureChatCompletion+from litellm.llms.azure.common_utils import (+    _cached_azure_ad_token_refresh_provider,+    _cached_entra_id_token_provider,+    get_azure_request_auth_headers,+    redact_azure_auth_headers,+) from litellm.llms.azure.image_generation.http_utils import (     azure_deployment_image_generation_json_body, )@@ -587,3 +593,293 @@ def test_azure_image_generation_v1_route_base_model_vs_deployment_name(respx_moc     sent_body = json.loads(request.content)     assert sent_body["model"] == model     assert sent_body["prompt"] == prompt+++@pytest.fixture+def fake_entra_id(monkeypatch: pytest.MonkeyPatch):+    built_credentials = []++    class FakeClientSecretCredential:+        def __init__(self, tenant_id: str, client_id: str, client_secret: str) -> None:+            built_credentials.append((tenant_id, client_id, client_secret))++    monkeypatch.setattr("azure.identity.ClientSecretCredential", FakeClientSecretCredential)+    monkeypatch.setattr("azure.identity.get_bearer_token_provider", lambda credential, scope: lambda: "entra-id-token")+    _cached_entra_id_token_provider.cache_clear()+    yield built_credentials+    _cached_entra_id_token_provider.cache_clear()+++def _mock_image_generation_route(respx_mock: respx.MockRouter, api_base: str, model: str) -> respx.Route:+    return respx_mock.post(f"{api_base}/openai/deployments/{model}/images/generations").mock(+        return_value=httpx.Response(200, json={"created": 1234567890, "data": [{"b64_json": "aaaa"}]})+    )+++@pytest.mark.parametrize("credentials_in_litellm_params", [False, True])+def test_azure_image_generation_keyless_entra_id_sends_bearer_token(+    respx_mock: respx.MockRouter,+    monkeypatch: pytest.MonkeyPatch,+    fake_entra_id: list,+    credentials_in_litellm_params: bool,+):+    for name in ("AZURE_TENANT_ID", "AZURE_CLIENT_ID", "AZURE_CLIENT_SECRET"):+        monkeypatch.delenv(name, raising=False)+    api_base = "https://my-resource.openai.azure.com"+    api_version = "2025-04-01-preview"+    litellm_params = {"api_base": api_base, "api_version": api_version}+    if credentials_in_litellm_params:+        litellm_params.update(+            tenant_id="tenant-from-params", client_id="client-from-params", client_secret="secret-from-params"+        )+        expected_credential = ("tenant-from-params", "client-from-params", "secret-from-params")+    else:+        monkeypatch.setenv("AZURE_TENANT_ID", "tenant-from-env")+        monkeypatch.setenv("AZURE_CLIENT_ID", "client-from-env")+        monkeypatch.setenv("AZURE_CLIENT_SECRET", "secret-from-env")+        expected_credential = ("tenant-from-env", "client-from-env", "secret-from-env")+    route = _mock_image_generation_route(respx_mock, api_base, "gpt-image-1")+    logging_obj = MagicMock()++    response = AzureChatCompletion().image_generation(+        prompt="a cat",+        timeout=60.0,+        optional_params={"n": 1, "size": "1024x1024"},+        logging_obj=logging_obj,+        headers={"Content-Type": "application/json"},+        model="gpt-image-1",+        api_key=None,+        api_base=api_base,+        api_version=api_version,+        litellm_params=litellm_params,+    )++    request = route.calls.last.request+    assert request.headers["Authorization"] == "Bearer entra-id-token"+    assert "api-key" not in request.headers+    assert fake_entra_id == [expected_credential]+    assert response.data[0].b64_json == "aaaa"+    logged_headers = logging_obj.pre_call.call_args.kwargs["additional_args"]["headers"]+    assert logged_headers == {"Content-Type": "application/json", "Authorization": "***REDACTED***"}+    assert "entra-id-token" not in str(logging_obj.pre_call.call_args)+++@pytest.mark.asyncio+async def test_azure_aimage_generation_keyless_entra_id_sends_bearer_token(+    respx_mock: respx.MockRouter, monkeypatch: pytest.MonkeyPatch, fake_entra_id: list+):+    monkeypatch.setattr(litellm, "disable_aiohttp_transport", True)+    monkeypatch.setattr(litellm, "in_memory_llm_clients_cache", LLMClientCache())+    api_base = "https://my-resource.openai.azure.com"+    api_version = "2025-04-01-preview"+    route = _mock_image_generation_route(respx_mock, api_base, "gpt-image-1")+    logging_obj = MagicMock()++    response = await AzureChatCompletion().image_generation(+        prompt="a cat",+        timeout=60.0,+        optional_params={"n": 1, "size": "1024x1024"},+        logging_obj=logging_obj,+        headers={"Content-Type": "application/json"},+        model="gpt-image-1",+        api_key=None,+        api_base=api_base,+        api_version=api_version,+        aimg_generation=True,+        litellm_params={+            "api_base": api_base,+            "api_version": api_version,+            "tenant_id": "tenant-from-params",+            "client_id": "client-from-params",+            "client_secret": "secret-from-params",+        },+    )++    request = route.calls.last.request+    assert request.headers["Authorization"] == "Bearer entra-id-token"+    assert "api-key" not in request.headers+    assert fake_entra_id == [("tenant-from-params", "client-from-params", "secret-from-params")]+    assert response.data[0].b64_json == "aaaa"+    logged_headers = logging_obj.pre_call.call_args.kwargs["additional_args"]["headers"]+    assert logged_headers == {"Content-Type": "application/json", "Authorization": "***REDACTED***"}+    assert "entra-id-token" not in str(logging_obj.pre_call.call_args)+++@pytest.mark.parametrize(+    "credential_kwargs, expected_authorization",+    [+        ({"azure_ad_token": "static-ad-token"}, "Bearer static-ad-token"),+        ({"azure_ad_token_provider": lambda: "provider-token"}, "Bearer provider-token"),+    ],+)+def test_azure_image_generation_explicit_azure_ad_credential_sends_bearer_token(+    respx_mock: respx.MockRouter,+    monkeypatch: pytest.MonkeyPatch,+    credential_kwargs: dict,+    expected_authorization: str,+):+    for name in ("AZURE_TENANT_ID", "AZURE_CLIENT_ID", "AZURE_CLIENT_SECRET"):+        monkeypatch.delenv(name, raising=False)+    api_base = "https://my-resource.openai.azure.com"+    api_version = "2025-04-01-preview"+    route = _mock_image_generation_route(respx_mock, api_base, "gpt-image-1")++    response = AzureChatCompletion().image_generation(+        prompt="a cat",+        timeout=60.0,+        optional_params={"n": 1, "size": "1024x1024"},+        logging_obj=MagicMock(),+        headers={"Content-Type": "application/json"},+        model="gpt-image-1",+        api_key=None,+        api_base=api_base,+        api_version=api_version,+        litellm_params={"api_base": api_base, "api_version": api_version},+        **credential_kwargs,+    )++    request = route.calls.last.request+    assert request.headers["Authorization"] == expected_authorization+    assert "api-key" not in request.headers+    assert response.data[0].b64_json == "aaaa"+++def test_azure_image_generation_with_api_key_keeps_api_key_header(+    respx_mock: respx.MockRouter, monkeypatch: pytest.MonkeyPatch, fake_entra_id: list+):+    monkeypatch.setenv("AZURE_TENANT_ID", "tenant-from-env")+    monkeypatch.setenv("AZURE_CLIENT_ID", "client-from-env")+    monkeypatch.setenv("AZURE_CLIENT_SECRET", "secret-from-env")+    api_base = "https://my-resource.openai.azure.com"+    api_version = "2025-04-01-preview"+    route = _mock_image_generation_route(respx_mock, api_base, "gpt-image-1")+    logging_obj = MagicMock()++    response = AzureChatCompletion().image_generation(+        prompt="a cat",+        timeout=60.0,+        optional_params={"n": 1, "size": "1024x1024"},+        logging_obj=logging_obj,+        headers={"Content-Type": "application/json", "api-key": "sk-test"},+        model="gpt-image-1",+        api_key="sk-test",+        api_base=api_base,+        api_version=api_version,+        litellm_params={"api_base": api_base, "api_version": api_version},+    )++    request = route.calls.last.request+    assert request.headers["api-key"] == "sk-test"+    assert "Authorization" not in request.headers+    assert fake_entra_id == []+    assert response.data[0].b64_json == "aaaa"+    assert logging_obj.pre_call.call_args.kwargs["additional_args"]["headers"]["api-key"] == "***REDACTED***"+++@pytest.fixture+def fake_default_azure_credential(monkeypatch: pytest.MonkeyPatch):+    built_credentials = []++    class FakeDefaultAzureCredential:+        def __init__(self) -> None:+            built_credentials.append(self)++    for name in ("AZURE_TENANT_ID", "AZURE_CLIENT_ID", "AZURE_CLIENT_SECRET", "AZURE_CREDENTIAL", "AZURE_AD_TOKEN"):+        monkeypatch.delenv(name, raising=False)+    monkeypatch.setattr("azure.identity.DefaultAzureCredential", FakeDefaultAzureCredential)+    monkeypatch.setattr(+        "azure.identity.get_bearer_token_provider", lambda credential, scope: lambda: "default-credential-token"+    )+    monkeypatch.setattr(litellm, "enable_azure_ad_token_refresh", True)+    _cached_azure_ad_token_refresh_provider.cache_clear()+    yield built_credentials+    _cached_azure_ad_token_refresh_provider.cache_clear()+++def test_azure_image_generation_token_refresh_reuses_credential_across_requests(+    respx_mock: respx.MockRouter, fake_default_azure_credential: list+):+    api_base = "https://my-resource.openai.azure.com"+    api_version = "2025-04-01-preview"+    route = _mock_image_generation_route(respx_mock, api_base, "gpt-image-1")++    for _ in range(3):+        AzureChatCompletion().image_generation(+            prompt="a cat",+            timeout=60.0,+            optional_params={"n": 1, "size": "1024x1024"},+            logging_obj=MagicMock(),+            headers={"Content-Type": "application/json"},+            model="gpt-image-1",+            api_key=None,+            api_base=api_base,+            api_version=api_version,+            litellm_params={"api_base": api_base, "api_version": api_version},+        )++    assert route.call_count == 3+    assert all(call.request.headers["Authorization"] == "Bearer default-credential-token" for call in route.calls)+    assert len(fake_default_azure_credential) == 1+++@pytest.mark.parametrize(+    "caller_auth_header",+    [{"api-key": "caller-key"}, {"Authorization": "Bearer caller-token"}, {"authorization": "Bearer caller-token"}],+)+def test_get_azure_request_auth_headers_keeps_caller_auth_header(caller_auth_header: dict):+    headers = {"Content-Type": "application/json", **caller_auth_header}+    azure_client_params = {+        "api_key": "sk-resolved",+        "azure_ad_token": "resolved-token",+        "azure_ad_token_provider": lambda: "provider-token",+    }+    assert get_azure_request_auth_headers(headers=headers, azure_client_params=azure_client_params) is headers+++def test_get_azure_request_auth_headers_prefers_azure_ad_token_over_provider_and_api_key():+    headers = {"Content-Type": "application/json"}+    azure_client_params = {+        "api_key": "sk-resolved",+        "azure_ad_token": "static-token",+        "azure_ad_token_provider": lambda: "provider-token",+    }+    out = get_azure_request_auth_headers(headers=headers, azure_client_params=azure_client_params)+    assert dict(out) == {"Content-Type": "application/json", "Authorization": "Bearer static-token"}+    assert headers == {"Content-Type": "application/json"}+++def test_get_azure_request_auth_headers_uses_token_provider_over_api_key():+    azure_client_params = {"api_key": "sk-resolved", "azure_ad_token": None, "azure_ad_token_provider": lambda: "pt"}+    out = get_azure_request_auth_headers(headers={}, azure_client_params=azure_client_params)+    assert dict(out) == {"Authorization": "Bearer pt"}+++def test_get_azure_request_auth_headers_falls_back_to_api_key():+    azure_client_params = {"api_key": "sk-resolved", "azure_ad_token": None, "azure_ad_token_provider": None}+    out = get_azure_request_auth_headers(headers={"Content-Type": "application/json"}, azure_client_params=azure_client_params)+    assert dict(out) == {"Content-Type": "application/json", "api-key": "sk-resolved"}+++@pytest.mark.parametrize(+    "azure_client_params",+    [+        {},+        {"api_key": "", "azure_ad_token": "", "azure_ad_token_provider": None},+        {"azure_ad_token_provider": lambda: None},+        {"azure_ad_token_provider": lambda: ""},+    ],+)+def test_get_azure_request_auth_headers_without_credential_leaves_headers_unchanged(azure_client_params: dict):+    headers = {"Content-Type": "application/json"}+    assert get_azure_request_auth_headers(headers=headers, azure_client_params=azure_client_params) is headers+++def test_redact_azure_auth_headers_masks_only_credential_values():+    headers = {"Content-Type": "application/json", "api-key": "sk-secret", "authorization": "Bearer secret"}+    assert redact_azure_auth_headers(headers) == {+        "Content-Type": "application/json",+        "api-key": "***REDACTED***",+        "authorization": "***REDACTED***",+    }+    assert headers["api-key"] == "sk-secret"+    assert headers["authorization"] == "Bearer secret"tests/test_litellm/llms/azure/test_azure_common_utils.pyindex c959c201ccb..83ec85f1176 100644@@ -9,6 +9,7 @@ import pytest import litellm from litellm.llms.azure.common_utils import (     BaseAzureLLM,+    _cached_azure_ad_token_refresh_provider,     _cached_entra_id_token_provider,     get_azure_ad_token,     get_azure_ad_token_from_entra_id,@@ -34,6 +35,7 @@ def setup_mocks(monkeypatch):     monkeypatch.delenv("AZURE_TENANT_ID", raising=False)     monkeypatch.delenv("AZURE_SCOPE", raising=False)     monkeypatch.delenv("AZURE_AD_TOKEN", raising=False)+    _cached_azure_ad_token_refresh_provider.cache_clear()      with (         patch(@@ -78,6 +80,7 @@ def setup_mocks(monkeypatch):             "logger": mock_logger,             "select_url": mock_select_url,         }+    _cached_azure_ad_token_refresh_provider.cache_clear()   def test_initialize_with_api_key(setup_mocks):