Backtestslitellmrun-20261111-220245#42959

Issue #42959 Merged fix PR #43415 Run 11 Nov 2026 · run-20261111-220245 BerriAI/litellm

[Bug]: per-turn-control-2026-07-01 filtered for azure_ai, but Azure AI Foundry supports it — Claude Code fails with "messages.1.output_config: Extra inputs are not permitted"

Better

Our patch was judged better than the merged fix.

In this judgment, Patch A = our agent, Patch B = the merged fix.

Both patches make the identical one-line config change, flipping per-turn-control-2026-07-01 from null to forwarded for azure_ai, and both remove azure_ai from the 'dropped' parametrization. Correctness and scope are therefore equal; the difference is entirely in test coverage. Patch B adds a minimal direct check that update_headers_with_filtered_beta keeps the beta for provider='azure_ai'. It would fail before and pass after, but it only exercises the lookup table. Patch A adds a test that goes through AzureAnthropicMessagesConfig end to end: validate_anthropic_messages_environment, the provider filter, and transform_anthropic_messages_request, with a Claude Code-shaped request body (output_config on a mid-conversation system message). It is parametrized over both the case where the client sends the beta and the case where LiteLLM must add it from the per-message output_config, and it asserts both that the final header set is exactly {per-turn-control, effort} and that the system message's output_config survives in the outgoing body. That directly mirrors reproduction step B in the issue and would catch regressions in the add-beta path or the body transformation that Patch B's test cannot see. A's test relies on an existing _claude_code_turn helper and the Azure config's signatures, which I cannot verify here, so there is some fragility risk, but given the surrounding file already tests output_config-driven beta injection for other providers the helper is very likely present. On the tests criterion A is meaningfully stronger, with no downside on correctness, scope, or craft.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

16m 17s

Wall-clock time in the container

Full test suite

No new failures

481 passed · 0 skipped · 481 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

43vs 10 merged

Changed lines, ours vs merged

Description

What happened?

Claude Code (v2.1.281) sets per-turn effort by putting output_config on a mid-conversation
role: system message, which requires the per-turn-control-2026-07-01 beta.

#41189 added this beta to the allowlist for anthropic only. For azure_ai it is set to null
in anthropic_beta_headers_config.json (the PR caveat notes azure_ai drops it on purpose).
As a result, every Claude Code turn with effort control fails through the proxy on Azure AI Foundry:

litellm.BadRequestError: Azure_aiException - {"type":"error","error":{"type":"invalid_request_error",
"message":"messages.1.output_config: Extra inputs are not permitted"}}

This breaks effort levels (xhigh / ultracode) for Claude Opus 5.5 on Azure AI Foundry.

Azure AI Foundry does support this beta

Direct call to the Foundry Anthropic endpoint, bypassing LiteLLM, same body
(output_config on a role: system message):

  • WITHOUT anthropic-beta: per-turn-control-2026-07-01 → 400 messages.1.output_config: Extra inputs are not permitted
  • WITH anthropic-beta: per-turn-control-2026-07-01,effort-2025-11-24 → 200, normal assistant message

curl https://<resource>.services.ai.azure.com/anthropic/v1/messages \
-H "x-api-key: ***" -H "anthropic-version: 2023-06-01" -H "Content-Type: application/json" \
-H "anthropic-beta: per-turn-control-2026-07-01,effort-2025-11-24" \
-d '{"model":"claude-opus-5-5","max_tokens":50,"output_config":{"effort":"xhigh"},
"messages":[{"role":"user","content":[{"type":"text","text":"Reply OK"}]},
{"role":"system","content":[{"type":"text","text":"# Environment"}],"output_config":{"effort":"low"}}]}'

Proposed fix

In litellm/anthropic_beta_headers_config.json, section azure_ai:

"per-turn-control-2026-07-01": "per-turn-control-2026-07-01"

and update the provider-filtering test added in #41189 accordingly.

Environment

  • LiteLLM proxy 1.101.0 (Docker), beta headers config fetched from main (no LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS)
  • Model: azure_ai/claude-opus-5-5, api_base on *.services.ai.azure.com
  • Client: Claude Code 2.1.281 (/v1/messages?beta=true)

Workaround

Local copy of the config with the value above, LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS=True.

Config

# Models are managed in the DB via the Admin UI (STORE_MODEL_IN_DB=True).
# Equivalent definition of the affected model:
model_list:
  - model_name: claude-opus-5-5
    litellm_params:
      model: azure_ai/claude-opus-5-5
      api_base: https://<resource>.services.ai.azure.com
      api_key: os.environ/AZURE_AI_API_KEY
      custom_llm_provider: azure_ai

litellm_settings:
  drop_params: true
  success_callback: ["prometheus"]
  failure_callback: ["prometheus"]

router_settings:
  routing_strategy: "least-busy"
  num_retries: 3

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL

LiteLLM Version

v1.101.0

Steps to Repro

Steps to Reproduce

A. With Claude Code

  1. Start LiteLLM proxy 1.101.0 with the config above (default beta headers config, fetched from main).
  2. Point Claude Code 2.1.281 at the proxy: ANTHROPIC_BASE_URL=https://<litellm-proxy> ANTHROPIC_AUTH_TOKEN=sk-... claude --model claude-opus-5-5
  3. Set an effort level: /effort xhigh (or ultracode).
  4. Send any prompt, e.g. hello.
  5. Result: API Error: 400 — the proxy log shows Azure_aiException - messages.1.output_config: Extra inputs are not permitted. With CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1 the same prompt succeeds (effort control disabled).

B. With curl, through the proxy (Claude Code-shaped request)

curl -s https://<litellm-proxy>/v1/messages \
-H "Authorization: Bearer sk-..." -H "anthropic-version: 2023-06-01" -H "Content-Type: application/json" \
-H "anthropic-beta: per-turn-control-2026-07-01,effort-2025-11-24" \
-d '{"model":"claude-opus-5-5","max_tokens":50,"output_config":{"effort":"xhigh"},
"messages":[{"role":"user","content":[{"type":"text","text":"Reply OK"}]},
{"role":"system","content":[{"type":"text","text":"# Environment"}],"output_config":{"effort":"low"}}]}'

→ 400 messages.1.output_config: Extra inputs are not permitted
(the proxy drops per-turn-control-2026-07-01 because it is null for azure_ai).

C. Same body, directly to Azure AI Foundry (bypassing LiteLLM)

Same curl against https://<resource>.services.ai.azure.com/anthropic/v1/messages
with x-api-key instead of Authorization:

  • with anthropic-beta: per-turn-control-2026-07-01,effort-2025-11-24 → 200
  • without that header → 400 (same error as B)

Expected

B should return 200, like C with the beta header: LiteLLM should forward
per-turn-control-2026-07-01 for azure_ai.

Actual

LiteLLM filters the beta for azure_ai, so every Claude Code turn with effort control fails.

Which part of LiteLLM is this about?

_No response_

How are you deploying?

_No response_

The two patches

Our patch+41 −2
litellm/anthropic_beta_headers_config.jsonindex 3a28d65e47..71e7081b44 100644@@ -57,7 +57,7 @@     "mcp-servers-2025-12-04": null,     "output-128k-2025-02-19": null,     "structured-output-2024-03-01": null,-    "per-turn-control-2026-07-01": null,+    "per-turn-control-2026-07-01": "per-turn-control-2026-07-01",     "prompt-caching-scope-2026-01-05": "prompt-caching-scope-2026-01-05",     "skills-2025-10-02": "skills-2025-10-02",     "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",tests/unit/llms/anthropic/pass_through/messages/test_anthropic_messages_per_turn_control.pyindex 557305a945..a0f2663f37 100644@@ -5,10 +5,14 @@ from litellm.anthropic_beta_headers_manager import update_headers_with_filtered_ from litellm.llms.anthropic.pass_through.messages.transformation import (     AnthropicMessagesConfig, )+from litellm.llms.azure_ai.anthropic.messages_transformation import (+    AzureAnthropicMessagesConfig,+) from litellm.llms.openai_like.json_loader import SimpleProviderConfig from litellm.llms.openai_like.messages.transformation import (     JSONProviderAnthropicMessagesConfig, )+from litellm.types.router import GenericLiteLLMParams  PER_TURN_CONTROL = "per-turn-control-2026-07-01" @@ -95,13 +99,48 @@ def test_added_per_turn_control_beta_survives_the_anthropic_allowlist():     assert PER_TURN_CONTROL in _betas(filtered)  -@pytest.mark.parametrize("provider", ["bedrock", "bedrock_converse", "vertex_ai", "azure_ai", "databricks"])+@pytest.mark.parametrize("provider", ["bedrock", "bedrock_converse", "vertex_ai", "databricks"]) def test_per_turn_control_beta_is_dropped_for_providers_without_it(provider):     filtered = update_headers_with_filtered_beta(headers={"anthropic-beta": PER_TURN_CONTROL}, provider=provider)      assert "anthropic-beta" not in filtered  +@pytest.mark.parametrize(+    "client_betas",+    [f"{PER_TURN_CONTROL},effort-2025-11-24", "effort-2025-11-24"],+    ids=["client_sent", "added_from_output_config"],+)+def test_azure_ai_forwards_per_turn_control_beta_with_the_per_message_output_config(client_betas):+    config = AzureAnthropicMessagesConfig()+    messages = _claude_code_turn({"effort": "low"})+    headers, _ = config.validate_anthropic_messages_environment(+        headers={"anthropic-beta": client_betas},+        model="claude-opus-5-5",+        messages=messages,+        optional_params={"max_tokens": 50, "output_config": {"effort": "xhigh"}},+        litellm_params={},+        api_key="azure-test-key",+        api_base="https://example.services.ai.azure.com",+    )+    assert config.should_filter_anthropic_beta_headers()+    filtered = update_headers_with_filtered_beta(headers=headers, provider="azure_ai")+    body = config.transform_anthropic_messages_request(+        model="claude-opus-5-5",+        messages=messages,+        anthropic_messages_optional_request_params={"max_tokens": 50, "output_config": {"effort": "xhigh"}},+        litellm_params=GenericLiteLLMParams(),+        headers=filtered,+    )++    assert _betas(filtered) == {PER_TURN_CONTROL, "effort-2025-11-24"}, filtered+    assert body["messages"][1] == {+        "role": "system",+        "content": [{"type": "text", "text": "# Environment"}],+        "output_config": {"effort": "low"},+    }, body++ def test_json_provider_passthrough_adds_per_turn_control_beta():     config = JSONProviderAnthropicMessagesConfig(         SimpleProviderConfig( 
Merged fix · PR #43415+8 −2
litellm/anthropic_beta_headers_config.jsonindex 3a28d65e47c..71e7081b440 100644@@ -57,7 +57,7 @@     "mcp-servers-2025-12-04": null,     "output-128k-2025-02-19": null,     "structured-output-2024-03-01": null,-    "per-turn-control-2026-07-01": null,+    "per-turn-control-2026-07-01": "per-turn-control-2026-07-01",     "prompt-caching-scope-2026-01-05": "prompt-caching-scope-2026-01-05",     "skills-2025-10-02": "skills-2025-10-02",     "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",tests/unit/llms/anthropic/pass_through/messages/test_anthropic_messages_per_turn_control.pyindex 557305a945c..4197192e4af 100644@@ -95,13 +95,19 @@ def test_added_per_turn_control_beta_survives_the_anthropic_allowlist():     assert PER_TURN_CONTROL in _betas(filtered)  -@pytest.mark.parametrize("provider", ["bedrock", "bedrock_converse", "vertex_ai", "azure_ai", "databricks"])+@pytest.mark.parametrize("provider", ["bedrock", "bedrock_converse", "vertex_ai", "databricks"]) def test_per_turn_control_beta_is_dropped_for_providers_without_it(provider):     filtered = update_headers_with_filtered_beta(headers={"anthropic-beta": PER_TURN_CONTROL}, provider=provider)      assert "anthropic-beta" not in filtered  +def test_per_turn_control_beta_is_forwarded_for_azure_ai():+    filtered = update_headers_with_filtered_beta(headers={"anthropic-beta": PER_TURN_CONTROL}, provider="azure_ai")++    assert _betas(filtered) == {PER_TURN_CONTROL}++ def test_json_provider_passthrough_adds_per_turn_control_beta():     config = JSONProviderAnthropicMessagesConfig(         SimpleProviderConfig(