Backtestslitellmrun-20261019-033420#41202

Issue #41202 Merged fix PR #41203 Run 19 Oct 2026 · run-20261019-033420 BerriAI/litellm

[Bug]: thinking-binding-controls-2026-08-01 beta header is stripped on /v1/messages, so thinking.block_binding 400s on Bedrock/Vertex

Worse

The merged fix was judged better than our patch.

In this judgment, Patch A = our agent, Patch B = the merged fix.

Both patches fix the root cause the same way: register `thinking-binding-controls-2026-08-01` in `anthropic_beta_headers_config.json` so `filter_and_transform_beta_headers` stops dropping it. The difference is which provider sections get a forwarded value versus `null`.

The six sections in the diff follow the file's existing layout. The three larger sections that carry a `tool-search-tool-2025-10-19` key (hunks at lines 90, 122, 154) are the Claude-on-cloud providers (bedrock, bedrock_converse, vertex_ai in file order); the smaller ones are anthropic and the thin pass-through providers. Patch A forwards the header only in sections 1, 4 and 5 and sets section 3 (the first bedrock section, which is the invoke path used by `bedrock/us.anthropic.claude-fable-5-1` on /v1/messages) to `null`. That leaves the exact Bedrock scenario in the issue still stripping the header, and it is internally inconsistent with A's own bedrock-invoke test, which asserts the beta appears in the outgoing `anthropic_beta` body list. Patch B forwards the header in every Claude-serving section (sections 1, 3, 4, 5, 6) and only nulls the one non-Claude pass-through section, which is the correct coverage for a header whose only purpose is to accompany a body field the proxy already forwards untouched.

Tests: B's parametrized test calls `filter_and_transform_beta_headers` directly for anthropic, bedrock, bedrock_converse, vertex_ai and databricks and asserts the header survives. It fails before the change (unknown headers are dropped) and passes after, and it pins the behaviour for both bedrock keys, which is where A is weakest. A's tests are heavier integration tests with mocked Vertex auth and environment fiddling; they cover anthropic, vertex_ai and bedrock-invoke but not bedrock_converse, and as noted the bedrock-invoke assertion looks at odds with A's own config.

Scope and craft: A also adds the enum member in `ANTHROPIC_BETA_HEADER_VALUES`, which is harmless but unnecessary since the filter reads the JSON. B is smaller, follows the existing test file's conventions, and the config change is self-explanatory. Given the config-only nature of the fix and lack of runnable independent tests, B's more complete provider coverage and more direct regression test make it the better fix.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

44m 5s

Wall-clock time in the container

Full test suite

No new failures

1,234 passed · 0 skipped · 1,234 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

117vs 22 merged

Changed lines, ours vs merged

Check for existing issues

  • I have searched the existing issues and this is not a duplicate (no hits for thinking-binding-controls or block_binding).

What happened?

Anthropic's preserved-thinking controls for Claude Fable 5.1 (thinking.block_binding.prefix_mismatch_behavior, https://platform.claude.com/docs/en/build-with-claude/preserved-thinking) are only accepted alongside the beta header thinking-binding-controls-2026-08-01.

When a client sends both through the proxy to a bedrock/ or vertex_ai/ Claude deployment on /v1/messages, the proxy forwards the block_binding body field untouched but strips the beta header, because the header has no entry in anthropic_beta_headers_config.json (bundled at v1.101.0-rc.1, and not on main as of 03a4c806c4 either). The provider then rejects the request with:

thinking.adaptive.block_binding: Extra inputs are not permitted

Consequences:

  • Nobody behind a LiteLLM gateway can opt into preserved-thinking enforcement on Bedrock/Vertex, which Anthropic's docs tell integrators to do now to test their append-only behaviour before later models enforce it for everyone.
  • Accounts created on or after 2026-08-31 are enforced by default on Fable 5.1; the SDKs send block_binding for them, so this is a hard 400 for those users rather than an opt-in inconvenience.

The same request sent directly to the provider (header + block_binding) succeeds and returns input_transformations: [].

User Flow

Before a (hypothetical) fix: a developer testing their integration against preserved thinking gets a 400 from every Claude Fable 5.1 deployment behind the gateway.

  1. They send POST https://litellm-domain/v1/messages with header anthropic-beta: thinking-binding-controls-2026-08-01 and body "model": "claude-fable-5-1", "thinking": {"type": "adaptive", "block_binding": {"prefix_mismatch_behavior": "drop_block"}}, "messages": [{"role": "user", "content": "..."}]
  2. They get HTTP 400 invalid_request_error: Vertex_aiException BadRequestError - thinking.adaptive.block_binding: Extra inputs are not permitted, followed by the configured Bedrock fallback failing the same way (BedrockException - {"message":"thinking.adaptive.block_binding: Extra inputs are not permitted"})
  3. They remove the block_binding field: the request succeeds, but now they cannot see input_transformations or make the API enforce the prefix check, so they cannot tell whether their integration is affected

After a (hypothetical) fix: the same request goes through and the API reports what it dropped.

  1. They send the same POST https://litellm-domain/v1/messages with the same header and body
  2. They get HTTP 200 with the usual content array plus a top-level input_transformations: []
  3. On a later turn where they edited an earlier message, input_transformations lists the dropped thinking blocks with reason: "prefix_binding_mismatch" (or, with "error", the 400 Anthropic documents naming what changed)

Proof the bug occurs

Config / setup the proxy ran with (our staging gateway, litellm/litellm-database:v1.101.0-rc.1, Vertex primary with Bedrock fallback; irrelevant deployments omitted):

model_list:
  - model_name: claude-fable-5-1
    litellm_params:
      model: vertex_ai/claude-fable-5-1
      additional_drop_params: ["temperature"]
  - model_name: bedrock-claude-fable-5-1
    litellm_params:
      model: bedrock/us.anthropic.claude-fable-5-1
      aws_region_name: us-east-1
      thinking: {"type": "adaptive"}
litellm_settings:
  num_retries: 3
  drop_params: true
  fallbacks: [{"claude-fable-5-1": ["bedrock-claude-fable-5-1"]}]
  vertex_location: "global"

Env: GOOGLE_APPLICATION_CREDENTIALS for Vertex, IRSA for Bedrock, no LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS.

Version: v1.101.0-rc.1 (image litellm/litellm-database:v1.101.0-rc.1).

Command (a 3-turn probe; turn 1 is enough to show it):

curl -sS https://<gateway>/v1/messages \
  -H "Authorization: Bearer <virtual key>" \
  -H "anthropic-version: 2023-06-01" \
  -H "anthropic-beta: thinking-binding-controls-2026-08-01" \
  -H "content-type: application/json" \
  -d '{"model":"claude-fable-5-1","max_tokens":4000,"output_config":{"effort":"high"},
       "thinking":{"type":"adaptive","block_binding":{"prefix_mismatch_behavior":"drop_block"}},
       "messages":[{"role":"user","content":"Think carefully before answering. How many positive integers below 5000 have exactly 12 positive divisors? Reply with the number only."}]}'

Output (HTTP 400, x-litellm-call-id: eb073562-5aa2-40f5-92f5-612477a7619c):

{"type":"error","error":{"type":"invalid_request_error","message":"litellm.BadRequestError: Vertex_aiException BadRequestError - {\"type\":\"error\",\"error\":{\"type\":\"invalid_request_error\",\"message\":\"thinking.adaptive.block_binding: Extra inputs are not permitted\"},\"request_id\":\"req_vrtx_011Cf4R9GhDLYQ5oqHS46VnE\"}. Received Model Group=claude-fable-5-1\nAvailable Model Group Fallbacks=['bedrock-claude-fable-5-1']\nError doing the fallback: litellm.BadRequestError: BedrockException - {\"message\":\"thinking.adaptive.block_binding: Extra inputs are not permitted\"} ..."}}

Same request against bedrock-claude-fable-5-1 directly (x-litellm-call-id: e0ddc96d-a555-4c57-84db-9266ab6adf15): HTTP 400, BedrockException - {"message":"thinking.adaptive.block_binding: Extra inputs are not permitted"}.

Same request body without the beta header and without block_binding: HTTP 200 on all three turns, each response carries a thinking block with a signature and no input_transformations field (expected for an account created before 2026-08-31 with no opt-in).

After registering the header (branch fix/anthropic-thinking-binding-beta-header), the identical request to the same two deployments returns HTTP 200 and the response carries input_transformations: [] — the field only exists when the header reached the provider — on both bedrock/us.anthropic.claude-fable-5-1 and vertex_ai/claude-fable-5-1.

Where the header goes: filter_and_transform_beta_headers drops any value not present in the provider's mapping in anthropic_beta_headers_config.json; thinking-binding-controls-2026-08-01 has no entry for any provider. Confirmed on a local litellm --config proxy against a recording upstream: the outgoing Bedrock invoke body has block_binding but no anthropic_beta, and the outgoing first-party request has no anthropic-beta header.

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.101.0-rc.1

Twitter / LinkedIn details

_No response_

The two patches

Our patch+117 −0
litellm/anthropic_beta_headers_config.jsonindex 8dc9204af8..92eb7a721e 100644@@ -28,6 +28,7 @@     "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01",     "token-efficient-tools-2025-02-19": "token-efficient-tools-2025-02-19",     "web-fetch-2025-09-10": "web-fetch-2025-09-10",     "web-search-2025-03-05": "web-search-2025-03-05"@@ -59,6 +60,7 @@     "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": null,     "token-efficient-tools-2025-02-19": null,     "web-fetch-2025-09-10": "web-fetch-2025-09-10",     "web-search-2025-03-05": "web-search-2025-03-05"@@ -90,6 +92,7 @@     "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": null,     "token-efficient-tools-2025-02-19": null,     "tool-search-tool-2025-10-19": null,     "web-fetch-2025-09-10": null,@@ -122,6 +125,7 @@     "structured-outputs-2025-11-13": null,     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01",     "token-efficient-tools-2025-02-19": null,     "tool-search-tool-2025-10-19": "tool-search-tool-2025-10-19",     "web-fetch-2025-09-10": null,@@ -154,6 +158,7 @@     "structured-outputs-2025-11-13": null,     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01",     "token-efficient-tools-2025-02-19": null,     "tool-search-tool-2025-10-19": "tool-search-tool-2025-10-19",     "web-fetch-2025-09-10": null,@@ -187,6 +192,7 @@     "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": null,     "token-efficient-tools-2025-02-19": "token-efficient-tools-2025-02-19",     "web-fetch-2025-09-10": "web-fetch-2025-09-10",     "web-search-2025-03-05": "web-search-2025-03-05"litellm/types/llms/anthropic.pyindex bcdee86360..9a02f24a75 100644@@ -747,6 +747,7 @@ class ANTHROPIC_BETA_HEADER_VALUES(str, Enum):     FAST_MODE_2026_02_01 = "fast-mode-2026-02-01"     ADVISOR_TOOL_2026_03_01 = "advisor-tool-2026-03-01"     PER_TURN_CONTROL_2026_07_01 = "per-turn-control-2026-07-01"+    THINKING_BINDING_CONTROLS_2026_08_01 = "thinking-binding-controls-2026-08-01"   # Tool search beta header constant (for Anthropic direct API and Microsoft Foundry)tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_anthropic_messages_thinking_binding_controls.pynew file mode 100644index 0000000000..9d0557449d@@ -0,0 +1,110 @@+"""+`thinking.block_binding` (preserved thinking) is only accepted alongside the+`thinking-binding-controls-2026-08-01` beta. The /v1/messages path must forward+that beta to every provider that serves the feature, otherwise the provider+400s with `thinking.adaptive.block_binding: Extra inputs are not permitted`.+"""++from unittest.mock import patch++import pytest++from litellm import anthropic_beta_headers_manager+from litellm.anthropic_beta_headers_manager import update_headers_with_filtered_beta+from litellm.llms.anthropic.experimental_pass_through.messages.transformation import (+    AnthropicMessagesConfig,+)+from litellm.llms.bedrock.messages.invoke_transformations.anthropic_claude3_transformation import (+    AmazonAnthropicClaudeMessagesConfig,+)+from litellm.llms.vertex_ai.vertex_ai_partner_models.anthropic.experimental_pass_through.transformation import (+    VertexAIPartnerModelsAnthropicMessagesConfig,+)+from litellm.types.llms.anthropic import ANTHROPIC_BETA_HEADER_VALUES+from litellm.types.router import GenericLiteLLMParams++THINKING_BINDING_CONTROLS = ANTHROPIC_BETA_HEADER_VALUES.THINKING_BINDING_CONTROLS_2026_08_01.value++THINKING = {+    "type": "adaptive",+    "block_binding": {"prefix_mismatch_behavior": "drop_block"},+}++MESSAGES = [{"role": "user", "content": [{"type": "text", "text": "Hello"}]}]+++def _betas(headers):+    return {beta.strip() for beta in headers.get("anthropic-beta", "").split(",") if beta.strip()}+++@pytest.fixture(autouse=True)+def bundled_beta_allowlist(monkeypatch):+    monkeypatch.setenv("LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS", "True")+    monkeypatch.setattr(anthropic_beta_headers_manager, "_BETA_HEADERS_CONFIG", None)+    yield+    monkeypatch.setattr(anthropic_beta_headers_manager, "_BETA_HEADERS_CONFIG", None)+++def test_anthropic_messages_forwards_thinking_binding_controls_beta():+    config = AnthropicMessagesConfig()+    headers, _ = config.validate_anthropic_messages_environment(+        headers={"anthropic-beta": THINKING_BINDING_CONTROLS},+        model="claude-fable-5-1",+        messages=MESSAGES,+        optional_params={"max_tokens": 4000, "thinking": THINKING},+        litellm_params={},+        api_key="sk-ant-test",+    )+    assert config.should_filter_anthropic_beta_headers()+    headers = update_headers_with_filtered_beta(headers=headers, provider="anthropic")++    assert THINKING_BINDING_CONTROLS in _betas(headers)+++def test_vertex_ai_messages_forwards_thinking_binding_controls_beta_and_block_binding():+    config = VertexAIPartnerModelsAnthropicMessagesConfig()+    optional_params = {"max_tokens": 4000, "thinking": THINKING}+    with (+        patch.object(config, "_ensure_access_token", return_value=("token", "test-project")),+        patch.object(config, "get_complete_vertex_url", return_value="https://mock-url"),+    ):+        headers, _ = config.validate_anthropic_messages_environment(+            headers={"anthropic-beta": THINKING_BINDING_CONTROLS},+            model="claude-fable-5-1",+            messages=MESSAGES,+            optional_params=optional_params,+            litellm_params={+                "vertex_ai_project": "test-project",+                "vertex_ai_location": "global",+                "vertex_credentials": "{}",+            },+            api_base=None,+        )+    assert config.should_filter_anthropic_beta_headers()+    headers = update_headers_with_filtered_beta(headers=headers, provider="vertex_ai")++    assert THINKING_BINDING_CONTROLS in _betas(headers)++    body = config.transform_anthropic_messages_request(+        model="claude-fable-5-1",+        messages=MESSAGES,+        anthropic_messages_optional_request_params=dict(optional_params),+        litellm_params=GenericLiteLLMParams(),+        headers=headers,+    )+    assert body["thinking"] == THINKING+++def test_bedrock_invoke_messages_forwards_thinking_binding_controls_beta_and_block_binding():+    config = AmazonAnthropicClaudeMessagesConfig()++    body = config.transform_anthropic_messages_request(+        model="us.anthropic.claude-fable-5-1",+        messages=MESSAGES,+        anthropic_messages_optional_request_params={"max_tokens": 4000, "thinking": THINKING},+        litellm_params=GenericLiteLLMParams(),+        headers={"anthropic-beta": THINKING_BINDING_CONTROLS},+    )++    assert THINKING_BINDING_CONTROLS in (body.get("anthropic_beta") or [])+    assert body["thinking"] == THINKING 
Merged fix · PR #41203+22 −0
litellm/anthropic_beta_headers_config.jsonindex 8dc9204af8d..eb31cc17a15 100644@@ -28,6 +28,7 @@     "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01",     "token-efficient-tools-2025-02-19": "token-efficient-tools-2025-02-19",     "web-fetch-2025-09-10": "web-fetch-2025-09-10",     "web-search-2025-03-05": "web-search-2025-03-05"@@ -59,6 +60,7 @@     "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": null,     "token-efficient-tools-2025-02-19": null,     "web-fetch-2025-09-10": "web-fetch-2025-09-10",     "web-search-2025-03-05": "web-search-2025-03-05"@@ -90,6 +92,7 @@     "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01",     "token-efficient-tools-2025-02-19": null,     "tool-search-tool-2025-10-19": null,     "web-fetch-2025-09-10": null,@@ -122,6 +125,7 @@     "structured-outputs-2025-11-13": null,     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01",     "token-efficient-tools-2025-02-19": null,     "tool-search-tool-2025-10-19": "tool-search-tool-2025-10-19",     "web-fetch-2025-09-10": null,@@ -154,6 +158,7 @@     "structured-outputs-2025-11-13": null,     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01",     "token-efficient-tools-2025-02-19": null,     "tool-search-tool-2025-10-19": "tool-search-tool-2025-10-19",     "web-fetch-2025-09-10": null,@@ -187,6 +192,7 @@     "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01",     "token-efficient-tools-2025-02-19": "token-efficient-tools-2025-02-19",     "web-fetch-2025-09-10": "web-fetch-2025-09-10",     "web-search-2025-03-05": "web-search-2025-03-05"tests/test_litellm/test_anthropic_beta_headers_filtering.pyindex 59bab22de74..3c967283abf 100644@@ -426,6 +426,22 @@ class TestAnthropicBetaHeadersFiltering:          assert filtered == ["fine-grained-tool-streaming-2025-05-14"] +    @pytest.mark.parametrize(+        "provider", ["anthropic", "bedrock", "bedrock_converse", "vertex_ai", "databricks"]+    )+    def test_thinking_binding_controls_forwarded(self, provider):+        """`thinking.block_binding` (preserved thinking, Claude Fable 5.1) is only+        accepted alongside thinking-binding-controls-2026-08-01. The body field is+        forwarded untouched, so stripping the header (previously unknown, hence+        dropped) makes Bedrock and Vertex reject the request with+        "thinking.adaptive.block_binding: Extra inputs are not permitted"."""+        filtered = filter_and_transform_beta_headers(+            beta_headers=["thinking-binding-controls-2026-08-01"],+            provider=provider,+        )++        assert filtered == ["thinking-binding-controls-2026-08-01"]+     def test_null_value_headers_filtered(self):         """Test that headers with null values are always filtered out."""         for provider in [