Backtestslitellmrun-20261018-182052#41202

Issue #41202 Merged fix PR #41203 Run 18 Oct 2026 · run-20261018-182052 BerriAI/litellm

[Bug]: thinking-binding-controls-2026-08-01 beta header is stripped on /v1/messages, so thinking.block_binding 400s on Bedrock/Vertex

Better

Our patch was judged better than the merged fix.

In this judgment, Patch B = our agent, Patch A = the merged fix.

Both patches fix the root cause identically for the providers the issue names: they register `thinking-binding-controls-2026-08-01` in `anthropic_beta_headers_config.json` so `filter_and_transform_beta_headers` stops dropping it for anthropic, bedrock (invoke), and vertex_ai. The independent tests confirm both pass all five scenarios (Vertex, Bedrock invoke body `anthropic_beta`, first-party header, comma-separated list, Router with the issue's deployment config). The config edits are identical for the first five provider blocks; they differ only in the sixth block, where A forwards the header and B nulls it. That block is not a provider the issue names, so it does not decide the comparison.

B goes further in ways that matter for the same feature: (1) `AnthropicModelInfo` rewrites legacy `thinking: {type: enabled}` to `{type: adaptive}` for Fable 5.1 and previously discarded every other key, which would silently drop `block_binding` even once the header is forwarded; B carries `block_binding` through that rewrite, a low-risk two-line change directly tied to the issue's feature. (2) B adds `block_binding` to `AnthropicThinkingParam` and registers the beta in the `ANTHROPIC_BETA_HEADER_VALUES` enum, matching the convention visible in the surrounding code where each beta header has an enum member. 

On tests, A adds one parametrized unit test on the filter function, which does fail-before/pass-after but only exercises the allowlist lookup. B adds the same filter test plus end-to-end tests through `litellm.anthropic_messages` for anthropic, vertex_ai, and bedrock invoke that capture the outgoing request and assert both the beta (header or `anthropic_beta` body array) and `block_binding` survive, plus a regression test for the enabled→adaptive rewrite. These mirror the issue's reproduction far more faithfully and would catch regressions in the transformation layers, not just the JSON lookup.

A's broader coverage of a sixth provider and its `databricks` test parameter are a small point in its favor, but the issue does not ask for it and nothing in front of me confirms that block is databricks. Weighing correctness breadth within the feature, test strength, and consistency with codebase conventions, B is the better fix.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

42m 39s

Wall-clock time in the container

Full test suite

No new failures

2,539 passed · 0 skipped · 2,539 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

161vs 22 merged

Changed lines, ours vs merged

Check for existing issues

  • I have searched the existing issues and this is not a duplicate (no hits for thinking-binding-controls or block_binding).

What happened?

Anthropic's preserved-thinking controls for Claude Fable 5.1 (thinking.block_binding.prefix_mismatch_behavior, https://platform.claude.com/docs/en/build-with-claude/preserved-thinking) are only accepted alongside the beta header thinking-binding-controls-2026-08-01.

When a client sends both through the proxy to a bedrock/ or vertex_ai/ Claude deployment on /v1/messages, the proxy forwards the block_binding body field untouched but strips the beta header, because the header has no entry in anthropic_beta_headers_config.json (bundled at v1.101.0-rc.1, and not on main as of 03a4c806c4 either). The provider then rejects the request with:

thinking.adaptive.block_binding: Extra inputs are not permitted

Consequences:

  • Nobody behind a LiteLLM gateway can opt into preserved-thinking enforcement on Bedrock/Vertex, which Anthropic's docs tell integrators to do now to test their append-only behaviour before later models enforce it for everyone.
  • Accounts created on or after 2026-08-31 are enforced by default on Fable 5.1; the SDKs send block_binding for them, so this is a hard 400 for those users rather than an opt-in inconvenience.

The same request sent directly to the provider (header + block_binding) succeeds and returns input_transformations: [].

User Flow

Before a (hypothetical) fix: a developer testing their integration against preserved thinking gets a 400 from every Claude Fable 5.1 deployment behind the gateway.

  1. They send POST https://litellm-domain/v1/messages with header anthropic-beta: thinking-binding-controls-2026-08-01 and body "model": "claude-fable-5-1", "thinking": {"type": "adaptive", "block_binding": {"prefix_mismatch_behavior": "drop_block"}}, "messages": [{"role": "user", "content": "..."}]
  2. They get HTTP 400 invalid_request_error: Vertex_aiException BadRequestError - thinking.adaptive.block_binding: Extra inputs are not permitted, followed by the configured Bedrock fallback failing the same way (BedrockException - {"message":"thinking.adaptive.block_binding: Extra inputs are not permitted"})
  3. They remove the block_binding field: the request succeeds, but now they cannot see input_transformations or make the API enforce the prefix check, so they cannot tell whether their integration is affected

After a (hypothetical) fix: the same request goes through and the API reports what it dropped.

  1. They send the same POST https://litellm-domain/v1/messages with the same header and body
  2. They get HTTP 200 with the usual content array plus a top-level input_transformations: []
  3. On a later turn where they edited an earlier message, input_transformations lists the dropped thinking blocks with reason: "prefix_binding_mismatch" (or, with "error", the 400 Anthropic documents naming what changed)

Proof the bug occurs

Config / setup the proxy ran with (our staging gateway, litellm/litellm-database:v1.101.0-rc.1, Vertex primary with Bedrock fallback; irrelevant deployments omitted):

model_list:
  - model_name: claude-fable-5-1
    litellm_params:
      model: vertex_ai/claude-fable-5-1
      additional_drop_params: ["temperature"]
  - model_name: bedrock-claude-fable-5-1
    litellm_params:
      model: bedrock/us.anthropic.claude-fable-5-1
      aws_region_name: us-east-1
      thinking: {"type": "adaptive"}
litellm_settings:
  num_retries: 3
  drop_params: true
  fallbacks: [{"claude-fable-5-1": ["bedrock-claude-fable-5-1"]}]
  vertex_location: "global"

Env: GOOGLE_APPLICATION_CREDENTIALS for Vertex, IRSA for Bedrock, no LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS.

Version: v1.101.0-rc.1 (image litellm/litellm-database:v1.101.0-rc.1).

Command (a 3-turn probe; turn 1 is enough to show it):

curl -sS https://<gateway>/v1/messages \
  -H "Authorization: Bearer <virtual key>" \
  -H "anthropic-version: 2023-06-01" \
  -H "anthropic-beta: thinking-binding-controls-2026-08-01" \
  -H "content-type: application/json" \
  -d '{"model":"claude-fable-5-1","max_tokens":4000,"output_config":{"effort":"high"},
       "thinking":{"type":"adaptive","block_binding":{"prefix_mismatch_behavior":"drop_block"}},
       "messages":[{"role":"user","content":"Think carefully before answering. How many positive integers below 5000 have exactly 12 positive divisors? Reply with the number only."}]}'

Output (HTTP 400, x-litellm-call-id: eb073562-5aa2-40f5-92f5-612477a7619c):

{"type":"error","error":{"type":"invalid_request_error","message":"litellm.BadRequestError: Vertex_aiException BadRequestError - {\"type\":\"error\",\"error\":{\"type\":\"invalid_request_error\",\"message\":\"thinking.adaptive.block_binding: Extra inputs are not permitted\"},\"request_id\":\"req_vrtx_011Cf4R9GhDLYQ5oqHS46VnE\"}. Received Model Group=claude-fable-5-1\nAvailable Model Group Fallbacks=['bedrock-claude-fable-5-1']\nError doing the fallback: litellm.BadRequestError: BedrockException - {\"message\":\"thinking.adaptive.block_binding: Extra inputs are not permitted\"} ..."}}

Same request against bedrock-claude-fable-5-1 directly (x-litellm-call-id: e0ddc96d-a555-4c57-84db-9266ab6adf15): HTTP 400, BedrockException - {"message":"thinking.adaptive.block_binding: Extra inputs are not permitted"}.

Same request body without the beta header and without block_binding: HTTP 200 on all three turns, each response carries a thinking block with a signature and no input_transformations field (expected for an account created before 2026-08-31 with no opt-in).

After registering the header (branch fix/anthropic-thinking-binding-beta-header), the identical request to the same two deployments returns HTTP 200 and the response carries input_transformations: [] — the field only exists when the header reached the provider — on both bedrock/us.anthropic.claude-fable-5-1 and vertex_ai/claude-fable-5-1.

Where the header goes: filter_and_transform_beta_headers drops any value not present in the provider's mapping in anthropic_beta_headers_config.json; thinking-binding-controls-2026-08-01 has no entry for any provider. Confirmed on a local litellm --config proxy against a recording upstream: the outgoing Bedrock invoke body has block_binding but no anthropic_beta, and the outgoing first-party request has no anthropic-beta header.

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.101.0-rc.1

Twitter / LinkedIn details

_No response_

The two patches

Our patch+160 −1
litellm/anthropic_beta_headers_config.jsonindex 8dc9204af8..2481fb00f2 100644@@ -28,6 +28,7 @@     "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01",     "token-efficient-tools-2025-02-19": "token-efficient-tools-2025-02-19",     "web-fetch-2025-09-10": "web-fetch-2025-09-10",     "web-search-2025-03-05": "web-search-2025-03-05"@@ -59,6 +60,7 @@     "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": null,     "token-efficient-tools-2025-02-19": null,     "web-fetch-2025-09-10": "web-fetch-2025-09-10",     "web-search-2025-03-05": "web-search-2025-03-05"@@ -90,6 +92,7 @@     "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01",     "token-efficient-tools-2025-02-19": null,     "tool-search-tool-2025-10-19": null,     "web-fetch-2025-09-10": null,@@ -122,6 +125,7 @@     "structured-outputs-2025-11-13": null,     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01",     "token-efficient-tools-2025-02-19": null,     "tool-search-tool-2025-10-19": "tool-search-tool-2025-10-19",     "web-fetch-2025-09-10": null,@@ -154,6 +158,7 @@     "structured-outputs-2025-11-13": null,     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01",     "token-efficient-tools-2025-02-19": null,     "tool-search-tool-2025-10-19": "tool-search-tool-2025-10-19",     "web-fetch-2025-09-10": null,@@ -187,6 +192,7 @@     "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": null,     "token-efficient-tools-2025-02-19": "token-efficient-tools-2025-02-19",     "web-fetch-2025-09-10": "web-fetch-2025-09-10",     "web-search-2025-03-05": "web-search-2025-03-05"litellm/llms/anthropic/common_utils.pyindex d35a937205..69b14cff02 100644@@ -669,7 +669,11 @@ class AnthropicModelInfo(BaseLLMModelInfo):             custom_llm_provider=custom_llm_provider,         )         existing_output_config: Final = optional_params.get("output_config")-        optional_params["thinking"] = {"type": "adaptive"}+        optional_params["thinking"] = {+            "type": "adaptive",+            # Carry preserved-thinking controls over to the adaptive shape.+            **({"block_binding": thinking["block_binding"]} if "block_binding" in thinking else {}),+        }         optional_params["output_config"] = {             "effort": effort,             **(existing_output_config if isinstance(existing_output_config, dict) else MappingProxyType({})),litellm/types/llms/anthropic.pyindex bcdee86360..8af0f271e0 100644@@ -721,6 +721,8 @@ class AnthropicThinkingParam(TypedDict, total=False):     type: ReadOnly[Literal["enabled", "adaptive", "disabled"]]     budget_tokens: int     display: ReadOnly[Literal["summarized", "omitted"]]+    # Preserved-thinking controls; requires the thinking-binding-controls-2026-08-01 beta.+    block_binding: dict   class ANTHROPIC_HOSTED_TOOLS(str, Enum):@@ -747,6 +749,7 @@ class ANTHROPIC_BETA_HEADER_VALUES(str, Enum):     FAST_MODE_2026_02_01 = "fast-mode-2026-02-01"     ADVISOR_TOOL_2026_03_01 = "advisor-tool-2026-03-01"     PER_TURN_CONTROL_2026_07_01 = "per-turn-control-2026-07-01"+    THINKING_BINDING_CONTROLS_2026_08_01 = "thinking-binding-controls-2026-08-01"   # Tool search beta header constant (for Anthropic direct API and Microsoft Foundry)tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_anthropic_messages_thinking_binding_controls.pynew file mode 100644index 0000000000..0500972186@@ -0,0 +1,146 @@+"""+The ``thinking-binding-controls-2026-08-01`` beta gates+``thinking.block_binding`` (preserved-thinking controls). The beta allowlist+used to have no entry for it, so /v1/messages forwarded ``block_binding`` but+stripped the beta, and Bedrock / Vertex / Anthropic 400'd with+``thinking.adaptive.block_binding: Extra inputs are not permitted``.+"""++import json+from unittest.mock import patch++import httpx+import pytest++import litellm+from litellm import anthropic_beta_headers_manager+from litellm.anthropic_beta_headers_manager import filter_and_transform_beta_headers++THINKING_BINDING_BETA = "thinking-binding-controls-2026-08-01"+THINKING = {"type": "adaptive", "block_binding": {"prefix_mismatch_behavior": "drop_block"}}++_ANTHROPIC_RESPONSE = {+    "id": "msg_test",+    "type": "message",+    "role": "assistant",+    "model": "claude-fable-5-1",+    "content": [{"type": "text", "text": "ok"}],+    "stop_reason": "end_turn",+    "stop_sequence": None,+    "usage": {"input_tokens": 1, "output_tokens": 1},+    "input_transformations": [],+}+++@pytest.fixture(autouse=True)+def bundled_beta_allowlist(monkeypatch):+    monkeypatch.setenv("LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS", "True")+    monkeypatch.setattr(anthropic_beta_headers_manager, "_BETA_HEADERS_CONFIG", None)+    yield+    monkeypatch.setattr(anthropic_beta_headers_manager, "_BETA_HEADERS_CONFIG", None)+++def _betas_from_header(headers):+    return {b.strip() for b in (headers.get("anthropic-beta") or "").split(",") if b.strip()}+++async def _send_messages_request(model, **kwargs):+    captured = []++    async def mock_post(self, url, data=None, headers=None, **kwargs):+        body = kwargs.get("json")+        if body is None:+            body = data if isinstance(data, dict) else json.loads(data)+        captured.append({"url": str(url), "headers": dict(headers or {}), "body": body})+        return httpx.Response(+            status_code=200,+            content=json.dumps(_ANTHROPIC_RESPONSE).encode(),+            headers={"content-type": "application/json"},+            request=httpx.Request("POST", str(url)),+        )++    try:+        with patch("litellm.llms.custom_httpx.http_handler.AsyncHTTPHandler.post", new=mock_post):+            response = await litellm.anthropic_messages(+                model=model,+                messages=[{"role": "user", "content": "How many divisors?"}],+                max_tokens=4000,+                thinking=kwargs.pop("thinking", THINKING),+                extra_headers={"anthropic-beta": THINKING_BINDING_BETA},+                **kwargs,+            )+    finally:+        await litellm.close_litellm_async_clients()++    assert len(captured) == 1+    return response, captured[0]+++@pytest.mark.asyncio+async def test_anthropic_messages_forwards_thinking_binding_beta_to_anthropic():+    response, request = await _send_messages_request(+        model="anthropic/claude-fable-5-1",+        api_key="sk-ant-test",+    )++    assert THINKING_BINDING_BETA in _betas_from_header(request["headers"])+    assert request["body"]["thinking"]["block_binding"] == THINKING["block_binding"]+    assert response["content"][0]["text"] == "ok"+++@pytest.mark.asyncio+async def test_anthropic_messages_forwards_thinking_binding_beta_to_vertex_ai():+    from litellm.llms.vertex_ai.vertex_llm_base import VertexBase++    with patch.object(VertexBase, "_ensure_access_token", return_value=("ya29.token", "test-project")):+        response, request = await _send_messages_request(+            model="vertex_ai/claude-fable-5-1",+            vertex_project="test-project",+            vertex_location="global",+        )++    assert THINKING_BINDING_BETA in _betas_from_header(request["headers"])+    assert request["body"]["thinking"]["block_binding"] == THINKING["block_binding"]+    assert response["content"][0]["text"] == "ok"+++@pytest.mark.asyncio+async def test_anthropic_messages_forwards_thinking_binding_beta_to_bedrock_invoke():+    response, request = await _send_messages_request(+        model="bedrock/us.anthropic.claude-fable-5-1",+        aws_access_key_id="AKIAFAKE",+        aws_secret_access_key="fake-secret",+        aws_region_name="us-east-1",+    )++    assert THINKING_BINDING_BETA in request["body"].get("anthropic_beta", [])+    assert request["body"]["thinking"]["block_binding"] == THINKING["block_binding"]+    assert response["content"][0]["text"] == "ok"+++@pytest.mark.asyncio+@pytest.mark.parametrize(+    "model, provider_kwargs",+    [+        ("anthropic/claude-fable-5-1", {"api_key": "sk-ant-test"}),+        (+            "bedrock/us.anthropic.claude-fable-5-1",+            {"aws_access_key_id": "AKIAFAKE", "aws_secret_access_key": "fake-secret", "aws_region_name": "us-east-1"},+        ),+    ],+)+async def test_legacy_enabled_thinking_keeps_block_binding_on_adaptive_model(model, provider_kwargs):+    """Legacy ``type: enabled`` is rewritten to adaptive for Fable 5.1; the+    rewrite must not drop ``block_binding``."""+    _, request = await _send_messages_request(+        model=model,+        thinking={"type": "enabled", "budget_tokens": 2048, "block_binding": THINKING["block_binding"]},+        **provider_kwargs,+    )++    assert request["body"]["thinking"] == {"type": "adaptive", "block_binding": THINKING["block_binding"]}+++@pytest.mark.parametrize("provider", ["anthropic", "bedrock", "bedrock_converse", "vertex_ai"])+def test_thinking_binding_beta_survives_provider_allowlist(provider):+    assert filter_and_transform_beta_headers([THINKING_BINDING_BETA], provider=provider) == [THINKING_BINDING_BETA] 
Merged fix · PR #41203+22 −0
litellm/anthropic_beta_headers_config.jsonindex 8dc9204af8d..eb31cc17a15 100644@@ -28,6 +28,7 @@     "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01",     "token-efficient-tools-2025-02-19": "token-efficient-tools-2025-02-19",     "web-fetch-2025-09-10": "web-fetch-2025-09-10",     "web-search-2025-03-05": "web-search-2025-03-05"@@ -59,6 +60,7 @@     "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": null,     "token-efficient-tools-2025-02-19": null,     "web-fetch-2025-09-10": "web-fetch-2025-09-10",     "web-search-2025-03-05": "web-search-2025-03-05"@@ -90,6 +92,7 @@     "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01",     "token-efficient-tools-2025-02-19": null,     "tool-search-tool-2025-10-19": null,     "web-fetch-2025-09-10": null,@@ -122,6 +125,7 @@     "structured-outputs-2025-11-13": null,     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01",     "token-efficient-tools-2025-02-19": null,     "tool-search-tool-2025-10-19": "tool-search-tool-2025-10-19",     "web-fetch-2025-09-10": null,@@ -154,6 +158,7 @@     "structured-outputs-2025-11-13": null,     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01",     "token-efficient-tools-2025-02-19": null,     "tool-search-tool-2025-10-19": "tool-search-tool-2025-10-19",     "web-fetch-2025-09-10": null,@@ -187,6 +192,7 @@     "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",     "text_editor_20241022": null,     "text_editor_20250124": null,+    "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01",     "token-efficient-tools-2025-02-19": "token-efficient-tools-2025-02-19",     "web-fetch-2025-09-10": "web-fetch-2025-09-10",     "web-search-2025-03-05": "web-search-2025-03-05"tests/test_litellm/test_anthropic_beta_headers_filtering.pyindex 59bab22de74..3c967283abf 100644@@ -426,6 +426,22 @@ class TestAnthropicBetaHeadersFiltering:          assert filtered == ["fine-grained-tool-streaming-2025-05-14"] +    @pytest.mark.parametrize(+        "provider", ["anthropic", "bedrock", "bedrock_converse", "vertex_ai", "databricks"]+    )+    def test_thinking_binding_controls_forwarded(self, provider):+        """`thinking.block_binding` (preserved thinking, Claude Fable 5.1) is only+        accepted alongside thinking-binding-controls-2026-08-01. The body field is+        forwarded untouched, so stripping the header (previously unknown, hence+        dropped) makes Bedrock and Vertex reject the request with+        "thinking.adaptive.block_binding: Extra inputs are not permitted"."""+        filtered = filter_and_transform_beta_headers(+            beta_headers=["thinking-binding-controls-2026-08-01"],+            provider=provider,+        )++        assert filtered == ["thinking-binding-controls-2026-08-01"]+     def test_null_value_headers_filtered(self):         """Test that headers with null values are always filtered out."""         for provider in [