Backtestslitellmrun-20261018-182052#41202
[Bug]: thinking-binding-controls-2026-08-01 beta header is stripped on /v1/messages, so thinking.block_binding 400s on Bedrock/Vertex
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Both patches fix the root cause identically for the providers the issue names: they register `thinking-binding-controls-2026-08-01` in `anthropic_beta_headers_config.json` so `filter_and_transform_beta_headers` stops dropping it for anthropic, bedrock (invoke), and vertex_ai. The independent tests confirm both pass all five scenarios (Vertex, Bedrock invoke body `anthropic_beta`, first-party header, comma-separated list, Router with the issue's deployment config). The config edits are identical for the first five provider blocks; they differ only in the sixth block, where A forwards the header and B nulls it. That block is not a provider the issue names, so it does not decide the comparison.
B goes further in ways that matter for the same feature: (1) `AnthropicModelInfo` rewrites legacy `thinking: {type: enabled}` to `{type: adaptive}` for Fable 5.1 and previously discarded every other key, which would silently drop `block_binding` even once the header is forwarded; B carries `block_binding` through that rewrite, a low-risk two-line change directly tied to the issue's feature. (2) B adds `block_binding` to `AnthropicThinkingParam` and registers the beta in the `ANTHROPIC_BETA_HEADER_VALUES` enum, matching the convention visible in the surrounding code where each beta header has an enum member.
On tests, A adds one parametrized unit test on the filter function, which does fail-before/pass-after but only exercises the allowlist lookup. B adds the same filter test plus end-to-end tests through `litellm.anthropic_messages` for anthropic, vertex_ai, and bedrock invoke that capture the outgoing request and assert both the beta (header or `anthropic_beta` body array) and `block_binding` survive, plus a regression test for the enabled→adaptive rewrite. These mirror the issue's reproduction far more faithfully and would catch regressions in the transformation layers, not just the JSON lookup.
A's broader coverage of a sixth provider and its `databricks` test parameter are a small point in its favor, but the issue does not ask for it and nothing in front of me confirms that block is databricks. Weighing correctness breadth within the feature, test strength, and consistency with codebase conventions, B is the better fix.
Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.
Solve
42m 39s
Wall-clock time in the container
Full test suite
No new failures
2,539 passed · 0 skipped · 2,539 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
161vs 22 merged
Changed lines, ours vs merged
The issue
View on GitHubCheck for existing issues
- I have searched the existing issues and this is not a duplicate (no hits for
thinking-binding-controlsorblock_binding).
What happened?
Anthropic's preserved-thinking controls for Claude Fable 5.1 (thinking.block_binding.prefix_mismatch_behavior, https://platform.claude.com/docs/en/build-with-claude/preserved-thinking) are only accepted alongside the beta header thinking-binding-controls-2026-08-01.
When a client sends both through the proxy to a bedrock/ or vertex_ai/ Claude deployment on /v1/messages, the proxy forwards the block_binding body field untouched but strips the beta header, because the header has no entry in anthropic_beta_headers_config.json (bundled at v1.101.0-rc.1, and not on main as of 03a4c806c4 either). The provider then rejects the request with:
thinking.adaptive.block_binding: Extra inputs are not permitted
Consequences:
- Nobody behind a LiteLLM gateway can opt into preserved-thinking enforcement on Bedrock/Vertex, which Anthropic's docs tell integrators to do now to test their append-only behaviour before later models enforce it for everyone.
- Accounts created on or after 2026-08-31 are enforced by default on Fable 5.1; the SDKs send
block_bindingfor them, so this is a hard 400 for those users rather than an opt-in inconvenience.
The same request sent directly to the provider (header + block_binding) succeeds and returns input_transformations: [].
User Flow
Before a (hypothetical) fix: a developer testing their integration against preserved thinking gets a 400 from every Claude Fable 5.1 deployment behind the gateway.
- They send
POST https://litellm-domain/v1/messageswith headeranthropic-beta: thinking-binding-controls-2026-08-01and body"model": "claude-fable-5-1", "thinking": {"type": "adaptive", "block_binding": {"prefix_mismatch_behavior": "drop_block"}}, "messages": [{"role": "user", "content": "..."}] - They get HTTP 400
invalid_request_error:Vertex_aiException BadRequestError - thinking.adaptive.block_binding: Extra inputs are not permitted, followed by the configured Bedrock fallback failing the same way (BedrockException - {"message":"thinking.adaptive.block_binding: Extra inputs are not permitted"}) - They remove the
block_bindingfield: the request succeeds, but now they cannot seeinput_transformationsor make the API enforce the prefix check, so they cannot tell whether their integration is affected
After a (hypothetical) fix: the same request goes through and the API reports what it dropped.
- They send the same
POST https://litellm-domain/v1/messageswith the same header and body - They get HTTP 200 with the usual
contentarray plus a top-levelinput_transformations: [] - On a later turn where they edited an earlier message,
input_transformationslists the dropped thinking blocks withreason: "prefix_binding_mismatch"(or, with"error", the 400 Anthropic documents naming what changed)
Proof the bug occurs
Config / setup the proxy ran with (our staging gateway, litellm/litellm-database:v1.101.0-rc.1, Vertex primary with Bedrock fallback; irrelevant deployments omitted):
model_list:
- model_name: claude-fable-5-1
litellm_params:
model: vertex_ai/claude-fable-5-1
additional_drop_params: ["temperature"]
- model_name: bedrock-claude-fable-5-1
litellm_params:
model: bedrock/us.anthropic.claude-fable-5-1
aws_region_name: us-east-1
thinking: {"type": "adaptive"}
litellm_settings:
num_retries: 3
drop_params: true
fallbacks: [{"claude-fable-5-1": ["bedrock-claude-fable-5-1"]}]
vertex_location: "global"
Env: GOOGLE_APPLICATION_CREDENTIALS for Vertex, IRSA for Bedrock, no LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS.
Version: v1.101.0-rc.1 (image litellm/litellm-database:v1.101.0-rc.1).
Command (a 3-turn probe; turn 1 is enough to show it):
curl -sS https://<gateway>/v1/messages \
-H "Authorization: Bearer <virtual key>" \
-H "anthropic-version: 2023-06-01" \
-H "anthropic-beta: thinking-binding-controls-2026-08-01" \
-H "content-type: application/json" \
-d '{"model":"claude-fable-5-1","max_tokens":4000,"output_config":{"effort":"high"},
"thinking":{"type":"adaptive","block_binding":{"prefix_mismatch_behavior":"drop_block"}},
"messages":[{"role":"user","content":"Think carefully before answering. How many positive integers below 5000 have exactly 12 positive divisors? Reply with the number only."}]}'
Output (HTTP 400, x-litellm-call-id: eb073562-5aa2-40f5-92f5-612477a7619c):
{"type":"error","error":{"type":"invalid_request_error","message":"litellm.BadRequestError: Vertex_aiException BadRequestError - {\"type\":\"error\",\"error\":{\"type\":\"invalid_request_error\",\"message\":\"thinking.adaptive.block_binding: Extra inputs are not permitted\"},\"request_id\":\"req_vrtx_011Cf4R9GhDLYQ5oqHS46VnE\"}. Received Model Group=claude-fable-5-1\nAvailable Model Group Fallbacks=['bedrock-claude-fable-5-1']\nError doing the fallback: litellm.BadRequestError: BedrockException - {\"message\":\"thinking.adaptive.block_binding: Extra inputs are not permitted\"} ..."}}
Same request against bedrock-claude-fable-5-1 directly (x-litellm-call-id: e0ddc96d-a555-4c57-84db-9266ab6adf15): HTTP 400, BedrockException - {"message":"thinking.adaptive.block_binding: Extra inputs are not permitted"}.
Same request body without the beta header and without block_binding: HTTP 200 on all three turns, each response carries a thinking block with a signature and no input_transformations field (expected for an account created before 2026-08-31 with no opt-in).
After registering the header (branch fix/anthropic-thinking-binding-beta-header), the identical request to the same two deployments returns HTTP 200 and the response carries input_transformations: [] — the field only exists when the header reached the provider — on both bedrock/us.anthropic.claude-fable-5-1 and vertex_ai/claude-fable-5-1.
Where the header goes: filter_and_transform_beta_headers drops any value not present in the provider's mapping in anthropic_beta_headers_config.json; thinking-binding-controls-2026-08-01 has no entry for any provider. Confirmed on a local litellm --config proxy against a recording upstream: the outgoing Bedrock invoke body has block_binding but no anthropic_beta, and the outgoing first-party request has no anthropic-beta header.
What part of LiteLLM is this about?
Proxy
What LiteLLM version are you on ?
v1.101.0-rc.1
Twitter / LinkedIn details
_No response_
The two patches
litellm/anthropic_beta_headers_config.jsonindex 8dc9204af8..2481fb00f2 100644@@ -28,6 +28,7 @@ "structured-outputs-2025-11-13": "structured-outputs-2025-11-13", "text_editor_20241022": null, "text_editor_20250124": null,+ "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01", "token-efficient-tools-2025-02-19": "token-efficient-tools-2025-02-19", "web-fetch-2025-09-10": "web-fetch-2025-09-10", "web-search-2025-03-05": "web-search-2025-03-05"@@ -59,6 +60,7 @@ "structured-outputs-2025-11-13": "structured-outputs-2025-11-13", "text_editor_20241022": null, "text_editor_20250124": null,+ "thinking-binding-controls-2026-08-01": null, "token-efficient-tools-2025-02-19": null, "web-fetch-2025-09-10": "web-fetch-2025-09-10", "web-search-2025-03-05": "web-search-2025-03-05"@@ -90,6 +92,7 @@ "structured-outputs-2025-11-13": "structured-outputs-2025-11-13", "text_editor_20241022": null, "text_editor_20250124": null,+ "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01", "token-efficient-tools-2025-02-19": null, "tool-search-tool-2025-10-19": null, "web-fetch-2025-09-10": null,@@ -122,6 +125,7 @@ "structured-outputs-2025-11-13": null, "text_editor_20241022": null, "text_editor_20250124": null,+ "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01", "token-efficient-tools-2025-02-19": null, "tool-search-tool-2025-10-19": "tool-search-tool-2025-10-19", "web-fetch-2025-09-10": null,@@ -154,6 +158,7 @@ "structured-outputs-2025-11-13": null, "text_editor_20241022": null, "text_editor_20250124": null,+ "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01", "token-efficient-tools-2025-02-19": null, "tool-search-tool-2025-10-19": "tool-search-tool-2025-10-19", "web-fetch-2025-09-10": null,@@ -187,6 +192,7 @@ "structured-outputs-2025-11-13": "structured-outputs-2025-11-13", "text_editor_20241022": null, "text_editor_20250124": null,+ "thinking-binding-controls-2026-08-01": null, "token-efficient-tools-2025-02-19": "token-efficient-tools-2025-02-19", "web-fetch-2025-09-10": "web-fetch-2025-09-10", "web-search-2025-03-05": "web-search-2025-03-05"litellm/llms/anthropic/common_utils.pyindex d35a937205..69b14cff02 100644@@ -669,7 +669,11 @@ class AnthropicModelInfo(BaseLLMModelInfo): custom_llm_provider=custom_llm_provider, ) existing_output_config: Final = optional_params.get("output_config")- optional_params["thinking"] = {"type": "adaptive"}+ optional_params["thinking"] = {+ "type": "adaptive",+ # Carry preserved-thinking controls over to the adaptive shape.+ **({"block_binding": thinking["block_binding"]} if "block_binding" in thinking else {}),+ } optional_params["output_config"] = { "effort": effort, **(existing_output_config if isinstance(existing_output_config, dict) else MappingProxyType({})),litellm/types/llms/anthropic.pyindex bcdee86360..8af0f271e0 100644@@ -721,6 +721,8 @@ class AnthropicThinkingParam(TypedDict, total=False): type: ReadOnly[Literal["enabled", "adaptive", "disabled"]] budget_tokens: int display: ReadOnly[Literal["summarized", "omitted"]]+ # Preserved-thinking controls; requires the thinking-binding-controls-2026-08-01 beta.+ block_binding: dict class ANTHROPIC_HOSTED_TOOLS(str, Enum):@@ -747,6 +749,7 @@ class ANTHROPIC_BETA_HEADER_VALUES(str, Enum): FAST_MODE_2026_02_01 = "fast-mode-2026-02-01" ADVISOR_TOOL_2026_03_01 = "advisor-tool-2026-03-01" PER_TURN_CONTROL_2026_07_01 = "per-turn-control-2026-07-01"+ THINKING_BINDING_CONTROLS_2026_08_01 = "thinking-binding-controls-2026-08-01" # Tool search beta header constant (for Anthropic direct API and Microsoft Foundry)tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_anthropic_messages_thinking_binding_controls.pynew file mode 100644index 0000000000..0500972186@@ -0,0 +1,146 @@+"""+The ``thinking-binding-controls-2026-08-01`` beta gates+``thinking.block_binding`` (preserved-thinking controls). The beta allowlist+used to have no entry for it, so /v1/messages forwarded ``block_binding`` but+stripped the beta, and Bedrock / Vertex / Anthropic 400'd with+``thinking.adaptive.block_binding: Extra inputs are not permitted``.+"""++import json+from unittest.mock import patch++import httpx+import pytest++import litellm+from litellm import anthropic_beta_headers_manager+from litellm.anthropic_beta_headers_manager import filter_and_transform_beta_headers++THINKING_BINDING_BETA = "thinking-binding-controls-2026-08-01"+THINKING = {"type": "adaptive", "block_binding": {"prefix_mismatch_behavior": "drop_block"}}++_ANTHROPIC_RESPONSE = {+ "id": "msg_test",+ "type": "message",+ "role": "assistant",+ "model": "claude-fable-5-1",+ "content": [{"type": "text", "text": "ok"}],+ "stop_reason": "end_turn",+ "stop_sequence": None,+ "usage": {"input_tokens": 1, "output_tokens": 1},+ "input_transformations": [],+}+++@pytest.fixture(autouse=True)+def bundled_beta_allowlist(monkeypatch):+ monkeypatch.setenv("LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS", "True")+ monkeypatch.setattr(anthropic_beta_headers_manager, "_BETA_HEADERS_CONFIG", None)+ yield+ monkeypatch.setattr(anthropic_beta_headers_manager, "_BETA_HEADERS_CONFIG", None)+++def _betas_from_header(headers):+ return {b.strip() for b in (headers.get("anthropic-beta") or "").split(",") if b.strip()}+++async def _send_messages_request(model, **kwargs):+ captured = []++ async def mock_post(self, url, data=None, headers=None, **kwargs):+ body = kwargs.get("json")+ if body is None:+ body = data if isinstance(data, dict) else json.loads(data)+ captured.append({"url": str(url), "headers": dict(headers or {}), "body": body})+ return httpx.Response(+ status_code=200,+ content=json.dumps(_ANTHROPIC_RESPONSE).encode(),+ headers={"content-type": "application/json"},+ request=httpx.Request("POST", str(url)),+ )++ try:+ with patch("litellm.llms.custom_httpx.http_handler.AsyncHTTPHandler.post", new=mock_post):+ response = await litellm.anthropic_messages(+ model=model,+ messages=[{"role": "user", "content": "How many divisors?"}],+ max_tokens=4000,+ thinking=kwargs.pop("thinking", THINKING),+ extra_headers={"anthropic-beta": THINKING_BINDING_BETA},+ **kwargs,+ )+ finally:+ await litellm.close_litellm_async_clients()++ assert len(captured) == 1+ return response, captured[0]+++@pytest.mark.asyncio+async def test_anthropic_messages_forwards_thinking_binding_beta_to_anthropic():+ response, request = await _send_messages_request(+ model="anthropic/claude-fable-5-1",+ api_key="sk-ant-test",+ )++ assert THINKING_BINDING_BETA in _betas_from_header(request["headers"])+ assert request["body"]["thinking"]["block_binding"] == THINKING["block_binding"]+ assert response["content"][0]["text"] == "ok"+++@pytest.mark.asyncio+async def test_anthropic_messages_forwards_thinking_binding_beta_to_vertex_ai():+ from litellm.llms.vertex_ai.vertex_llm_base import VertexBase++ with patch.object(VertexBase, "_ensure_access_token", return_value=("ya29.token", "test-project")):+ response, request = await _send_messages_request(+ model="vertex_ai/claude-fable-5-1",+ vertex_project="test-project",+ vertex_location="global",+ )++ assert THINKING_BINDING_BETA in _betas_from_header(request["headers"])+ assert request["body"]["thinking"]["block_binding"] == THINKING["block_binding"]+ assert response["content"][0]["text"] == "ok"+++@pytest.mark.asyncio+async def test_anthropic_messages_forwards_thinking_binding_beta_to_bedrock_invoke():+ response, request = await _send_messages_request(+ model="bedrock/us.anthropic.claude-fable-5-1",+ aws_access_key_id="AKIAFAKE",+ aws_secret_access_key="fake-secret",+ aws_region_name="us-east-1",+ )++ assert THINKING_BINDING_BETA in request["body"].get("anthropic_beta", [])+ assert request["body"]["thinking"]["block_binding"] == THINKING["block_binding"]+ assert response["content"][0]["text"] == "ok"+++@pytest.mark.asyncio+@pytest.mark.parametrize(+ "model, provider_kwargs",+ [+ ("anthropic/claude-fable-5-1", {"api_key": "sk-ant-test"}),+ (+ "bedrock/us.anthropic.claude-fable-5-1",+ {"aws_access_key_id": "AKIAFAKE", "aws_secret_access_key": "fake-secret", "aws_region_name": "us-east-1"},+ ),+ ],+)+async def test_legacy_enabled_thinking_keeps_block_binding_on_adaptive_model(model, provider_kwargs):+ """Legacy ``type: enabled`` is rewritten to adaptive for Fable 5.1; the+ rewrite must not drop ``block_binding``."""+ _, request = await _send_messages_request(+ model=model,+ thinking={"type": "enabled", "budget_tokens": 2048, "block_binding": THINKING["block_binding"]},+ **provider_kwargs,+ )++ assert request["body"]["thinking"] == {"type": "adaptive", "block_binding": THINKING["block_binding"]}+++@pytest.mark.parametrize("provider", ["anthropic", "bedrock", "bedrock_converse", "vertex_ai"])+def test_thinking_binding_beta_survives_provider_allowlist(provider):+ assert filter_and_transform_beta_headers([THINKING_BINDING_BETA], provider=provider) == [THINKING_BINDING_BETA]
litellm/anthropic_beta_headers_config.jsonindex 8dc9204af8d..eb31cc17a15 100644@@ -28,6 +28,7 @@ "structured-outputs-2025-11-13": "structured-outputs-2025-11-13", "text_editor_20241022": null, "text_editor_20250124": null,+ "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01", "token-efficient-tools-2025-02-19": "token-efficient-tools-2025-02-19", "web-fetch-2025-09-10": "web-fetch-2025-09-10", "web-search-2025-03-05": "web-search-2025-03-05"@@ -59,6 +60,7 @@ "structured-outputs-2025-11-13": "structured-outputs-2025-11-13", "text_editor_20241022": null, "text_editor_20250124": null,+ "thinking-binding-controls-2026-08-01": null, "token-efficient-tools-2025-02-19": null, "web-fetch-2025-09-10": "web-fetch-2025-09-10", "web-search-2025-03-05": "web-search-2025-03-05"@@ -90,6 +92,7 @@ "structured-outputs-2025-11-13": "structured-outputs-2025-11-13", "text_editor_20241022": null, "text_editor_20250124": null,+ "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01", "token-efficient-tools-2025-02-19": null, "tool-search-tool-2025-10-19": null, "web-fetch-2025-09-10": null,@@ -122,6 +125,7 @@ "structured-outputs-2025-11-13": null, "text_editor_20241022": null, "text_editor_20250124": null,+ "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01", "token-efficient-tools-2025-02-19": null, "tool-search-tool-2025-10-19": "tool-search-tool-2025-10-19", "web-fetch-2025-09-10": null,@@ -154,6 +158,7 @@ "structured-outputs-2025-11-13": null, "text_editor_20241022": null, "text_editor_20250124": null,+ "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01", "token-efficient-tools-2025-02-19": null, "tool-search-tool-2025-10-19": "tool-search-tool-2025-10-19", "web-fetch-2025-09-10": null,@@ -187,6 +192,7 @@ "structured-outputs-2025-11-13": "structured-outputs-2025-11-13", "text_editor_20241022": null, "text_editor_20250124": null,+ "thinking-binding-controls-2026-08-01": "thinking-binding-controls-2026-08-01", "token-efficient-tools-2025-02-19": "token-efficient-tools-2025-02-19", "web-fetch-2025-09-10": "web-fetch-2025-09-10", "web-search-2025-03-05": "web-search-2025-03-05"tests/test_litellm/test_anthropic_beta_headers_filtering.pyindex 59bab22de74..3c967283abf 100644@@ -426,6 +426,22 @@ class TestAnthropicBetaHeadersFiltering: assert filtered == ["fine-grained-tool-streaming-2025-05-14"] + @pytest.mark.parametrize(+ "provider", ["anthropic", "bedrock", "bedrock_converse", "vertex_ai", "databricks"]+ )+ def test_thinking_binding_controls_forwarded(self, provider):+ """`thinking.block_binding` (preserved thinking, Claude Fable 5.1) is only+ accepted alongside thinking-binding-controls-2026-08-01. The body field is+ forwarded untouched, so stripping the header (previously unknown, hence+ dropped) makes Bedrock and Vertex reject the request with+ "thinking.adaptive.block_binding: Extra inputs are not permitted"."""+ filtered = filter_and_transform_beta_headers(+ beta_headers=["thinking-binding-controls-2026-08-01"],+ provider=provider,+ )++ assert filtered == ["thinking-binding-controls-2026-08-01"]+ def test_null_value_headers_filtered(self): """Test that headers with null values are always filtered out.""" for provider in [