Backtestslitellmrun-20261018-182052#26113

Issue #26113 Merged fix PR # Run 18 Oct 2026 · run-20261018-182052 BerriAI/litellm

[Bug]: remove_custom_field_from_tools() strips defer_loading — breaks Tool Search on Bedrock, wastes ~20k tokens per request

Better

Our patch was judged better than the merged fix.

In this judgment, Patch B = our agent, Patch A = the merged fix.

Patch A deletes remove_custom_field_from_tools() from common_utils.py but does not touch the two transformation files that import and call it. The independent tests confirm this breaks the codebase: both fail under Patch A with ImportError 'cannot import name remove_custom_field_from_tools', and the chat path surfaces it as an APIConnectionError. Every Bedrock Invoke Claude request would fail, so Patch A is strictly worse than the base commit. Patch B keeps the helper, gates both call sites (chat and messages invoke transformations) behind a new litellm.bedrock_remove_tool_custom_field flag defaulting to False, so defer_loading is preserved by default and Tool Search works, while retaining an opt-in for users on regions where Bedrock still rejects the field. Both independent tests pass on B. B also adds tests on both paths covering preserve-by-default and strip-when-opted-in, updates the docstring, and documents the setting and env var. The config knob is modest extra scope but the issue itself suggested making it conditional/configurable, and it adds no risk. B is clearly better.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

25m 52s

Wall-clock time in the container

Full test suite

No new failures

5,989 passed · 92 skipped · 6,084 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

133vs 21 merged

Changed lines, ours vs merged

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

PR #22861 (merged 2026-03-05, fixing #22847) added remove_custom_field_from_tools() to strip the custom: {defer_loading: true} field from tool definitions before forwarding to Bedrock. At the time, Bedrock rejected this field with Extra inputs are not permitted.

Bedrock has since updated and now accepts custom.defer_loading. When using Claude Code directly against Bedrock (without LiteLLM), Tool Search works correctly — deferred MCP tools are excluded from the system prompt prefix and only ~1.2% of context is consumed by MCP tool definitions.

However, when routing through LiteLLM → Bedrock, remove_custom_field_from_tools() unconditionally strips the custom field. Bedrock then sees all tools as non-deferred and loads every tool definition into the context. This causes ~20k+ extra input tokens per request for a typical MCP setup (22 tools across 5 MCP servers).

Impact

PathMCP tools token costBehavior
Claude Code → Bedrock (direct)~12k tokens (1.2% of context)defer_loading preserved, Tool Search works
Claude Code → LiteLLM → Bedrock~30k+ tokensdefer_loading stripped, all tools fully loaded

This is not just a logging/display issue — it's real additional token consumption and cost on every request.

Root cause

remove_custom_field_from_tools() in litellm/llms/bedrock/common_utils.py unconditionally removes the custom field:

  • litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py
  • litellm/llms/bedrock/chat/invoke_transformations/anthropic_claude3_transformation.py

Suggested fix

Remove remove_custom_field_from_tools() entirely, or make it conditional/configurable. Bedrock now accepts this field natively.

Steps to Reproduce

  1. Configure Claude Code to use LiteLLM proxy pointing to Bedrock
  2. Add multiple MCP servers (e.g., exa, aws-knowledge, aws-pricing, context7, calculator — 22 tools total)
  3. Start a conversation with a simple prompt like "hi"
  4. Compare token usage in LiteLLM logs vs direct Bedrock connection
  5. LiteLLM path shows ~30k+ tokens for messages; direct Bedrock shows ~2.5k

Environment

  • LiteLLM: v1.82.3-stable (Production)
  • Claude Code: latest
  • Bedrock region: ap-northeast-1
  • Model: Claude Opus 4.6 / Sonnet 4.6

Relevant log output

In LiteLLM request logs, the forwarded request to Bedrock contains tool definitions without custom.defer_loading, while the original request from Claude Code includes it.

Are you a LiteLLM Enterprise user?

No

Twitter / LinkedIn details

_No response_

The two patches

Our patch+125 −8
docs/my-website/docs/proxy/config_settings.mdindex a886a754f5..0e199e80b0 100644@@ -178,6 +178,7 @@ router_settings: | service_callbacks | array of strings | System health monitoring - Logs redis, postgres failures on specified services (e.g. datadog, prometheus) [Doc Metrics](prometheus) | | turn_off_message_logging | boolean | If true, prevents messages and responses from being logged to callbacks, but request metadata will still be logged. Useful for privacy/compliance when handling sensitive data [Proxy Logging](logging) | | modify_params | boolean | If true, allows modifying the parameters of the request before it is sent to the LLM provider |+| bedrock_remove_tool_custom_field | boolean | If true, strips the `custom` field (e.g. `custom: {defer_loading: true}` sent by Claude Code) from tool definitions sent to Bedrock Invoke. Default false - stripping it disables Tool Search. | | enable_preview_features | boolean | If true, enables preview features - e.g. Azure O1 Models with streaming support.| | LITELLM_DISABLE_STOP_SEQUENCE_LIMIT | Disable validation for stop sequence limit (default: 4) |   | redact_user_api_key_info | boolean | If true, redacts information about the user api key from logs [Proxy Logging](logging#redacting-userapikeyinfo) |@@ -809,6 +810,7 @@ router_settings: | LITELLM_ANTHROPIC_BETA_HEADERS_URL | Custom URL for fetching Anthropic beta headers configuration. Default is the GitHub main branch URL | LITELLM_ANTHROPIC_DISABLE_URL_SUFFIX | Disable automatic URL suffix appending for Anthropic API base URLs. When set to `true`, prevents LiteLLM from automatically adding `/v1/messages` or `/v1/complete` to custom Anthropic API endpoints | LITELLM_ASSETS_PATH | Path to directory for UI assets and logos. Used when running with read-only filesystem (e.g., Kubernetes). Default is `/var/lib/litellm/assets` in Docker.+| LITELLM_BEDROCK_REMOVE_TOOL_CUSTOM_FIELD | When `true`, strips the `custom` field (e.g. `custom: {defer_loading: true}`) from tool definitions sent to Bedrock Invoke. **Default is False** (field is preserved so Tool Search works) | LITELLM_BLOG_POSTS_URL | Custom URL for fetching LiteLLM blog posts JSON. Default is the GitHub main branch URL | LITELLM_CLI_JWT_EXPIRATION_HOURS | Expiration time in hours for CLI-generated JWT tokens. Default is 24 hours | LITELLM_CORS_ALLOW_CREDENTIALS | Set to `true` to explicitly allow credentials in CORS responses. When not set, credentials are disabled automatically if `LITELLM_CORS_ORIGINS` is `*` (wildcard) to prevent the browser security misconfiguration of reflecting any origin with credentialslitellm/__init__.pyindex 3acbb49535..3f5c53f685 100644@@ -445,6 +445,14 @@ network_mock: bool = False  # When True, use mock transport — no real network ####### STOP SEQUENCE LIMIT ####### disable_stop_sequence_limit: bool = False  # when True, stop sequence limit is disabled +####### BEDROCK INVOKE TOOL `custom` FIELD #######+# When True, strip the `custom` field (e.g. `custom: {defer_loading: true}` sent by Claude Code)+# from tool definitions before sending to Bedrock Invoke. Off by default - Bedrock accepts it,+# and stripping it disables Tool Search (deferred tools get fully loaded into context).+bedrock_remove_tool_custom_field: bool = (+    os.getenv("LITELLM_BEDROCK_REMOVE_TOOL_CUSTOM_FIELD", "false").lower() == "true"+)+ #### RETRIES #### num_retries: Optional[int] = None  # per model endpoint max_fallbacks: Optional[int] = Nonelitellm/llms/bedrock/chat/invoke_transformations/anthropic_claude3_transformation.pyindex cff415d49e..3c277cc0f9 100644@@ -2,6 +2,7 @@ from typing import TYPE_CHECKING, Any, List, Optional  import httpx +import litellm from litellm.anthropic_beta_headers_manager import filter_and_transform_beta_headers from litellm.litellm_core_utils.prompt_templates.factory import (     convert_to_anthropic_image_obj,@@ -173,8 +174,10 @@ class AmazonAnthropicClaudeConfig(AmazonInvokeConfig, AnthropicConfig):         if "anthropic_version" not in anthropic_request:             anthropic_request["anthropic_version"] = self.anthropic_version -        # Remove `custom` field from tools (Bedrock doesn't support it)-        remove_custom_field_from_tools(anthropic_request)+        # Preserve `custom` field on tools by default (Bedrock accepts it; stripping+        # `custom.defer_loading` disables Tool Search). Opt-in stripping only.+        if litellm.bedrock_remove_tool_custom_field:+            remove_custom_field_from_tools(anthropic_request)         normalize_tool_input_schema_types_for_bedrock_invoke(anthropic_request)         return anthropic_request litellm/llms/bedrock/common_utils.pyindex 52697d752b..fa86fdeb21 100644@@ -54,8 +54,9 @@ def remove_custom_field_from_tools(request_body: dict) -> None:     Remove ``custom`` field from each tool in the request body.      Claude Code (v2.1.69+) sends ``custom: {defer_loading: true}`` on tool-    definitions, which Anthropic's API accepts but Bedrock rejects with-    ``"Extra inputs are not permitted"``.+    definitions. Bedrock used to reject this with ``"Extra inputs are not+    permitted"`` but now accepts it, and stripping it disables Tool Search.+    Only applied when ``litellm.bedrock_remove_tool_custom_field`` is True.      Args:         request_body: The request dictionary to modify in-place.litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.pyindex 31e0e76fd9..d8437d7746 100644@@ -12,6 +12,7 @@ from typing import (  import httpx +import litellm from litellm.anthropic_beta_headers_manager import filter_and_transform_beta_headers from litellm.constants import BEDROCK_MIN_THINKING_BUDGET_TOKENS from litellm.litellm_core_utils.litellm_logging import verbose_logger@@ -504,11 +505,12 @@ class AmazonAnthropicClaudeMessagesConfig(         # Fixes: https://github.com/BerriAI/litellm/issues/22797         anthropic_messages_request.pop("output_config", None) -        # 5a. Remove `custom` field from tools (Bedrock doesn't support it)-        # Claude Code sends `custom: {defer_loading: true}` on tool definitions,-        # which causes Bedrock to reject the request with "Extra inputs are not permitted"+        # 5a. Preserve `custom` field on tools by default - Bedrock accepts+        # `custom: {defer_loading: true}` (sent by Claude Code) and stripping it+        # disables Tool Search. Opt-in stripping for older Bedrock behavior.         # Ref: https://github.com/BerriAI/litellm/issues/22847-        remove_custom_field_from_tools(anthropic_messages_request)+        if litellm.bedrock_remove_tool_custom_field:+            remove_custom_field_from_tools(anthropic_messages_request)         normalize_tool_input_schema_types_for_bedrock_invoke(anthropic_messages_request)         ensure_bedrock_anthropic_messages_tool_names(anthropic_messages_request) tests/test_litellm/llms/bedrock/chat/invoke_transformations/test_bedrock_chat_invoke_transformations_anthropic_claude3_transformation.pyindex 80d6d26e7e..cb6cccf7f2 100644@@ -475,3 +475,48 @@ def test_output_format_removed_from_bedrock_invoke_request():     assert (         "output_format" not in result     ), f"output_format should be removed for Bedrock Invoke, got keys: {result.keys()}"+++def _bedrock_chat_invoke_request_with_custom_defer_loading_tool():+    config = AmazonAnthropicClaudeConfig()+    optional_params = {+        "max_tokens": 100,+        "tools": [+            {+                "name": "mcp__exa__search",+                "description": "Search the web",+                "input_schema": {"type": "object", "properties": {}},+                "custom": {"defer_loading": True},+            }+        ],+    }+    return config.transform_request(+        model="anthropic.claude-sonnet-4-20250514-v1:0",+        messages=[{"role": "user", "content": "hi"}],+        optional_params=optional_params,+        litellm_params={},+        headers={},+    )+++def test_bedrock_chat_invoke_preserves_tool_custom_defer_loading(monkeypatch):+    """+    Bedrock accepts `custom: {defer_loading: true}` on tools; stripping it+    disables Tool Search and loads every deferred tool into context.+    """+    import litellm++    monkeypatch.setattr(litellm, "bedrock_remove_tool_custom_field", False)+    result = _bedrock_chat_invoke_request_with_custom_defer_loading_tool()++    assert result["tools"][0]["custom"] == {"defer_loading": True}+++def test_bedrock_chat_invoke_strips_tool_custom_field_when_opted_in(monkeypatch):+    import litellm++    monkeypatch.setattr(litellm, "bedrock_remove_tool_custom_field", True)+    result = _bedrock_chat_invoke_request_with_custom_defer_loading_tool()++    assert "custom" not in result["tools"][0]+    assert result["tools"][0]["name"] == "mcp__exa__search"tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.pyindex d3a9c94ea5..63b8cb9c05 100644@@ -300,6 +300,62 @@ def test_remove_custom_field_from_tools():     assert request4["tools"] is None  +def _bedrock_invoke_messages_request_with_custom_defer_loading_tool():+    from litellm.types.router import GenericLiteLLMParams++    cfg = AmazonAnthropicClaudeMessagesConfig()+    optional_params = {+        "max_tokens": 128,+        "tools": [+            {+                "name": "mcp__exa__search",+                "description": "Search the web",+                "input_schema": {"type": "object", "properties": {}},+                "custom": {"defer_loading": True},+            },+            {+                "name": "Read",+                "description": "Read a file",+                "input_schema": {"type": "object", "properties": {}},+            },+        ],+        "stream": False,+    }+    return cfg.transform_anthropic_messages_request(+        model="global.anthropic.claude-sonnet-4-6-v1:0",+        messages=[{"role": "user", "content": "hi"}],+        anthropic_messages_optional_request_params=copy.deepcopy(optional_params),+        litellm_params=GenericLiteLLMParams(),+        headers={},+    )+++def test_bedrock_invoke_messages_preserves_tool_custom_defer_loading(monkeypatch):+    """+    Bedrock now accepts `custom: {defer_loading: true}` (sent by Claude Code).+    Stripping it disables Tool Search, so every deferred MCP tool gets loaded+    into context (~20k extra input tokens per request).+    """+    import litellm++    monkeypatch.setattr(litellm, "bedrock_remove_tool_custom_field", False)+    result = _bedrock_invoke_messages_request_with_custom_defer_loading_tool()++    assert result["tools"][0]["custom"] == {"defer_loading": True}+    assert "custom" not in result["tools"][1]+++def test_bedrock_invoke_messages_strips_tool_custom_field_when_opted_in(monkeypatch):+    import litellm++    monkeypatch.setattr(litellm, "bedrock_remove_tool_custom_field", True)+    result = _bedrock_invoke_messages_request_with_custom_defer_loading_tool()++    for tool in result["tools"]:+        assert "custom" not in tool+    assert result["tools"][0]["name"] == "mcp__exa__search"++ def test_normalize_tool_input_schema_types_for_bedrock_invoke():     """     Claude Code sends ``input_schema.type: \"custom\"`` for custom tools. 
Merged fix · PR #+0 −21
litellm/llms/bedrock/common_utils.pyindex 52697d752be..3e462cf73aa 100644@@ -49,27 +49,6 @@ def get_cached_model_info():     return _get_model_info  -def remove_custom_field_from_tools(request_body: dict) -> None:-    """-    Remove ``custom`` field from each tool in the request body.--    Claude Code (v2.1.69+) sends ``custom: {defer_loading: true}`` on tool-    definitions, which Anthropic's API accepts but Bedrock rejects with-    ``"Extra inputs are not permitted"``.--    Args:-        request_body: The request dictionary to modify in-place.--    Ref: https://github.com/BerriAI/litellm/issues/22847-    """-    tools = request_body.get("tools")-    if not tools or not isinstance(tools, list):-        return-    for tool in tools:-        if isinstance(tool, dict):-            tool.pop("custom", None)-- def normalize_json_schema_custom_types_to_object(schema: dict) -> None:     """     In-place: replace JSON Schema ``type: \"custom\"`` with ``\"object\"`` (iterative walk).