Backtestslitellmrun-20261018-182052#26113
[Bug]: remove_custom_field_from_tools() strips defer_loading — breaks Tool Search on Bedrock, wastes ~20k tokens per request
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Patch A deletes remove_custom_field_from_tools() from common_utils.py but does not touch the two transformation files that import and call it. The independent tests confirm this breaks the codebase: both fail under Patch A with ImportError 'cannot import name remove_custom_field_from_tools', and the chat path surfaces it as an APIConnectionError. Every Bedrock Invoke Claude request would fail, so Patch A is strictly worse than the base commit. Patch B keeps the helper, gates both call sites (chat and messages invoke transformations) behind a new litellm.bedrock_remove_tool_custom_field flag defaulting to False, so defer_loading is preserved by default and Tool Search works, while retaining an opt-in for users on regions where Bedrock still rejects the field. Both independent tests pass on B. B also adds tests on both paths covering preserve-by-default and strip-when-opted-in, updates the docstring, and documents the setting and env var. The config knob is modest extra scope but the issue itself suggested making it conditional/configurable, and it adds no risk. B is clearly better.
Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.
Solve
25m 52s
Wall-clock time in the container
Full test suite
No new failures
5,989 passed · 92 skipped · 6,084 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
133vs 21 merged
Changed lines, ours vs merged
The issue
View on GitHubCheck for existing issues
- I have searched the existing issues and checked that my issue is not a duplicate.
What happened?
PR #22861 (merged 2026-03-05, fixing #22847) added remove_custom_field_from_tools() to strip the custom: {defer_loading: true} field from tool definitions before forwarding to Bedrock. At the time, Bedrock rejected this field with Extra inputs are not permitted.
Bedrock has since updated and now accepts custom.defer_loading. When using Claude Code directly against Bedrock (without LiteLLM), Tool Search works correctly — deferred MCP tools are excluded from the system prompt prefix and only ~1.2% of context is consumed by MCP tool definitions.
However, when routing through LiteLLM → Bedrock, remove_custom_field_from_tools() unconditionally strips the custom field. Bedrock then sees all tools as non-deferred and loads every tool definition into the context. This causes ~20k+ extra input tokens per request for a typical MCP setup (22 tools across 5 MCP servers).
Impact
| Path | MCP tools token cost | Behavior |
|---|---|---|
| Claude Code → Bedrock (direct) | ~12k tokens (1.2% of context) | defer_loading preserved, Tool Search works |
| Claude Code → LiteLLM → Bedrock | ~30k+ tokens | defer_loading stripped, all tools fully loaded |
This is not just a logging/display issue — it's real additional token consumption and cost on every request.
Root cause
remove_custom_field_from_tools() in litellm/llms/bedrock/common_utils.py unconditionally removes the custom field:
litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.pylitellm/llms/bedrock/chat/invoke_transformations/anthropic_claude3_transformation.py
Suggested fix
Remove remove_custom_field_from_tools() entirely, or make it conditional/configurable. Bedrock now accepts this field natively.
Steps to Reproduce
- Configure Claude Code to use LiteLLM proxy pointing to Bedrock
- Add multiple MCP servers (e.g., exa, aws-knowledge, aws-pricing, context7, calculator — 22 tools total)
- Start a conversation with a simple prompt like "hi"
- Compare token usage in LiteLLM logs vs direct Bedrock connection
- LiteLLM path shows ~30k+ tokens for messages; direct Bedrock shows ~2.5k
Environment
- LiteLLM: v1.82.3-stable (Production)
- Claude Code: latest
- Bedrock region: ap-northeast-1
- Model: Claude Opus 4.6 / Sonnet 4.6
Relevant log output
In LiteLLM request logs, the forwarded request to Bedrock contains tool definitions without custom.defer_loading, while the original request from Claude Code includes it.
Are you a LiteLLM Enterprise user?
No
Twitter / LinkedIn details
_No response_
The two patches
docs/my-website/docs/proxy/config_settings.mdindex a886a754f5..0e199e80b0 100644@@ -178,6 +178,7 @@ router_settings: | service_callbacks | array of strings | System health monitoring - Logs redis, postgres failures on specified services (e.g. datadog, prometheus) [Doc Metrics](prometheus) | | turn_off_message_logging | boolean | If true, prevents messages and responses from being logged to callbacks, but request metadata will still be logged. Useful for privacy/compliance when handling sensitive data [Proxy Logging](logging) | | modify_params | boolean | If true, allows modifying the parameters of the request before it is sent to the LLM provider |+| bedrock_remove_tool_custom_field | boolean | If true, strips the `custom` field (e.g. `custom: {defer_loading: true}` sent by Claude Code) from tool definitions sent to Bedrock Invoke. Default false - stripping it disables Tool Search. | | enable_preview_features | boolean | If true, enables preview features - e.g. Azure O1 Models with streaming support.| | LITELLM_DISABLE_STOP_SEQUENCE_LIMIT | Disable validation for stop sequence limit (default: 4) | | redact_user_api_key_info | boolean | If true, redacts information about the user api key from logs [Proxy Logging](logging#redacting-userapikeyinfo) |@@ -809,6 +810,7 @@ router_settings: | LITELLM_ANTHROPIC_BETA_HEADERS_URL | Custom URL for fetching Anthropic beta headers configuration. Default is the GitHub main branch URL | LITELLM_ANTHROPIC_DISABLE_URL_SUFFIX | Disable automatic URL suffix appending for Anthropic API base URLs. When set to `true`, prevents LiteLLM from automatically adding `/v1/messages` or `/v1/complete` to custom Anthropic API endpoints | LITELLM_ASSETS_PATH | Path to directory for UI assets and logos. Used when running with read-only filesystem (e.g., Kubernetes). Default is `/var/lib/litellm/assets` in Docker.+| LITELLM_BEDROCK_REMOVE_TOOL_CUSTOM_FIELD | When `true`, strips the `custom` field (e.g. `custom: {defer_loading: true}`) from tool definitions sent to Bedrock Invoke. **Default is False** (field is preserved so Tool Search works) | LITELLM_BLOG_POSTS_URL | Custom URL for fetching LiteLLM blog posts JSON. Default is the GitHub main branch URL | LITELLM_CLI_JWT_EXPIRATION_HOURS | Expiration time in hours for CLI-generated JWT tokens. Default is 24 hours | LITELLM_CORS_ALLOW_CREDENTIALS | Set to `true` to explicitly allow credentials in CORS responses. When not set, credentials are disabled automatically if `LITELLM_CORS_ORIGINS` is `*` (wildcard) to prevent the browser security misconfiguration of reflecting any origin with credentialslitellm/__init__.pyindex 3acbb49535..3f5c53f685 100644@@ -445,6 +445,14 @@ network_mock: bool = False # When True, use mock transport — no real network ####### STOP SEQUENCE LIMIT ####### disable_stop_sequence_limit: bool = False # when True, stop sequence limit is disabled +####### BEDROCK INVOKE TOOL `custom` FIELD #######+# When True, strip the `custom` field (e.g. `custom: {defer_loading: true}` sent by Claude Code)+# from tool definitions before sending to Bedrock Invoke. Off by default - Bedrock accepts it,+# and stripping it disables Tool Search (deferred tools get fully loaded into context).+bedrock_remove_tool_custom_field: bool = (+ os.getenv("LITELLM_BEDROCK_REMOVE_TOOL_CUSTOM_FIELD", "false").lower() == "true"+)+ #### RETRIES #### num_retries: Optional[int] = None # per model endpoint max_fallbacks: Optional[int] = Nonelitellm/llms/bedrock/chat/invoke_transformations/anthropic_claude3_transformation.pyindex cff415d49e..3c277cc0f9 100644@@ -2,6 +2,7 @@ from typing import TYPE_CHECKING, Any, List, Optional import httpx +import litellm from litellm.anthropic_beta_headers_manager import filter_and_transform_beta_headers from litellm.litellm_core_utils.prompt_templates.factory import ( convert_to_anthropic_image_obj,@@ -173,8 +174,10 @@ class AmazonAnthropicClaudeConfig(AmazonInvokeConfig, AnthropicConfig): if "anthropic_version" not in anthropic_request: anthropic_request["anthropic_version"] = self.anthropic_version - # Remove `custom` field from tools (Bedrock doesn't support it)- remove_custom_field_from_tools(anthropic_request)+ # Preserve `custom` field on tools by default (Bedrock accepts it; stripping+ # `custom.defer_loading` disables Tool Search). Opt-in stripping only.+ if litellm.bedrock_remove_tool_custom_field:+ remove_custom_field_from_tools(anthropic_request) normalize_tool_input_schema_types_for_bedrock_invoke(anthropic_request) return anthropic_request litellm/llms/bedrock/common_utils.pyindex 52697d752b..fa86fdeb21 100644@@ -54,8 +54,9 @@ def remove_custom_field_from_tools(request_body: dict) -> None: Remove ``custom`` field from each tool in the request body. Claude Code (v2.1.69+) sends ``custom: {defer_loading: true}`` on tool- definitions, which Anthropic's API accepts but Bedrock rejects with- ``"Extra inputs are not permitted"``.+ definitions. Bedrock used to reject this with ``"Extra inputs are not+ permitted"`` but now accepts it, and stripping it disables Tool Search.+ Only applied when ``litellm.bedrock_remove_tool_custom_field`` is True. Args: request_body: The request dictionary to modify in-place.litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.pyindex 31e0e76fd9..d8437d7746 100644@@ -12,6 +12,7 @@ from typing import ( import httpx +import litellm from litellm.anthropic_beta_headers_manager import filter_and_transform_beta_headers from litellm.constants import BEDROCK_MIN_THINKING_BUDGET_TOKENS from litellm.litellm_core_utils.litellm_logging import verbose_logger@@ -504,11 +505,12 @@ class AmazonAnthropicClaudeMessagesConfig( # Fixes: https://github.com/BerriAI/litellm/issues/22797 anthropic_messages_request.pop("output_config", None) - # 5a. Remove `custom` field from tools (Bedrock doesn't support it)- # Claude Code sends `custom: {defer_loading: true}` on tool definitions,- # which causes Bedrock to reject the request with "Extra inputs are not permitted"+ # 5a. Preserve `custom` field on tools by default - Bedrock accepts+ # `custom: {defer_loading: true}` (sent by Claude Code) and stripping it+ # disables Tool Search. Opt-in stripping for older Bedrock behavior. # Ref: https://github.com/BerriAI/litellm/issues/22847- remove_custom_field_from_tools(anthropic_messages_request)+ if litellm.bedrock_remove_tool_custom_field:+ remove_custom_field_from_tools(anthropic_messages_request) normalize_tool_input_schema_types_for_bedrock_invoke(anthropic_messages_request) ensure_bedrock_anthropic_messages_tool_names(anthropic_messages_request) tests/test_litellm/llms/bedrock/chat/invoke_transformations/test_bedrock_chat_invoke_transformations_anthropic_claude3_transformation.pyindex 80d6d26e7e..cb6cccf7f2 100644@@ -475,3 +475,48 @@ def test_output_format_removed_from_bedrock_invoke_request(): assert ( "output_format" not in result ), f"output_format should be removed for Bedrock Invoke, got keys: {result.keys()}"+++def _bedrock_chat_invoke_request_with_custom_defer_loading_tool():+ config = AmazonAnthropicClaudeConfig()+ optional_params = {+ "max_tokens": 100,+ "tools": [+ {+ "name": "mcp__exa__search",+ "description": "Search the web",+ "input_schema": {"type": "object", "properties": {}},+ "custom": {"defer_loading": True},+ }+ ],+ }+ return config.transform_request(+ model="anthropic.claude-sonnet-4-20250514-v1:0",+ messages=[{"role": "user", "content": "hi"}],+ optional_params=optional_params,+ litellm_params={},+ headers={},+ )+++def test_bedrock_chat_invoke_preserves_tool_custom_defer_loading(monkeypatch):+ """+ Bedrock accepts `custom: {defer_loading: true}` on tools; stripping it+ disables Tool Search and loads every deferred tool into context.+ """+ import litellm++ monkeypatch.setattr(litellm, "bedrock_remove_tool_custom_field", False)+ result = _bedrock_chat_invoke_request_with_custom_defer_loading_tool()++ assert result["tools"][0]["custom"] == {"defer_loading": True}+++def test_bedrock_chat_invoke_strips_tool_custom_field_when_opted_in(monkeypatch):+ import litellm++ monkeypatch.setattr(litellm, "bedrock_remove_tool_custom_field", True)+ result = _bedrock_chat_invoke_request_with_custom_defer_loading_tool()++ assert "custom" not in result["tools"][0]+ assert result["tools"][0]["name"] == "mcp__exa__search"tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.pyindex d3a9c94ea5..63b8cb9c05 100644@@ -300,6 +300,62 @@ def test_remove_custom_field_from_tools(): assert request4["tools"] is None +def _bedrock_invoke_messages_request_with_custom_defer_loading_tool():+ from litellm.types.router import GenericLiteLLMParams++ cfg = AmazonAnthropicClaudeMessagesConfig()+ optional_params = {+ "max_tokens": 128,+ "tools": [+ {+ "name": "mcp__exa__search",+ "description": "Search the web",+ "input_schema": {"type": "object", "properties": {}},+ "custom": {"defer_loading": True},+ },+ {+ "name": "Read",+ "description": "Read a file",+ "input_schema": {"type": "object", "properties": {}},+ },+ ],+ "stream": False,+ }+ return cfg.transform_anthropic_messages_request(+ model="global.anthropic.claude-sonnet-4-6-v1:0",+ messages=[{"role": "user", "content": "hi"}],+ anthropic_messages_optional_request_params=copy.deepcopy(optional_params),+ litellm_params=GenericLiteLLMParams(),+ headers={},+ )+++def test_bedrock_invoke_messages_preserves_tool_custom_defer_loading(monkeypatch):+ """+ Bedrock now accepts `custom: {defer_loading: true}` (sent by Claude Code).+ Stripping it disables Tool Search, so every deferred MCP tool gets loaded+ into context (~20k extra input tokens per request).+ """+ import litellm++ monkeypatch.setattr(litellm, "bedrock_remove_tool_custom_field", False)+ result = _bedrock_invoke_messages_request_with_custom_defer_loading_tool()++ assert result["tools"][0]["custom"] == {"defer_loading": True}+ assert "custom" not in result["tools"][1]+++def test_bedrock_invoke_messages_strips_tool_custom_field_when_opted_in(monkeypatch):+ import litellm++ monkeypatch.setattr(litellm, "bedrock_remove_tool_custom_field", True)+ result = _bedrock_invoke_messages_request_with_custom_defer_loading_tool()++ for tool in result["tools"]:+ assert "custom" not in tool+ assert result["tools"][0]["name"] == "mcp__exa__search"++ def test_normalize_tool_input_schema_types_for_bedrock_invoke(): """ Claude Code sends ``input_schema.type: \"custom\"`` for custom tools.
litellm/llms/bedrock/common_utils.pyindex 52697d752be..3e462cf73aa 100644@@ -49,27 +49,6 @@ def get_cached_model_info(): return _get_model_info -def remove_custom_field_from_tools(request_body: dict) -> None:- """- Remove ``custom`` field from each tool in the request body.-- Claude Code (v2.1.69+) sends ``custom: {defer_loading: true}`` on tool- definitions, which Anthropic's API accepts but Bedrock rejects with- ``"Extra inputs are not permitted"``.-- Args:- request_body: The request dictionary to modify in-place.-- Ref: https://github.com/BerriAI/litellm/issues/22847- """- tools = request_body.get("tools")- if not tools or not isinstance(tools, list):- return- for tool in tools:- if isinstance(tool, dict):- tool.pop("custom", None)-- def normalize_json_schema_custom_types_to_object(schema: dict) -> None: """ In-place: replace JSON Schema ``type: \"custom\"`` with ``\"object\"`` (iterative walk).