Backtestslitellmrun-20261111-220245#46999
[Bug]: /tag/update 500s "BudgetNewRequest() got multiple values for keyword argument 'budget_id'" on any tag that has a budget
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Both patches apply the identical root-cause fix the issue suggests: drop budget_id from the LiteLLM_BudgetTable field set so it is no longer copied into _budget_data and collides with the explicit budget_id kwarg in the existing-budget branch. Independent tests confirm both resolve every scenario, including the access-group path. The difference is in the follow-through. Patch B also changes the update branch so that when a different budget_id is sent alongside budget settings, those settings are written to the budget the entity is being re-pointed at rather than to the budget it is leaving. With Patch A, that combination (previously unreachable because it 500'd) would mutate the old budget and then re-point, which silently edits a budget the tag no longer uses; the issue's expectation that a different budget_id re-points the tag makes B's semantics the coherent one, and since the path never worked before there is no regression risk for existing callers. On tests, B is stronger: it covers the helper for same/different budget_id, exercises /tag/update end-to-end with a four-case parametrization asserting both the tag-row re-point and which budget row receives the settings, and adds two access-group tests for the second affected endpoint the issue calls out. Patch A adds one helper unit test and an integration test that depends on a tests/integration/_support/client module whose existence cannot be confirmed from the diff, and does not cover the re-point or access-group cases. Scope-wise B's extra behavioral change is small, well-commented, and tested, so it does not obscure the fix. B wins on tests and on handling the edge the issue's expected behavior implies.
Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.
Solve
12m 13s
Wall-clock time in the container
Full test suite
Not run
The harness did not reach the test stage
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
184vs 51 merged
Changed lines, ours vs merged
The issue
View on GitHubDescription
Please note this has been found and written up by AI but hand checked by the author before sharing.
Description
What happensPOST /tag/update returns 500 whenever the request carries budget_id and the tag already has a budget, even when it is the tag's own budget id, unchanged. Nothing is updated:
500 {"detail":"litellm.proxy._types.BudgetNewRequest() got multiple values for keyword argument 'budget_id'"}
The Terraform provider's litellm_tag sends budget_id on every update once it is in state, so a litellm_tag with budget_id = litellm_budget.x.budget_id can't be updated at all, whichever field changed. (A tag with no budget updates fine.)
Root cause (v1.105.1; unchanged on main today)handle_budget_for_entity (litellm/proxy/management_helpers/utils.py) collects the request's budget fields by iterating LiteLLM_BudgetTable.model_fields (utils.py:211-222). LiteLLM_BudgetTable declares budget_id (litellm/models/budget.py:24), so a request that sends budget_id puts it in _budget_data, and the "existing budget" branch then calls
BudgetNewRequest(budget_id=existing_budget_id, **_budget_data) # utils.py:255
which raises TypeError. update_tag's except Exception (tag_management_endpoints.py:481-483) turns it into the 500. A side effect is that lines 259-261 ("a different budget_id is explicitly provided, use that instead") can never run for a tag that already has a budget, so a tag can't be moved to another budget either.
Expected/tag/update with the tag's current budget_id (plus any other fields) succeeds. With a different budget_id, the tag is re-pointed to that budget.
Suggested fix
In handle_budget_for_entity (litellm/proxy/management_helpers/utils.py:180), the request's budget_id is read twice:
- as a budget setting:
budget_params(:211) is every field ofLiteLLM_BudgetTable, which includesbudget_id, so the request'sbudget_idis copied into_budget_data(:214-222) alongsidemax_budget,tpm_limitand the rest; - as "which budget this tag should point at":
data_budget_id = getattr(data, "budget_id", None)(:225). The create branch returns it (:228-230), and the update branch re-points the tag when it differs from the current one (:260-261).
Only the second is wanted. The first is what breaks BudgetNewRequest(budget_id=existing_budget_id, **_budget_data) (:255). So exclude budget_id from budget_params, and leave the data_budget_id handling to do its job:
- budget_params: Final = LiteLLM_BudgetTable.model_fields.keys()
+ # budget_id selects the budget (data_budget_id below); it is not a budget setting
+ budget_params: Final = LiteLLM_BudgetTable.model_fields.keys() - {"budget_id"}
With that change, step 2 of the repro updates the description and keeps the budget. A budget_id that names a different budget re-points the tag (:260-261) instead of 500ing.
Related, from the same helper (please confirm which is intended)
- Licence gating depends on whether the tag already has a budget. When there is no budget yet, the helper writes the row directly (
utils.py:234-243) withoutvalidate_model_max_budget(key_management_endpoints.py:7809, which holds the Enterprise check and the shape validation). When a budget exists, the update goes throughupdate_budget, which does call it (budget_management_endpoints.py:168-176). So without a licence,model_max_budgetis accepted, unvalidated, by/tag/newand by/tag/updateon a tag with no budget. It is refused ("You must have an enterprise license to set model_max_budget") by/tag/updateonce the tag has a budget, and by/budget/newand/budget/update. A tag created withmodel_max_budgettherefore can't have it changed later without a licence. PUT /access_group/{access_group}/budgetuses the same helper (model_access_group_management_endpoints.py:1173), andAccessGroupBudgetRequestalso hasbudget_id. From reading the code, the sameTypeErrorshould hit it once the group has a budget row. Not reproduced.
Config
# docker-compose.yml: the proxy with no config.yaml, just Postgres
services:
postgres:
image: postgres:16-alpine
environment: {POSTGRES_DB: litellm, POSTGRES_USER: litellm, POSTGRES_PASSWORD: litellm}
healthcheck: {test: ["CMD", "pg_isready", "-U", "litellm"], interval: 3s, retries: 20}
litellm:
image: ghcr.io/berriai/litellm:v1.105.1
command: ["--port", "4000"]
depends_on: {postgres: {condition: service_healthy}}
environment:
LITELLM_MASTER_KEY: sk-master-key
DATABASE_URL: postgresql://litellm:litellm@postgres:5432/litellm
ports: ["4000:4000"]
LiteLLM Version
v1.105.1
Steps to Repro
# the docker-compose values above; change them to point at another proxy
export LITELLM=http://localhost:4000 MASTER_KEY=sk-master-key
H=(-H "Authorization: Bearer $MASTER_KEY" -H 'Content-Type: application/json') # bash or zsh
# 1. a budget, and a tag that uses it
curl -s $LITELLM/budget/new "${H[@]}" -d '{"budget_id":"b-repro","max_budget":50}'
# -> {"budget_id":"b-repro","max_budget":50.0,…}
curl -s $LITELLM/tag/new "${H[@]}" -d '{"name":"t-repro","budget_id":"b-repro","description":"before"}'
# -> {"message":"Tag t-repro created successfully",…}
# 2. change only the description, re-sending the same budget_id
curl -s -w '\n%{http_code}\n' $LITELLM/tag/update "${H[@]}" \
-d '{"name":"t-repro","budget_id":"b-repro","description":"after"}'
# -> {"detail":"litellm.proxy._types.BudgetNewRequest() got multiple values for keyword argument 'budget_id'"}
# 500
# 3. the same update without budget_id succeeds
curl -s -w '\n%{http_code}\n' $LITELLM/tag/update "${H[@]}" -d '{"name":"t-repro","description":"after"}'
# -> {"message":"Tag t-repro updated successfully",…,"description":"after",…}
# 200
### Which part of LiteLLM is this about?
Budgets and rate limits: budgets, tpm/rpm, 429s
### How are you deploying?
Helm chart, componentized (recommended)
The two patches
litellm/proxy/management_helpers/utils.pyindex ea19befccc..89b6042907 100644@@ -216,8 +216,9 @@ async def handle_budget_for_entity( update_budget, ) - # Get all budget field names- budget_params: Final = LiteLLM_BudgetTable.model_fields.keys()+ # Get all budget setting field names. budget_id selects which budget the+ # entity points at (data_budget_id below); it is not a budget setting.+ budget_params: Final = LiteLLM_BudgetTable.model_fields.keys() - {"budget_id"} # Extract budget fields from data _json_data: Final = _BUDGET_DATA_MAPPING.validate_python(@@ -259,10 +260,11 @@ async def handle_budget_for_entity( # Case 2: Updating existing entity - has existing budget_id else:- # If budget fields are provided, update the existing budget+ # If budget fields are provided, update the budget the entity will point at+ # (an explicitly provided budget_id, else the existing one), never the one it leaves if _budget_data: await update_budget(- budget_obj=BudgetNewRequest(budget_id=existing_budget_id, **_budget_data),+ budget_obj=BudgetNewRequest(budget_id=data_budget_id or existing_budget_id, **_budget_data), user_api_key_dict=user_api_key_dict, ) tests/unit/proxy/management_endpoints/test_access_group_management.pyindex d79af0af51..39c899b142 100644@@ -1027,6 +1027,66 @@ async def test_put_access_group_budget_links_an_existing_budget_without_creating assert prisma.access_group_budget_table.rows["prod-models"].budget_id == "shared-budget" +@pytest.mark.asyncio+async def test_put_access_group_budget_with_its_own_budget_id_updates_that_budget():+ """Re-sending the group's current budget_id next to a budget setting must update the linked+ budget, not raise "BudgetNewRequest() got multiple values for keyword argument 'budget_id'"."""+ from litellm.proxy.management_endpoints.model_access_group_management_endpoints import (+ set_access_group_budget,+ )+ from litellm.types.proxy.management_endpoints.model_management_endpoints import (+ AccessGroupBudgetRequest,+ )++ prisma = _FakePrismaClient([], deployments=[_deployment()])+ _seed_budget(prisma, "prod-models", budget_id="budget-seed", max_budget=10.0)+ cache = _FakeAuthCache()++ with _proxy(prisma):+ response = await set_access_group_budget(+ access_group="prod-models",+ data=AccessGroupBudgetRequest(budget_id="budget-seed", max_budget=40.0),+ user_api_key_dict=_admin(),+ auth_cache=cache,+ )++ assert response.budget is not None+ assert response.budget.budget_id == "budget-seed"+ assert response.budget.max_budget == 40.0+ assert prisma.budget_table.create_calls == []+ assert prisma.access_group_budget_table.rows["prod-models"].budget_id == "budget-seed"+++@pytest.mark.asyncio+async def test_put_access_group_budget_with_another_budget_id_repoints_a_budgeted_group():+ from litellm.proxy.management_endpoints.model_access_group_management_endpoints import (+ set_access_group_budget,+ )+ from litellm.types.proxy.management_endpoints.model_management_endpoints import (+ AccessGroupBudgetRequest,+ )++ prisma = _FakePrismaClient([], deployments=[_deployment()])+ _seed_budget(prisma, "prod-models", budget_id="budget-seed", max_budget=10.0)+ prisma.budget_table.rows["shared-budget"] = _FakeBudgetRow(budget_id="shared-budget", max_budget=7.0)+ cache = _FakeAuthCache()++ with _proxy(prisma):+ response = await set_access_group_budget(+ access_group="prod-models",+ data=AccessGroupBudgetRequest(budget_id="shared-budget"),+ user_api_key_dict=_admin(),+ auth_cache=cache,+ )++ assert response.budget is not None+ assert response.budget.budget_id == "shared-budget"+ assert response.budget.max_budget == 7.0+ assert prisma.budget_table.create_calls == []+ assert prisma.budget_table.rows["budget-seed"].max_budget == 10.0+ assert prisma.access_group_budget_table.rows["prod-models"].budget_id == "shared-budget"++ @pytest.mark.asyncio async def test_put_access_group_budget_rejects_an_empty_body(): """An empty PUT would register the group as budgeted while enforcing nothing."""tests/unit/proxy/management_endpoints/test_tag_management_endpoints.pyindex 531bf98b60..78cba0985f 100644@@ -421,6 +421,87 @@ async def test_update_tag_clears_or_sets_only_provided_budget_fields( } +@pytest.mark.parametrize(+ ("request_budget_fields", "expected_tag_budget_update", "expected_budget_update"),+ cast(+ Sequence[tuple[Mapping[str, object], Mapping[str, object], Mapping[str, object] | None]],+ (+ ({"budget_id": "budget-1"}, {}, None),+ ({"budget_id": "budget-1", "max_budget": 75.0}, {}, {"max_budget": 75.0}),+ ({"budget_id": "budget-2"}, {"budget_id": "budget-2"}, None),+ ({"budget_id": "budget-2", "max_budget": 75.0}, {"budget_id": "budget-2"}, {"max_budget": 75.0}),+ ),+ ),+)+def test_update_tag_with_budget_id_on_a_budgeted_tag_succeeds(+ request_budget_fields: Mapping[str, object],+ expected_tag_budget_update: Mapping[str, object],+ expected_budget_update: Mapping[str, object] | None,+) -> None:+ """/tag/update re-sending the tag's budget_id (as the Terraform provider does) must not 500 with+ "BudgetNewRequest() got multiple values for keyword argument 'budget_id'"; another budget_id re-points the tag."""+ from litellm.proxy.auth.user_api_key_auth import user_api_key_auth++ updated_tag: Final = SimpleNamespace(+ tag_name="budget-tag",+ description="after",+ models=[],+ created_at=datetime(2024, 1, 1),+ updated_at=datetime(2024, 1, 1),+ created_by="admin",+ )+ update_tag_row: Final = AsyncMock(return_value=updated_tag)+ update_budget: Final = AsyncMock()+ mock_prisma: Final = SimpleNamespace(+ db=SimpleNamespace(+ litellm_tagtable=SimpleNamespace(+ find_unique=AsyncMock(return_value=SimpleNamespace(budget_id="budget-1")),+ update=update_tag_row,+ ),+ litellm_proxymodeltable=SimpleNamespace(find_many=AsyncMock(return_value=[])),+ litellm_budgettable=SimpleNamespace(update=update_budget),+ )+ )+ app.dependency_overrides[user_api_key_auth] = lambda: UserAPIKeyAuth(+ user_id="admin", user_role=LitellmUserRoles.PROXY_ADMIN+ )+ try:+ with (+ patch( # test-quality-ok: endpoint resolves the fake database through proxy_server+ "litellm.proxy.proxy_server.prisma_client", mock_prisma+ ),+ patch( # test-quality-ok: endpoint reads the audit actor from proxy_server+ "litellm.proxy.proxy_server.litellm_proxy_admin_name", "admin"+ ),+ patch( # test-quality-ok: cache invalidation is outside this budget contract+ "litellm.proxy.management_endpoints.tag_management_endpoints._evict_tag_cache_keys", new=AsyncMock()+ ),+ ):+ response = client.post(+ "/tag/update",+ json={"name": "budget-tag", "description": "after", **request_budget_fields},+ headers={"Authorization": "Bearer sk-9876"},+ )+ finally:+ app.dependency_overrides.clear()++ assert response.status_code == 200, response.text+ assert response.json()["tag"]["description"] == "after"+ tag_update_args: Final = update_tag_row.await_args+ assert tag_update_args is not None+ tag_update_data: Final = cast(Mapping[str, object], tag_update_args.kwargs["data"])+ assert {k: v for k, v in tag_update_data.items() if k == "budget_id"} == expected_tag_budget_update+ if expected_budget_update is None:+ update_budget.assert_not_awaited()+ else:+ budget_update_args = update_budget.await_args+ assert budget_update_args is not None+ # budget settings land on the budget the tag points at after the update+ assert budget_update_args.kwargs["where"] == {"budget_id": request_budget_fields["budget_id"]}+ budget_data = cast(Mapping[str, object], budget_update_args.kwargs["data"])+ assert {k: v for k, v in budget_data.items() if k in expected_budget_update} == expected_budget_update++ @pytest.mark.asyncio async def test_update_tag_explicit_null_clears_budget_duration(): from datetime import datetimetests/unit/proxy/management_helpers/test_management_helpers_utils.pyindex 644f0656c7..8116477024 100644@@ -309,6 +309,39 @@ async def test_handle_budget_for_entity_updates_only_cleared_or_supplied_fields( } +@pytest.mark.parametrize(+ ("request_budget_id", "expected_budget_id"),+ (("budget-1", "budget-1"), ("budget-2", "budget-2")),+)+@pytest.mark.asyncio+async def test_handle_budget_for_entity_treats_budget_id_as_a_link_not_a_budget_setting(+ request_budget_id: str, expected_budget_id: str+) -> None:+ from litellm.proxy.management_helpers.utils import handle_budget_for_entity+ from litellm.types.tag_management import TagUpdateRequest++ budget_update: Final = AsyncMock()+ prisma_client: Final = cast(+ PrismaClient,+ SimpleNamespace(db=SimpleNamespace(litellm_budgettable=SimpleNamespace(update=budget_update))),+ )++ with (+ patch("litellm.proxy.proxy_server.prisma_client", prisma_client),+ patch("litellm.proxy.proxy_server.litellm_proxy_admin_name", "admin"),+ ):+ budget_id = await handle_budget_for_entity(+ data=TagUpdateRequest(name="budget-tag", budget_id=request_budget_id),+ existing_budget_id="budget-1",+ user_api_key_dict=UserAPIKeyAuth(user_id="admin"),+ prisma_client=prisma_client,+ litellm_proxy_admin_name="admin",+ )++ assert budget_id == expected_budget_id+ budget_update.assert_not_awaited()++ @pytest.mark.asyncio async def test_add_new_member_no_budget_when_default_budget_row_is_missing(): from litellm.proxy._types import LitellmUserRoles
litellm/proxy/management_helpers/utils.pyindex ea19befccc6..1c4eada67fb 100644@@ -217,7 +217,7 @@ async def handle_budget_for_entity( ) # Get all budget field names- budget_params: Final = LiteLLM_BudgetTable.model_fields.keys()+ budget_params: Final = LiteLLM_BudgetTable.model_fields.keys() - {"budget_id"} # Extract budget fields from data _json_data: Final = _BUDGET_DATA_MAPPING.validate_python(tests/integration/management/test_tag_budget_update.pynew file mode 100644index 00000000000..c8ce1bdc8f4@@ -0,0 +1,20 @@+from typing import Final+from uuid import uuid4++from tests.integration._support.client import Gateway, object_value, string_value+++def test_tag_budget_can_be_updated_with_its_existing_budget_id(gateway: Gateway) -> None:+ name: Final = f"integration-tag-{uuid4().hex}"+ gateway.post("/tag/new", {"name": name, "max_budget": 10})+ with gateway.scenario() as scenario:+ scenario.cleanups.callback(gateway.post, "/tag/delete", {"name": name})+ before: Final = object_value(gateway.post("/tag/info", {"names": [name]})[name])+ budget_id: Final = string_value(object_value(before["litellm_budget_table"])["budget_id"])+ scenario.cleanups.callback(gateway.post, "/budget/delete", {"id": budget_id})+ response: Final = gateway.request(+ "POST", "/tag/update", {"name": name, "budget_id": budget_id, "max_budget": 20}+ )+ assert response.status_code == 200, response.text+ after: Final = object_value(gateway.post("/tag/info", {"names": [name]})[name])+ assert object_value(after["litellm_budget_table"])["max_budget"] == 20, aftertests/unit/proxy/management_helpers/test_management_helpers_utils.pyindex 644f0656c7a..640e17aa58e 100644@@ -249,6 +249,35 @@ async def test_add_new_member_links_default_team_budget_id(): assert create_data["budget_id"] == test_default_budget_id +@pytest.mark.asyncio+async def test_handle_budget_for_entity_updates_with_existing_budget_id_in_request() -> None:+ from litellm.proxy.management_helpers.utils import handle_budget_for_entity+ from litellm.types.tag_management import TagUpdateRequest++ budget_update: Final = AsyncMock()+ prisma_client: Final = cast(+ PrismaClient,+ SimpleNamespace(db=SimpleNamespace(litellm_budgettable=SimpleNamespace(update=budget_update))),+ )+ with (+ patch("litellm.proxy.proxy_server.prisma_client", prisma_client),+ patch("litellm.proxy.proxy_server.litellm_proxy_admin_name", "admin"),+ ):+ budget_id: Final = await handle_budget_for_entity(+ data=TagUpdateRequest(name="budget-tag", budget_id="budget-1", max_budget=20),+ existing_budget_id="budget-1",+ user_api_key_dict=UserAPIKeyAuth(user_id="admin"),+ prisma_client=prisma_client,+ litellm_proxy_admin_name="admin",+ )++ assert budget_id == "budget-1"+ budget_update.assert_awaited_once_with(+ where={"budget_id": "budget-1"},+ data={"budget_id": "budget-1", "max_budget": 20.0, "updated_by": "admin"},+ )++ @pytest.mark.parametrize( ("request_fields", "cleared_budget_fields", "should_update", "expected_max_budget"), cast(