Backtestslitellmrun-20261111-220245#25532
WebSocket /v1/responses requires ?model= query param, breaking OpenAI spec compatibility
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
The independent tests show the base tree already accepts a bare ws://.../v1/responses connection and resolves the model from the first response.create frame, so neither patch needs to change the endpoint; the test file in Patch B already references _enforce_responses_ws_first_frame_model_auth and a first-frame model reader, confirming that logic exists. The question is which patch better covers the issue as reported. Patch B adds targeted regression tests for exactly the issue's scenario: both /v1/responses and /responses, flat and nested model placement in the first frame, real master-key websocket auth on the bare URL, and an error frame when no model is supplied anywhere. These would fail if the ?model= query requirement were ever reintroduced, which is the issue's ask. Patch A does not touch the endpoint or the query-param behavior at all; it changes ManagedResponsesWebSocketHandler._update_proxy_request to stop passing litellm_params as a kwarg to aresponses, fixing a provider-side 'Extra inputs are not permitted' rejection that the issue never mentions. That is unrelated scope with some risk (it drops litellm_params.proxy_server_request that other code may read) and its tests verify the unrelated change, not the reported 403. On correctness relative to the issue, tests, and scope, B is the better match.
Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.
Solve
11m 22s
Wall-clock time in the container
Full test suite
No new failures
1,515 passed · 0 skipped · 1,515 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
83vs 56 merged
Changed lines, ours vs merged
The issue
View on GitHubBug
The WebSocket endpoint for /v1/responses requires model as a URL query parameter (fastapi.Query(...)), but the OpenAI WebSocket spec sends model inside the response.create message payload, not as a query param.
This means any spec-compliant client (e.g. Codex CLI) connecting to ws://.../v1/responses without ?model= gets a 403 Forbidden — FastAPI rejects the missing required query param before auth even runs.
Root cause
In litellm/proxy/response_api_endpoints/endpoints.py:
@router.websocket("/v1/responses")
@router.websocket("/responses")
async def responses_websocket_endpoint(
websocket: WebSocket,
model: str = fastapi.Query(
..., description="The model to use for the responses WebSocket session."
),
user_api_key_dict=Depends(user_api_key_auth_websocket),
):
model: str = fastapi.Query(...) makes it a required URL query parameter. When a client connects without it, FastAPI returns 403 at the framework level before the handler or auth logic runs.
Reproduction
import asyncio
from websockets.asyncio.client import connect
async def main():
# This FAILS with 403 (how Codex CLI and spec-compliant clients connect)
async with connect(
"ws://localhost:4000/v1/responses",
additional_headers={"Authorization": "Bearer sk-1234"},
) as ws:
await ws.send('{"type":"response.create","model":"gpt-5.4","input":"say hi"}')
# This WORKS (non-spec workaround)
async with connect(
"ws://localhost:4000/v1/responses?model=gpt-5.4",
additional_headers={"Authorization": "Bearer sk-1234"},
) as ws:
await ws.send('{"type":"response.create","model":"gpt-5.4","input":"say hi"}')
asyncio.run(main())
Expected behavior
The endpoint should accept model from the response.create WebSocket message payload (matching the OpenAI spec), not require it as a URL query parameter. The model query param should either be optional or removed entirely.
Impact
- Codex CLI always connects to bare
ws://.../v1/responsesper the OpenAI spec → gets 403 → retries 5 times (~6-8 seconds) → falls back to HTTP. This adds ~6-8s latency per turn. - Any client following the OpenAI WebSocket spec will hit the same issue.
Environment
- LiteLLM version: tested on both v1.82.3-stable and v1.83.5-nightly (same behavior)
- Codex CLI: @openai/codex@0.118.0
- Running LiteLLM as proxy in router mode with models configured via config.yaml
Related issues
- #22051 — WebSocket mode support for Responses API (covers streaming bugs after connection, not the connection failure itself)
- #18456 — WebSocket auth header issue for
/v1/realtime(different endpoint)
The two patches
tests/test_litellm/proxy/response_api_endpoints/test_endpoints.pyindex d7010de640..d1eac6a1c3 100644@@ -562,6 +562,89 @@ class TestResponsesWSFirstFrameModelAuth: ) +class TestResponsesWSConnectWithoutModelQueryParam:+ """Spec-compliant clients (e.g. Codex CLI) connect to bare /v1/responses and send+ the model inside the first response.create frame; the handshake must not be+ rejected for a missing ?model= query param."""++ @pytest.mark.parametrize("path", ["/v1/responses", "/responses"])+ @pytest.mark.parametrize(+ "first_frame",+ [+ {"type": "response.create", "model": "gpt-5.4", "input": "say hi"},+ {"type": "response.create", "response": {"model": "gpt-5.4", "input": "say hi"}},+ ],+ )+ def test_bare_url_accepts_and_uses_model_from_first_frame(self, path, first_frame):+ from litellm.proxy._types import UserAPIKeyAuth+ from litellm.proxy.auth.user_api_key_auth import user_api_key_auth_websocket++ processor = MagicMock()+ processor.common_processing_pre_call_logic = AsyncMock(side_effect=ValueError("stop after pre-call"))++ app.dependency_overrides[user_api_key_auth_websocket] = lambda: UserAPIKeyAuth(api_key="sk-1234")+ try:+ with (+ patch(+ "litellm.proxy.response_api_endpoints.endpoints._enforce_responses_ws_first_frame_model_auth",+ new_callable=AsyncMock,+ ) as mock_model_auth,+ patch(+ "litellm.proxy.response_api_endpoints.endpoints.ProxyBaseLLMRequestProcessing",+ return_value=processor,+ ) as mock_processor_cls,+ ):+ client = TestClient(app)+ with client.websocket_connect(path, headers={"Authorization": "Bearer sk-1234"}) as ws:+ raw = json.dumps(first_frame)+ ws.send_text(raw)+ error = json.loads(ws.receive_text())+ finally:+ app.dependency_overrides.pop(user_api_key_auth_websocket, None)++ assert error["error"]["message"] == "stop after pre-call"+ data = mock_processor_cls.call_args.kwargs["data"]+ assert data["model"] == "gpt-5.4"+ assert data["first_message"] == raw+ assert mock_model_auth.await_args.kwargs["model"] == "gpt-5.4"+ assert processor.common_processing_pre_call_logic.await_args.kwargs["model"] == "gpt-5.4"++ def test_bare_url_passes_real_websocket_auth_with_master_key(self):+ processor = MagicMock()+ processor.common_processing_pre_call_logic = AsyncMock(side_effect=ValueError("stop after pre-call"))++ with (+ patch("litellm.proxy.proxy_server.master_key", "sk-1234"),+ patch("litellm.proxy.proxy_server.prisma_client", None),+ patch(+ "litellm.proxy.response_api_endpoints.endpoints.ProxyBaseLLMRequestProcessing",+ return_value=processor,+ ) as mock_processor_cls,+ ):+ client = TestClient(app)+ with client.websocket_connect("/v1/responses", headers={"Authorization": "Bearer sk-1234"}) as ws:+ ws.send_text(json.dumps({"type": "response.create", "model": "gpt-5.4", "input": "say hi"}))+ error = json.loads(ws.receive_text())++ assert error["error"]["message"] == "stop after pre-call"+ assert mock_processor_cls.call_args.kwargs["data"]["model"] == "gpt-5.4"++ def test_bare_url_without_model_anywhere_gets_error_frame(self):+ from litellm.proxy._types import UserAPIKeyAuth+ from litellm.proxy.auth.user_api_key_auth import user_api_key_auth_websocket++ app.dependency_overrides[user_api_key_auth_websocket] = lambda: UserAPIKeyAuth(api_key="sk-1234")+ try:+ client = TestClient(app)+ with client.websocket_connect("/v1/responses", headers={"Authorization": "Bearer sk-1234"}) as ws:+ ws.send_text(json.dumps({"type": "response.create", "input": "say hi"}))+ error = json.loads(ws.receive_text())+ finally:+ app.dependency_overrides.pop(user_api_key_auth_websocket, None)++ assert "No model provided" in error["error"]["message"]++ class TestReadWSModelFromFirstFrameErrors: @pytest.mark.asyncio async def test_timeout_closes_without_error_frame(self):
litellm/responses/streaming_iterator.pyindex 8d766cf1cd0..61c93063826 100644@@ -2589,8 +2589,7 @@ class ManagedResponsesWebSocketHandler: if "litellm_metadata" not in call_kwargs: call_kwargs["litellm_metadata"] = {} call_kwargs["litellm_metadata"]["proxy_server_request"] = proxy_server_request- call_kwargs.setdefault("litellm_params", {})- call_kwargs["litellm_params"]["proxy_server_request"] = proxy_server_request+ call_kwargs["proxy_server_request"] = proxy_server_request async def _stream_and_forward(self, model: str, call_kwargs: dict[str, Any]) -> _MutableJsonObject | None: """tests/test_litellm/responses/test_responses_websocket_all_providers.pyindex fe3c4a0640d..0394e260606 100644@@ -669,6 +669,59 @@ class TestChunkTransformation: assert ManagedResponsesWebSocketHandler._input_to_messages({}) == [] +class TestUpdateProxyRequest:+ """Regression tests for ManagedResponsesWebSocketHandler._update_proxy_request.++ The managed WebSocket path calls ``litellm.aresponses(model=..., **call_kwargs)``.+ ``litellm_params`` is not a Responses API request field, so passing it as a+ top-level kwarg leaks it into the provider request body and providers that+ forbid extra inputs (e.g. Anthropic) reject the call with+ ``litellm_params: Extra inputs are not permitted``. The request-tracking data+ must ride along as ``proxy_server_request`` instead, which litellm consumes+ internally and never forwards to the provider.+ """++ def test_does_not_inject_litellm_params_kwarg(self):+ from litellm.responses.streaming_iterator import (+ ManagedResponsesWebSocketHandler,+ )++ call_kwargs = {+ "input": "hello",+ "store": True,+ "litellm_metadata": {+ "proxy_server_request": {"headers": {}, "body": {}},+ },+ }++ ManagedResponsesWebSocketHandler._update_proxy_request(+ call_kwargs, "anthropic/claude-sonnet-4-5"+ )++ assert "litellm_params" not in call_kwargs+ assert call_kwargs["proxy_server_request"]["body"]["model"] == (+ "anthropic/claude-sonnet-4-5"+ )+ assert call_kwargs["proxy_server_request"]["body"]["input"] == "hello"++ def test_proxy_server_request_matches_metadata(self):+ from litellm.responses.streaming_iterator import (+ ManagedResponsesWebSocketHandler,+ )++ call_kwargs = {+ "input": "hi",+ "litellm_metadata": {"proxy_server_request": {"body": {}}},+ }++ ManagedResponsesWebSocketHandler._update_proxy_request(call_kwargs, "gpt-4o")++ assert (+ call_kwargs["proxy_server_request"]+ == call_kwargs["litellm_metadata"]["proxy_server_request"]+ )++ class TestWebSocketEventTypes: """Test that all WebSocket event types are properly handled with dict-based chunks"""