Backtestslitellmrun-20261111-220245#25532

Issue #25532 Merged fix PR #33101 Run 11 Nov 2026 · run-20261111-220245 BerriAI/litellm

WebSocket /v1/responses requires ?model= query param, breaking OpenAI spec compatibility

Better

Our patch was judged better than the merged fix.

In this judgment, Patch B = our agent, Patch A = the merged fix.

The independent tests show the base tree already accepts a bare ws://.../v1/responses connection and resolves the model from the first response.create frame, so neither patch needs to change the endpoint; the test file in Patch B already references _enforce_responses_ws_first_frame_model_auth and a first-frame model reader, confirming that logic exists. The question is which patch better covers the issue as reported. Patch B adds targeted regression tests for exactly the issue's scenario: both /v1/responses and /responses, flat and nested model placement in the first frame, real master-key websocket auth on the bare URL, and an error frame when no model is supplied anywhere. These would fail if the ?model= query requirement were ever reintroduced, which is the issue's ask. Patch A does not touch the endpoint or the query-param behavior at all; it changes ManagedResponsesWebSocketHandler._update_proxy_request to stop passing litellm_params as a kwarg to aresponses, fixing a provider-side 'Extra inputs are not permitted' rejection that the issue never mentions. That is unrelated scope with some risk (it drops litellm_params.proxy_server_request that other code may read) and its tests verify the unrelated change, not the reported 403. On correctness relative to the issue, tests, and scope, B is the better match.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

11m 22s

Wall-clock time in the container

Full test suite

No new failures

1,515 passed · 0 skipped · 1,515 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

83vs 56 merged

Changed lines, ours vs merged

Bug

The WebSocket endpoint for /v1/responses requires model as a URL query parameter (fastapi.Query(...)), but the OpenAI WebSocket spec sends model inside the response.create message payload, not as a query param.

This means any spec-compliant client (e.g. Codex CLI) connecting to ws://.../v1/responses without ?model= gets a 403 Forbidden — FastAPI rejects the missing required query param before auth even runs.

Root cause

In litellm/proxy/response_api_endpoints/endpoints.py:

@router.websocket("/v1/responses")
@router.websocket("/responses")
async def responses_websocket_endpoint(
    websocket: WebSocket,
    model: str = fastapi.Query(
        ..., description="The model to use for the responses WebSocket session."
    ),
    user_api_key_dict=Depends(user_api_key_auth_websocket),
):

model: str = fastapi.Query(...) makes it a required URL query parameter. When a client connects without it, FastAPI returns 403 at the framework level before the handler or auth logic runs.

Reproduction

import asyncio
from websockets.asyncio.client import connect

async def main():
    # This FAILS with 403 (how Codex CLI and spec-compliant clients connect)
    async with connect(
        "ws://localhost:4000/v1/responses",
        additional_headers={"Authorization": "Bearer sk-1234"},
    ) as ws:
        await ws.send('{"type":"response.create","model":"gpt-5.4","input":"say hi"}')

    # This WORKS (non-spec workaround)
    async with connect(
        "ws://localhost:4000/v1/responses?model=gpt-5.4",
        additional_headers={"Authorization": "Bearer sk-1234"},
    ) as ws:
        await ws.send('{"type":"response.create","model":"gpt-5.4","input":"say hi"}')

asyncio.run(main())

Expected behavior

The endpoint should accept model from the response.create WebSocket message payload (matching the OpenAI spec), not require it as a URL query parameter. The model query param should either be optional or removed entirely.

Impact

  • Codex CLI always connects to bare ws://.../v1/responses per the OpenAI spec → gets 403 → retries 5 times (~6-8 seconds) → falls back to HTTP. This adds ~6-8s latency per turn.
  • Any client following the OpenAI WebSocket spec will hit the same issue.

Environment

  • LiteLLM version: tested on both v1.82.3-stable and v1.83.5-nightly (same behavior)
  • Codex CLI: @openai/codex@0.118.0
  • Running LiteLLM as proxy in router mode with models configured via config.yaml

Related issues

  • #22051 — WebSocket mode support for Responses API (covers streaming bugs after connection, not the connection failure itself)
  • #18456 — WebSocket auth header issue for /v1/realtime (different endpoint)

The two patches

Our patch+83 −0
tests/test_litellm/proxy/response_api_endpoints/test_endpoints.pyindex d7010de640..d1eac6a1c3 100644@@ -562,6 +562,89 @@ class TestResponsesWSFirstFrameModelAuth:         )  +class TestResponsesWSConnectWithoutModelQueryParam:+    """Spec-compliant clients (e.g. Codex CLI) connect to bare /v1/responses and send+    the model inside the first response.create frame; the handshake must not be+    rejected for a missing ?model= query param."""++    @pytest.mark.parametrize("path", ["/v1/responses", "/responses"])+    @pytest.mark.parametrize(+        "first_frame",+        [+            {"type": "response.create", "model": "gpt-5.4", "input": "say hi"},+            {"type": "response.create", "response": {"model": "gpt-5.4", "input": "say hi"}},+        ],+    )+    def test_bare_url_accepts_and_uses_model_from_first_frame(self, path, first_frame):+        from litellm.proxy._types import UserAPIKeyAuth+        from litellm.proxy.auth.user_api_key_auth import user_api_key_auth_websocket++        processor = MagicMock()+        processor.common_processing_pre_call_logic = AsyncMock(side_effect=ValueError("stop after pre-call"))++        app.dependency_overrides[user_api_key_auth_websocket] = lambda: UserAPIKeyAuth(api_key="sk-1234")+        try:+            with (+                patch(+                    "litellm.proxy.response_api_endpoints.endpoints._enforce_responses_ws_first_frame_model_auth",+                    new_callable=AsyncMock,+                ) as mock_model_auth,+                patch(+                    "litellm.proxy.response_api_endpoints.endpoints.ProxyBaseLLMRequestProcessing",+                    return_value=processor,+                ) as mock_processor_cls,+            ):+                client = TestClient(app)+                with client.websocket_connect(path, headers={"Authorization": "Bearer sk-1234"}) as ws:+                    raw = json.dumps(first_frame)+                    ws.send_text(raw)+                    error = json.loads(ws.receive_text())+        finally:+            app.dependency_overrides.pop(user_api_key_auth_websocket, None)++        assert error["error"]["message"] == "stop after pre-call"+        data = mock_processor_cls.call_args.kwargs["data"]+        assert data["model"] == "gpt-5.4"+        assert data["first_message"] == raw+        assert mock_model_auth.await_args.kwargs["model"] == "gpt-5.4"+        assert processor.common_processing_pre_call_logic.await_args.kwargs["model"] == "gpt-5.4"++    def test_bare_url_passes_real_websocket_auth_with_master_key(self):+        processor = MagicMock()+        processor.common_processing_pre_call_logic = AsyncMock(side_effect=ValueError("stop after pre-call"))++        with (+            patch("litellm.proxy.proxy_server.master_key", "sk-1234"),+            patch("litellm.proxy.proxy_server.prisma_client", None),+            patch(+                "litellm.proxy.response_api_endpoints.endpoints.ProxyBaseLLMRequestProcessing",+                return_value=processor,+            ) as mock_processor_cls,+        ):+            client = TestClient(app)+            with client.websocket_connect("/v1/responses", headers={"Authorization": "Bearer sk-1234"}) as ws:+                ws.send_text(json.dumps({"type": "response.create", "model": "gpt-5.4", "input": "say hi"}))+                error = json.loads(ws.receive_text())++        assert error["error"]["message"] == "stop after pre-call"+        assert mock_processor_cls.call_args.kwargs["data"]["model"] == "gpt-5.4"++    def test_bare_url_without_model_anywhere_gets_error_frame(self):+        from litellm.proxy._types import UserAPIKeyAuth+        from litellm.proxy.auth.user_api_key_auth import user_api_key_auth_websocket++        app.dependency_overrides[user_api_key_auth_websocket] = lambda: UserAPIKeyAuth(api_key="sk-1234")+        try:+            client = TestClient(app)+            with client.websocket_connect("/v1/responses", headers={"Authorization": "Bearer sk-1234"}) as ws:+                ws.send_text(json.dumps({"type": "response.create", "input": "say hi"}))+                error = json.loads(ws.receive_text())+        finally:+            app.dependency_overrides.pop(user_api_key_auth_websocket, None)++        assert "No model provided" in error["error"]["message"]++ class TestReadWSModelFromFirstFrameErrors:     @pytest.mark.asyncio     async def test_timeout_closes_without_error_frame(self): 
Merged fix · PR #33101+54 −2
litellm/responses/streaming_iterator.pyindex 8d766cf1cd0..61c93063826 100644@@ -2589,8 +2589,7 @@ class ManagedResponsesWebSocketHandler:         if "litellm_metadata" not in call_kwargs:             call_kwargs["litellm_metadata"] = {}         call_kwargs["litellm_metadata"]["proxy_server_request"] = proxy_server_request-        call_kwargs.setdefault("litellm_params", {})-        call_kwargs["litellm_params"]["proxy_server_request"] = proxy_server_request+        call_kwargs["proxy_server_request"] = proxy_server_request      async def _stream_and_forward(self, model: str, call_kwargs: dict[str, Any]) -> _MutableJsonObject | None:         """tests/test_litellm/responses/test_responses_websocket_all_providers.pyindex fe3c4a0640d..0394e260606 100644@@ -669,6 +669,59 @@ class TestChunkTransformation:         assert ManagedResponsesWebSocketHandler._input_to_messages({}) == []  +class TestUpdateProxyRequest:+    """Regression tests for ManagedResponsesWebSocketHandler._update_proxy_request.++    The managed WebSocket path calls ``litellm.aresponses(model=..., **call_kwargs)``.+    ``litellm_params`` is not a Responses API request field, so passing it as a+    top-level kwarg leaks it into the provider request body and providers that+    forbid extra inputs (e.g. Anthropic) reject the call with+    ``litellm_params: Extra inputs are not permitted``. The request-tracking data+    must ride along as ``proxy_server_request`` instead, which litellm consumes+    internally and never forwards to the provider.+    """++    def test_does_not_inject_litellm_params_kwarg(self):+        from litellm.responses.streaming_iterator import (+            ManagedResponsesWebSocketHandler,+        )++        call_kwargs = {+            "input": "hello",+            "store": True,+            "litellm_metadata": {+                "proxy_server_request": {"headers": {}, "body": {}},+            },+        }++        ManagedResponsesWebSocketHandler._update_proxy_request(+            call_kwargs, "anthropic/claude-sonnet-4-5"+        )++        assert "litellm_params" not in call_kwargs+        assert call_kwargs["proxy_server_request"]["body"]["model"] == (+            "anthropic/claude-sonnet-4-5"+        )+        assert call_kwargs["proxy_server_request"]["body"]["input"] == "hello"++    def test_proxy_server_request_matches_metadata(self):+        from litellm.responses.streaming_iterator import (+            ManagedResponsesWebSocketHandler,+        )++        call_kwargs = {+            "input": "hi",+            "litellm_metadata": {"proxy_server_request": {"body": {}}},+        }++        ManagedResponsesWebSocketHandler._update_proxy_request(call_kwargs, "gpt-4o")++        assert (+            call_kwargs["proxy_server_request"]+            == call_kwargs["litellm_metadata"]["proxy_server_request"]+        )++ class TestWebSocketEventTypes:     """Test that all WebSocket event types are properly handled with dict-based chunks"""