Backtestslitellmrun-20261019-033420#41421

Issue #41421 Merged fix PR #39512 Run 19 Oct 2026 · run-20261019-033420 BerriAI/litellm

[Bug]: LiteLLM Proxy 1.100.0 serializes image-edit uploads as strings for OpenAI GPT Image models

Better

Our patch was judged better than the merged fix.

In this judgment, Patch B = our agent, Patch A = the merged fix.

Both patches address the same root cause: `_read_request_body` re-parses the multipart body, so the `image[]`/`mask[]` parts survive in `data` as raw UploadFiles alongside the converted `image`/`mask` buffers, and the SDK stringifies them into text parts. Both independent tests pass for both patches. Patch A drops exactly the two bracketed keys; Patch B drops any UploadFile under the four known file keys and converts any other leaked UploadFile to BytesIO rather than letting it be stringified. B's approach is keyed on the actual failure condition (a raw UploadFile in the forwarded payload) rather than on two literal key names, so it is more robust at low risk; the extra conversion for unknown upload fields is out of scope but adds no risk and is tested. On tests, B is clearly stronger: beyond unit checks on the captured dict, it adds an end-to-end regression that drives `litellm.aimage_edit` through an httpx MockTransport and asserts the real multipart body contains a filename-bearing file part with the PNG bytes and no `UploadFile(`/`BytesIO` text, which mirrors the issue precisely. A's tests are adequate but only inspect the captured dict. On scope, A includes unrelated reformatting of existing test lines (the 8 removed lines), which is churn; B touches only what it needs. Craft is comparable; B's comment explains why the filtering exists, and its `StarletteUploadFile` annotation is correct since FastAPI's UploadFile subclasses it. B is the better fix.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

43m

Wall-clock time in the container

Full test suite

No new failures

414 passed · 0 skipped · 414 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

100vs 88 merged

Changed lines, ours vs merged

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

Image edits through the LiteLLM Proxy fail in version 1.100.0 for OpenAI GPT Image models. The uploaded image file is being converted to a string representation of a Python BytesIO object ("<_io.BytesIO object at 0x7f0d0bc73600>") before being sent to OpenAI.

OpenAI rejects this with:

{
  "error": {
    "message": "Invalid type for 'image[0]': expected a file, but got a string instead.",
    "type": "invalid_request_error",
    "param": "image[0]",
    "code": "invalid_type"
  }
}

The same request succeeds in LiteLLM version 1.95.0.

User Flow

Before a fix: A user sends an image-edit request through LiteLLM Proxy 1.100.0 using a multipart/form-data request to POST /v1/images/edits with an uploaded image file and a prompt (e.g., "Make this image photorealistic"). LiteLLM receives the UploadFile correctly but serializes it to a string before forwarding to OpenAI. OpenAI rejects the request because it expects a file object, not a string representation.

  1. User sends: curl -X POST "http://localhost:4000/v1/images/edits" -F "model=openai/gpt-image-2" -F "prompt=Make this image photorealistic" -F "image[]=@./input.png;type=image/png"
  2. LiteLLM receives UploadFile with filename, size, and content-type headers
  3. LiteLLM converts the UploadFile's BytesIO object to a string
  4. LiteLLM forwards stringified image to OpenAI
  5. OpenAI rejects with invalid_type error

After a fix: The same user flow succeeds. LiteLLM preserves the UploadFile/BytesIO object and forwards it to OpenAI as a multipart file, and OpenAI processes it successfully.

Proof the bug occurs

Config / setup the proxy ran with:
Default LiteLLM Proxy setup with OpenAI API key configured

Version or commit:
1.100.0 (regression; works in 1.95.0)

Commands and their full output:

curl -X POST "http://localhost:4000/v1/images/edits" \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -F "model=openai/gpt-image-2" \
  -F "prompt=Make this image photorealistic" \
  -F "image[]=@./input.png;type=image/png"

LiteLLM request logging shows:

  • Receives: image[]: UploadFile(filename='...png', size=2014638, headers=Headers({'content-type': 'image/png', 'content-disposition': 'form-data; name="image[]"; filename="...png"'}))
  • Sends to OpenAI: {"image": ["<_io.BytesIO object at 0x7f0d0bc73600>"], "model": "openai/gpt-image-2", "prompt": "..."}

OpenAI error response:

{
  "error": {
    "message": "Invalid type for 'image[0]': expected a file, but got a string instead.",
    "type": "invalid_request_error",
    "param": "image[0]",
    "code": "invalid_type"
  }
}

Stack trace passes through: litellm/proxy/image_endpoints/endpoints.py:313 and litellm/images/main.py:1038

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

1.100.0

Twitter / LinkedIn details

_No response_

The two patches

Our patch+93 −7
litellm/proxy/image_endpoints/endpoints.pyindex 5b90c0ff83..cb20a1db8c 100644@@ -6,6 +6,7 @@ from typing import Final, get_type_hints import orjson from fastapi import APIRouter, Depends, File, HTTPException, Request, Response, UploadFile, status from fastapi.responses import ORJSONResponse+from starlette.datastructures import UploadFile as StarletteUploadFile  import litellm from litellm.litellm_core_utils.prompt_templates.common_utils import (@@ -35,9 +36,10 @@ from litellm.types.llms.openai import ChatCompletionUserMessage router: Final = APIRouter()  IMAGE_EDIT_NUMERIC_FORM_FIELDS: Final = numeric_form_fields(get_type_hints(ImageEditRequestParams))+IMAGE_EDIT_FILE_FORM_FIELDS: Final = frozenset({"image", "image[]", "mask", "mask[]"})  -async def uploadfile_to_bytesio(upload: UploadFile) -> io.BytesIO:+async def uploadfile_to_bytesio(upload: StarletteUploadFile) -> io.BytesIO:     """     Read a FastAPI UploadFile into a BytesIO and set .name so OpenAI SDK     infers filename/content-type correctly.@@ -294,12 +296,20 @@ async def image_edit_api(     #########################################################     # Read request body and convert UploadFiles to BytesIO     #########################################################-    data: Final = dict(-        coerce_numeric_form_fields(-            parsed_body=await _read_request_body(request=request),-            numeric_fields=IMAGE_EDIT_NUMERIC_FORM_FIELDS,-        )-    )+    # Raw multipart parts must not reach the provider as UploadFiles (they get+    # sent as their str() repr). `image`/`mask` and their `[]` aliases are+    # replaced by the converted buffers below; any other upload (e.g. Stability+    # `style_image`) is converted to BytesIO in place.+    data: Final[dict[str, object]] = {}+    for key, value in coerce_numeric_form_fields(+        parsed_body=await _read_request_body(request=request),+        numeric_fields=IMAGE_EDIT_NUMERIC_FORM_FIELDS,+    ).items():+        if isinstance(value, StarletteUploadFile):+            if key in IMAGE_EDIT_FILE_FORM_FIELDS:+                continue+            value = await uploadfile_to_bytesio(value)+        data[key] = value     image_files: Final = await batch_to_bytesio(image)     mask_files: Final = await batch_to_bytesio(mask)     if image_files:tests/test_litellm/proxy/image_endpoints/test_endpoints.pyindex 31b87530c9..338926ba61 100644@@ -5,14 +5,18 @@ from collections.abc import Iterator, Mapping from types import SimpleNamespace from typing import Any, Dict +import httpx import orjson import pytest from fastapi import FastAPI, HTTPException from fastapi.testclient import TestClient+from starlette.datastructures import UploadFile as StarletteUploadFile from starlette.requests import Request from starlette.responses import Response +import litellm from litellm._logging import verbose_proxy_logger+from litellm.llms.custom_httpx.http_handler import AsyncHTTPHandler from litellm.proxy._types import ProxyException, UserAPIKeyAuth from litellm.proxy.auth.user_api_key_auth import user_api_key_auth from litellm.proxy.image_endpoints import endpoints@@ -172,6 +176,78 @@ def test_image_edit_multipart_n_that_is_not_a_number_is_left_alone(monkeypatch):     assert captured["n"] == "two"  +PNG_BYTES = b"\x89PNG\r\n\x1a\nfakepng"+++@pytest.mark.parametrize("field", ["image", "mask"])+def test_image_edit_bracket_upload_is_forwarded_only_as_a_file(monkeypatch, field):+    """An `image[]` / `mask[]` upload must not linger in the forwarded data as the+    raw UploadFile, which the SDK used to stringify into a text part."""+    captured: Dict[str, Any] = {}++    response = _image_edit_client(monkeypatch, captured).post(+        "/v1/images/edits",+        files={f"{field}[]": ("input.png", PNG_BYTES, "image/png")},+        data={"model": "openai/gpt-image-2", "prompt": "Make this image photorealistic"},+    )++    assert response.status_code == 200+    assert f"{field}[]" not in captured+    assert [buffer.getvalue() for buffer in captured[field]] == [PNG_BYTES]+    assert not any(isinstance(value, StarletteUploadFile) for value in captured.values())+++def test_image_edit_extra_file_upload_is_forwarded_as_a_readable_buffer(monkeypatch):+    """Other uploads (e.g. Stability `style_image`) must reach the provider as a+    readable buffer, not be dropped or forwarded as the raw UploadFile."""+    captured: Dict[str, Any] = {}++    response = _image_edit_client(monkeypatch, captured).post(+        "/v1/images/edits",+        files=[+            ("image[]", ("input.png", PNG_BYTES, "image/png")),+            ("style_image", ("style.png", b"style-bytes", "image/png")),+        ],+        data={"model": "stability/stable-style-transfer-v1:0", "prompt": "Restyle"},+    )++    assert response.status_code == 200+    assert captured["style_image"].getvalue() == b"style-bytes"+    assert captured["style_image"].name == "style.png"+    assert [buffer.getvalue() for buffer in captured["image"]] == [PNG_BYTES]+    assert not any(isinstance(value, StarletteUploadFile) for value in captured.values())+++@pytest.mark.asyncio+async def test_image_edit_bracket_upload_reaches_openai_as_a_file_part(monkeypatch):+    """Regression: `-F image[]=@input.png` reached OpenAI as a text part holding the+    UploadFile repr, which OpenAI rejects with "expected a file, but got a string"."""+    captured: Dict[str, Any] = {}+    response = _image_edit_client(monkeypatch, captured).post(+        "/v1/images/edits",+        files={"image[]": ("input.png", PNG_BYTES, "image/png")},+        data={"model": "openai/gpt-image-2", "prompt": "Make this image photorealistic"},+    )+    assert response.status_code == 200++    sent: Dict[str, bytes] = {}++    def respond(request: httpx.Request) -> httpx.Response:+        sent["body"] = request.content+        return httpx.Response(200, json={"created": 1712697600, "data": [{"b64_json": "aW1n"}]})++    client = AsyncHTTPHandler()+    client.client = httpx.AsyncClient(transport=httpx.MockTransport(respond))+    await litellm.aimage_edit(**captured, api_key="sk-test", api_base="https://edit.example/v1", client=client)++    image_parts = [part for part in sent["body"].split(b"\r\n--") if b'name="image[]"' in part]+    assert len(image_parts) == 1+    assert b"filename=" in image_parts[0]+    assert PNG_BYTES in image_parts[0]+    assert b"UploadFile(" not in sent["body"]+    assert b"BytesIO" not in sent["body"]++ @pytest.mark.asyncio async def test_a_model_the_router_cannot_serve_answers_an_openai_typed_error(monkeypatch: pytest.MonkeyPatch):     """A bare HTTPException carries no type or param, so the tail used to ship the 
Merged fix · PR #39512+74 −14
litellm/proxy/image_endpoints/endpoints.pyindex 5b90c0ff830..b9580ba3948 100644@@ -36,6 +36,10 @@ router: Final = APIRouter()  IMAGE_EDIT_NUMERIC_FORM_FIELDS: Final = numeric_form_fields(get_type_hints(ImageEditRequestParams)) +IMAGE_ARRAY_FIELD: Final = "image[]"+MASK_ARRAY_FIELD: Final = "mask[]"+BRACKETED_FILE_FIELDS: Final = frozenset({IMAGE_ARRAY_FIELD, MASK_ARRAY_FIELD})+  async def uploadfile_to_bytesio(upload: UploadFile) -> io.BytesIO:     """@@ -244,9 +248,9 @@ async def image_edit_api(     fastapi_response: Response,     user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth),     image: list[UploadFile] | None = File(None),-    image_array: list[UploadFile] | None = File(None, alias="image[]"),+    image_array: list[UploadFile] | None = File(None, alias=IMAGE_ARRAY_FIELD),     mask: list[UploadFile] | None = File(None),-    mask_array: list[UploadFile] | None = File(None, alias="mask[]"),+    mask_array: list[UploadFile] | None = File(None, alias=MASK_ARRAY_FIELD),     model: str | None = None, ):     """@@ -294,12 +298,14 @@ async def image_edit_api(     #########################################################     # Read request body and convert UploadFiles to BytesIO     #########################################################-    data: Final = dict(-        coerce_numeric_form_fields(+    data: Final = {+        key: value+        for key, value in coerce_numeric_form_fields(             parsed_body=await _read_request_body(request=request),             numeric_fields=IMAGE_EDIT_NUMERIC_FORM_FIELDS,-        )-    )+        ).items()+        if key not in BRACKETED_FILE_FIELDS+    }     image_files: Final = await batch_to_bytesio(image)     mask_files: Final = await batch_to_bytesio(mask)     if image_files:tests/test_litellm/proxy/image_endpoints/test_endpoints.pyindex 31b87530c94..ad0901e9eee 100644@@ -95,18 +95,14 @@ async def test_image_generation_prompt_rerouting(monkeypatch):     monkeypatch.setattr("litellm.proxy.proxy_server.general_settings", {})     monkeypatch.setattr("litellm.proxy.proxy_server.llm_router", None)     monkeypatch.setattr("litellm.proxy.proxy_server.proxy_config", {})-    monkeypatch.setattr(-        "litellm.proxy.proxy_server.proxy_logging_obj", fake_proxy_logger-    )+    monkeypatch.setattr("litellm.proxy.proxy_server.proxy_logging_obj", fake_proxy_logger)     monkeypatch.setattr("litellm.proxy.proxy_server.user_model", None)     monkeypatch.setattr("litellm.proxy.proxy_server.version", "test-version")     monkeypatch.setattr(         "litellm.proxy.common_request_processing.ProxyBaseLLMRequestProcessing.get_custom_headers",         classmethod(lambda *args, **kwargs: {}),     )-    monkeypatch.setattr(-        "litellm.proxy.image_endpoints.endpoints.route_request", fake_route_request-    )+    monkeypatch.setattr("litellm.proxy.image_endpoints.endpoints.route_request", fake_route_request)      result = await endpoints.image_generation(         request=request,@@ -141,6 +137,60 @@ def _image_edit_client(monkeypatch, captured: Dict[str, Any]) -> TestClient:     return TestClient(app)  +def test_image_edit_image_array_alias_is_not_forwarded(monkeypatch):+    """The documented `image[]` alias must reach the provider only as `image`."""+    captured: Dict[str, Any] = {}++    response = _image_edit_client(monkeypatch, captured).post(+        "/v1/images/edits",+        files={"image[]": ("tree.png", b"\x89PNG\r\n\x1a\ntree", "image/png")},+        data={"model": "gpt-image-1", "prompt": "add a hat"},+    )++    assert response.status_code == 200+    assert "image[]" not in captured+    assert [buffer.getvalue() for buffer in captured["image"]] == [b"\x89PNG\r\n\x1a\ntree"]+    assert [buffer.name for buffer in captured["image"]] == ["tree.png"]+++def test_image_edit_mask_array_alias_is_not_forwarded(monkeypatch):+    """`mask[]` has the same shape as `image[]` and must be dropped the same way."""+    captured: Dict[str, Any] = {}++    response = _image_edit_client(monkeypatch, captured).post(+        "/v1/images/edits",+        files={+            "image": ("tree.png", b"\x89PNG\r\n\x1a\ntree", "image/png"),+            "mask[]": ("mask.png", b"\x89PNG\r\n\x1a\nmask", "image/png"),+        },+        data={"model": "gpt-image-1", "prompt": "add a hat"},+    )++    assert response.status_code == 200+    assert "mask[]" not in captured+    assert [buffer.getvalue() for buffer in captured["mask"]] == [b"\x89PNG\r\n\x1a\nmask"]+    assert [buffer.getvalue() for buffer in captured["image"]] == [b"\x89PNG\r\n\x1a\ntree"]+++def test_image_edit_canonical_file_fields_still_reach_the_provider(monkeypatch):+    """Dropping the bracketed aliases must not touch the canonical fields."""+    captured: Dict[str, Any] = {}++    response = _image_edit_client(monkeypatch, captured).post(+        "/v1/images/edits",+        files={+            "image": ("tree.png", b"\x89PNG\r\n\x1a\ntree", "image/png"),+            "mask": ("mask.png", b"\x89PNG\r\n\x1a\nmask", "image/png"),+        },+        data={"model": "gpt-image-1", "prompt": "add a hat"},+    )++    assert response.status_code == 200+    assert [buffer.getvalue() for buffer in captured["image"]] == [b"\x89PNG\r\n\x1a\ntree"]+    assert [buffer.getvalue() for buffer in captured["mask"]] == [b"\x89PNG\r\n\x1a\nmask"]+    assert captured["prompt"] == "add a hat"++ def test_image_edit_multipart_n_reaches_the_provider_as_an_int(monkeypatch):     """A multipart `n` must not arrive as the string Starlette parsed it into."""     captured: Dict[str, Any] = {}@@ -180,7 +230,9 @@ async def test_a_model_the_router_cannot_serve_answers_an_openai_typed_error(mon     async def fake_add_litellm_data_to_request(**kwargs: object) -> object:         return kwargs["data"] -    async def fake_pre_call_hook(*, user_api_key_dict: UserAPIKeyAuth, data: dict[str, object], call_type: str) -> dict[str, object]:+    async def fake_pre_call_hook(+        *, user_api_key_dict: UserAPIKeyAuth, data: dict[str, object], call_type: str+    ) -> dict[str, object]:         return data      async def fake_post_call_failure_hook(**_: object) -> None:@@ -211,7 +263,9 @@ async def test_a_model_the_router_cannot_serve_answers_an_openai_typed_error(mon     request = Request({"type": "http", "method": "POST", "path": "/v1/images/generations", "headers": []}, receive)      with pytest.raises(ProxyException) as raised:-        await endpoints.image_generation(request=request, fastapi_response=Response(), user_api_key_dict=UserAPIKeyAuth())+        await endpoints.image_generation(+            request=request, fastapi_response=Response(), user_api_key_dict=UserAPIKeyAuth()+        )      assert (raised.value.type, raised.value.param, raised.value.code) == ("invalid_request_error", None, "404")