Backtestslitellmrun-20261018-182052#25985

Issue #25985 Merged fix PR # Run 18 Oct 2026 · run-20261018-182052 BerriAI/litellm

[Bug]: Ollama chat transformation fails with JSONDecodeError 'Unterminated string' when parsing tool call arguments

Better

Our patch was judged better than the merged fix.

In this judgment, Patch A = our agent, Patch B = the merged fix.

Both patches make the three independent tests pass, so the deciding factors are coverage, craft, and test quality. Patch A fixes the problem at both places the same `json.loads(call["function"]["arguments"])` pattern exists: `OllamaChatConfig.transform_request` (the one in the traceback) and `ollama_pt` in factory.py, which the issue's related-issues list implicitly points at. It routes through a single helper that reuses the existing `parse_tool_call_arguments` utility (so truncated JSON is repaired, the issue's optional ask), treats empty-string arguments as `{}`, passes through already-parsed dict/list values, logs the raw string, and raises a BadRequestError that includes the tool name, the real `model`, and the raw arguments. Its tests cover valid, repaired-truncated, empty, and unterminated-string cases for both code paths and assert on the error contents. Patch B is a minimal try/except in only the chat transformation, hardcodes `model="ollama"` instead of the actual model, does no repair, and still crashes on empty-string arguments. More seriously, its test edit replaces the `def test_tools_passed_directly_without_capability_check` line with the two new tests, leaving the original test's docstring and body dangling inside the new `test_transform_request_valid_tool_call_arguments_passes` method, so an existing regression test is effectively destroyed. A is the more correct, more complete, and better-crafted fix.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

15m 3s

Wall-clock time in the container

Full test suite

No new failures

771 passed · 8 skipped · 779 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

173vs 93 merged

Changed lines, ours vs merged

Bug Description

When using LiteLLM proxy with an Ollama-compatible provider (custom model glm-5), the request transformation fails with a JSONDecodeError: Unterminated string error when parsing tool call arguments. The error occurs in litellm/llms/ollama/chat/transformation.py at line 274.

Steps to Reproduce

  1. Configure LiteLLM proxy with an Ollama-compatible provider (e.g., custom local model server)
  2. Send a chat completion request that includes tool/function definitions
  3. The upstream model returns a tool call with arguments
  4. LiteLLM attempts to transform the request and fails to parse the tool arguments JSON

Expected Behavior

LiteLLM should gracefully handle tool call arguments, either:

  • Successfully parse valid JSON arguments
  • Handle malformed/incomplete JSON with proper error handling (catch exception, log the actual arguments string for debugging, and fail with a more informative error message)

Actual Behavior

LiteLLM crashes with:

json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 10 (char 9)

The error is raised at litellm/llms/ollama/chat/transformation.py:274:

arguments = json.loads(typed_tool["function"]["arguments"])

Root Cause Analysis

The code assumes typed_tool["function"]["arguments"] is always valid JSON. However, the upstream model may return:

  1. Incomplete/truncated JSON (e.g., due to streaming issues or token limits)
  2. Malformed JSON with unescaped characters
  3. Partial tool call arguments

The current implementation does not handle these edge cases.

Relevant Log Output

2026-04-17 18:28:21.669 | 23:28:21 - LiteLLM Proxy:ERROR: common_request_processing.py:1522 - litellm.proxy.proxy_server._handle_llm_api_exception(): Exception occured - litellm.APIConnectionError: Unterminated string starting at: line 1 column 10 (char 9)
2026-04-17 18:28:21.669 | Traceback (most recent call last):
2026-04-17 18:28:21.669 |   File "/usr/lib/python3.13/site-packages/litellm/main.py", line 4015, in completion
2026-04-17 18:28:21.669 |     response = base_llm_http_handler.completion(
2026-04-17 18:28:21.669 |         model=model,
2026-04-17 18:28:21.669 |     ...<14 lines>...
2026-04-17 18:28:21.669 |         client=client,
2026-04-17 18:28:21.669 |     )
2026-04-17 18:28:21.669 |   File "/usr/lib/python3.13/site-packages/litellm/llms/custom_httpx/llm_http_handler.py", line 410, in completion
2026-04-17 18:28:21.669 |     data = provider_config.transform_request(
2026-04-17 18:28:21.669 |         model=model,
2026-04-17 18:28:21.669 |     ...<3 lines>...
2026-04-17 18:28:21.669 |         headers=headers,
2026-04-17 18:28:21.669 |     )
2026-04-17 18:28:21.669 |   File "/usr/lib/python3.13/site-packages/litellm/llms/ollama/chat/transformation.py", line 274, in transform_request
2026-04-17 18:28:21.669 |     arguments = json.loads(typed_tool["function"]["arguments"])
2026-04-17 18:28:21.669 |   File "/usr/lib/python3.13/json/__init__.py", line 352, in loads
2026-04-17 18:28:21.669 |     return _default_decoder.decode(s)
2026-04-17 18:28:21.669 |            ~~~~~~~~~~~~~~~~~~~~~^^^
2026-04-17 18:28:21.669 |   File "/usr/lib/python3.13/json/decoder.py", line 345, in decode
2026-04-17 18:28:21.669 |     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
2026-04-17 18:28:21.669 |                ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
2026-04-17 18:28:21.669 |   File "/usr/lib/python3.13/json/decoder.py", line 361, in raw_decode
2026-04-17 18:28:21.669 |     obj, end = self.scan_once(s, idx)
2026-04-17 18:28:21.669 |                ~~~~~~~~~~~~~~^^^^^^^^
2026-04-17 18:28:21.669 | json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 10 (char 9)
2026-04-17 18:28:21.669 | . Received Model Group=glm-5
2026-04-17 18:28:21.669 | Available Model Group Fallbacks=None LiteLLM Retried: 3 times, LiteLLM Max Retries: 3

Environment

  • LiteLLM Proxy running via Docker
  • Python 3.13
  • Model: glm-5 (custom local provider via Ollama-compatible endpoint)

Suggested Fix

Add try/except around the JSON parsing in transformation.py to:

  1. Log the actual arguments string that failed to parse (for debugging)
  2. Return a more informative error message including the raw arguments string
  3. Optionally, attempt to repair common JSON issues (truncated strings, missing closing braces)
try:
    arguments = json.loads(typed_tool["function"]["arguments"])
except json.JSONDecodeError as e:
    # Log the malformed arguments for debugging
    verbose_logger.error(f"Failed to parse tool call arguments: {typed_tool['function']['arguments']!r}")
    raise litellm.InternalServerError(
        f"Model returned malformed tool call arguments: {e.msg}. Raw arguments: {typed_tool['function']['arguments']!r}"
    )

Related Issues

  • #13823 - Similar JSONDecodeError in Ollama completion transformation (response side)
  • #11267 - APIConnectionError parsing Tool call response from Ollama
  • #18920 - BadRequestError with Unterminated string (Anthropic)

The two patches

Our patch+168 −5
litellm/litellm_core_utils/prompt_templates/factory.pyindex a037360c87..ac79cff365 100644@@ -16,6 +16,7 @@ import litellm.types.llms from litellm import verbose_logger from litellm._uuid import uuid from litellm.llms.custom_httpx.http_handler import HTTPHandler, get_async_httpx_client+from litellm.llms.ollama.common_utils import parse_ollama_tool_call_arguments from litellm.types.files import get_file_extension_from_mime_type from litellm.types.llms.anthropic import * from litellm.types.llms.bedrock import CachePointBlock@@ -250,7 +251,11 @@ def ollama_pt(                 for call in tool_calls:                     call_id: str = call["id"]                     function_name: str = call["function"]["name"]-                    arguments = json.loads(call["function"]["arguments"])+                    arguments = parse_ollama_tool_call_arguments(+                        call["function"].get("arguments"),+                        tool_name=function_name,+                        model=model,+                    )                      ollama_tool_calls.append(                         {litellm/llms/ollama/chat/transformation.pyindex c990cc2e09..c405c4ee61 100644@@ -36,7 +36,7 @@ from litellm.types.llms.openai import ( ) from litellm.types.utils import ModelResponse, ModelResponseStream -from ..common_utils import OllamaError+from ..common_utils import OllamaError, parse_ollama_tool_call_arguments  if TYPE_CHECKING:     from litellm.litellm_core_utils.litellm_logging import Logging as _LiteLLMLoggingObj@@ -272,7 +272,11 @@ class OllamaChatConfig(BaseConfig):                     if typed_tool["type"] == "function":                         arguments = {}                         if "arguments" in typed_tool["function"]:-                            arguments = json.loads(typed_tool["function"]["arguments"])+                            arguments = parse_ollama_tool_call_arguments(+                                typed_tool["function"]["arguments"],+                                tool_name=typed_tool["function"].get("name"),+                                model=model,+                            )                         ollama_tool_call = OllamaToolCall(                             function=OllamaToolCallFunction(                                 name=typed_tool["function"].get("name") or "",litellm/llms/ollama/common_utils.pyindex 8aedd9b350..3714c1fc96 100644@@ -1,8 +1,12 @@-from typing import List, Optional, Union+from typing import Any, List, Optional, Union  import httpx  from litellm import verbose_logger+from litellm.exceptions import BadRequestError+from litellm.litellm_core_utils.prompt_templates.common_utils import (+    parse_tool_call_arguments,+) from litellm.llms.base_llm.chat.transformation import BaseLLMException  @@ -13,6 +17,48 @@ class OllamaError(BaseLLMException):         super().__init__(status_code=status_code, message=message, headers=headers)  +def parse_ollama_tool_call_arguments(+    arguments: Any,+    tool_name: Optional[str],+    model: str,+) -> Any:+    """+    Parse an assistant tool call's OpenAI-style `arguments` string into the+    object Ollama expects.++    Truncated JSON (missing closing brackets) is repaired. Arguments that cannot+    be parsed raise a BadRequestError that includes the raw arguments string,+    instead of an opaque JSONDecodeError.+    """+    if arguments is None or isinstance(arguments, (dict, list)):+        return arguments if arguments is not None else {}+    if not isinstance(arguments, str):+        raise BadRequestError(+            message=(+                f"OllamaException - Invalid tool call arguments for tool "+                f"'{tool_name or '<unknown>'}': expected a JSON string, got "+                f"{type(arguments).__name__}. Raw arguments: {arguments!r}"+            ),+            model=model,+            llm_provider="ollama",+        )+    try:+        return parse_tool_call_arguments(+            arguments, tool_name=tool_name, context="Ollama request"+        )+    except ValueError as e:+        verbose_logger.error(+            "Failed to parse tool call arguments for tool '%s': %r",+            tool_name or "<unknown>",+            arguments,+        )+        raise BadRequestError(+            message=f"OllamaException - Malformed tool call arguments in assistant message. {e}",+            model=model,+            llm_provider="ollama",+        ) from e++ def _convert_image(image):     """     Convert image to base64 encoded image if not already in base64 formattests/test_litellm/litellm_core_utils/prompt_templates/test_litellm_core_utils_prompt_templates_factory.pyindex 30b47a853e..27c613e2a3 100644@@ -2339,3 +2339,40 @@ def test_anthropic_messages_pt_file_block_preserves_cache_control():     assert text_block["type"] == "text"     assert "cache_control" in text_block     assert text_block["cache_control"]["type"] == "ephemeral"+++def _ollama_pt_tool_call_messages(arguments):+    return [+        {"role": "user", "content": "Read the file"},+        {+            "role": "assistant",+            "content": "",+            "tool_calls": [+                {+                    "id": "call_1",+                    "type": "function",+                    "function": {"name": "read_file", "arguments": arguments},+                }+            ],+        },+    ]+++def test_ollama_pt_tool_call_truncated_arguments_are_repaired():+    result = ollama_pt(+        model="llama2",+        messages=_ollama_pt_tool_call_messages('{"path": "/tmp/a.txt"'),+    )+    assert isinstance(result, dict)+    assert '"path": "/tmp/a.txt"' in result["prompt"]+    assert '"name": "read_file"' in result["prompt"]+++def test_ollama_pt_tool_call_unterminated_string_raises_bad_request():+    raw = '{"path": "/tmp/a.t'+    with pytest.raises(litellm.BadRequestError) as exc_info:+        ollama_pt(model="llama2", messages=_ollama_pt_tool_call_messages(raw))+    message = str(exc_info.value)+    assert "read_file" in message+    assert "Unterminated string" in message+    assert raw in messagetests/test_litellm/llms/ollama/test_ollama_chat_transformation.pyindex a1afdd3a36..22e5fd2cd8 100644@@ -11,12 +11,13 @@ sys.path.insert( )  from litellm.llms.ollama.chat.transformation import OllamaChatConfig, OllamaChatCompletionResponseIterator+from litellm.llms.ollama.common_utils import parse_ollama_tool_call_arguments  from litellm.types.llms.openai import AllMessageValues from litellm.utils import get_optional_params  import json-from unittest.mock import MagicMock+from unittest.mock import MagicMock, patch  import litellm from litellm.types.utils import Choices, Message, ModelResponse@@ -735,3 +736,73 @@ class TestOllamaReasoningContentStreaming:         assert result.choices[0].finish_reason == "stop"  +++class TestOllamaChatToolCallArgumentsParsing:+    """Regression tests for assistant tool_call arguments that are not valid JSON."""++    def _transform_and_capture(self, arguments):+        parsed = []++        def _record(*args, **kwargs):+            value = parse_ollama_tool_call_arguments(*args, **kwargs)+            parsed.append(value)+            return value++        with patch(+            "litellm.llms.ollama.chat.transformation.parse_ollama_tool_call_arguments",+            side_effect=_record,+        ):+            result = self._transform(arguments)+        return result, parsed++    def _transform(self, arguments):+        config = OllamaChatConfig()+        messages = [+            {"role": "user", "content": "Read the file"},+            {+                "role": "assistant",+                "content": "",+                "tool_calls": [+                    {+                        "id": "call_1",+                        "type": "function",+                        "function": {"name": "read_file", "arguments": arguments},+                    }+                ],+            },+            {"role": "tool", "tool_call_id": "call_1", "content": "contents"},+        ]+        return config.transform_request(+            model="glm-5",+            messages=cast(list, messages),+            optional_params={},+            litellm_params={},+            headers={},+        )++    def test_valid_arguments_are_parsed(self):+        result, parsed = self._transform_and_capture('{"path": "/tmp/a.txt"}')+        assert parsed == [{"path": "/tmp/a.txt"}]+        assert [m["role"] for m in result["messages"]] == ["user", "assistant", "tool"]++    def test_truncated_arguments_are_repaired(self):+        result, parsed = self._transform_and_capture(+            '{"path": "/tmp/a.txt", "opts": {"lines": [1, 2]'+        )+        assert parsed == [{"path": "/tmp/a.txt", "opts": {"lines": [1, 2]}}]+        assert [m["role"] for m in result["messages"]] == ["user", "assistant", "tool"]++    def test_empty_arguments_do_not_raise(self):+        result, parsed = self._transform_and_capture("")+        assert parsed == [{}]+        assert [m["role"] for m in result["messages"]] == ["user", "assistant", "tool"]++    def test_unterminated_string_raises_informative_bad_request(self):+        raw = '{"path": "/tmp/a.t'+        with pytest.raises(litellm.BadRequestError) as exc_info:+            self._transform(raw)+        message = str(exc_info.value)+        assert "read_file" in message+        assert "Unterminated string" in message+        assert raw in message 
Merged fix · PR #+91 −2
litellm/llms/ollama/chat/transformation.pyindex c990cc2e093..31e694ce798 100644@@ -1,6 +1,7 @@ import json import time from litellm._uuid import uuid+from litellm._logging import verbose_logger from typing import (     TYPE_CHECKING,     Any,@@ -272,7 +273,18 @@ class OllamaChatConfig(BaseConfig):                     if typed_tool["type"] == "function":                         arguments = {}                         if "arguments" in typed_tool["function"]:-                            arguments = json.loads(typed_tool["function"]["arguments"])+                            raw_args = typed_tool["function"]["arguments"]+                            try:+                                arguments = json.loads(raw_args)+                            except json.JSONDecodeError as e:+                                verbose_logger.error(+                                    f"Failed to parse tool call arguments as JSON: {raw_args!r}. Error: {e}"+                                )+                                raise litellm.BadRequestError(+                                    message=f"Tool call arguments contain malformed JSON: {e.msg}. Raw arguments: {raw_args!r}",+                                    model="ollama",+                                    llm_provider="ollama",+                                )                         ollama_tool_call = OllamaToolCall(                             function=OllamaToolCallFunction(                                 name=typed_tool["function"].get("name") or "",tests/test_litellm/llms/ollama/test_ollama_chat_transformation.pyindex a1afdd3a369..6cd3e1a4dd9 100644@@ -338,7 +338,84 @@ class TestOllamaToolCalling:     Issue: https://github.com/BerriAI/litellm/issues/18922     """ -    def test_tools_passed_directly_without_capability_check(self):+    def test_transform_request_malformed_tool_call_arguments_raises_bad_request(self):+        """Test that malformed JSON in tool call arguments raises BadRequestError.++        Regression: json.JSONDecodeError was previously raised directly, wrapping it+        in litellm.BadRequestError gives callers an actionable error with context.++        Issue: https://github.com/BerriAI/litellm/issues/25985+        """+        config = OllamaChatConfig()++        messages = cast(+            list[AllMessageValues],+            [+                {+                    "role": "assistant",+                    "content": None,+                    "tool_calls": [+                        {+                            "id": "call_123",+                            "type": "function",+                            "function": {+                                "name": "get_weather",+                                "arguments": '{"city": "Toky',  # truncated JSON+                            },+                        }+                    ],+                }+            ],+        )++        import litellm++        with pytest.raises(litellm.BadRequestError) as exc_info:+            config.transform_request(+                model="qwen3:14b",+                messages=messages,+                optional_params={},+                litellm_params={},+                headers={},+            )++        assert "malformed JSON" in str(exc_info.value)++    def test_transform_request_valid_tool_call_arguments_passes(self):+        """Test that valid JSON tool call arguments are parsed without raising an exception."""+        config = OllamaChatConfig()++        messages = cast(+            list[AllMessageValues],+            [+                {+                    "role": "assistant",+                    "content": None,+                    "tool_calls": [+                        {+                            "id": "call_123",+                            "type": "function",+                            "function": {+                                "name": "get_weather",+                                "arguments": '{"city": "Tokyo"}',+                            },+                        }+                    ],+                }+            ],+        )++        # Should not raise any exception+        result = config.transform_request(+            model="qwen3:14b",+            messages=messages,+            optional_params={},+            litellm_params={},+            headers={},+        )+        assert "messages" in result++         """Test that tools are passed directly to Ollama without model capability checks.          Previously, the code called litellm.get_model_info() which could fail