Backtestsopenai-agents-pythonrun-20261024-184640#4510

Issue #4510 Merged fix PR #4513 Run 24 Oct 2026 · run-20261024-184640 openai/openai-agents-python

A completion truncated at zero tokens (finish_reason=length) returns zero output items and is indistinguishable from an empty turn

Better

Our patch was judged better than the merged fix.

In this judgment, Patch B = our agent, Patch A = the merged fix.

Both patches pick the same design (option 3: raise ModelBehaviorError when finish_reason='length' arrives with no text, refusal, or tool call) and both fix the OpenAI Chat Completions non-streaming path, the shared stream handler, and the tool-call buffering layer that would otherwise swallow the empty terminal choice. The decisive difference is coverage. The issue explicitly names four paths that carry the sibling content_filter guard: openai_chatcompletions.py, litellm_model.py, any_llm_model.py, and the stream handler. Patch B guards all four, adding the non-streaming check to LitellmModel (with getattr/provider_specific_fields handling matching that file's conventions) and AnyLLMModel, and making the shared handler raise unconditionally so every provider's streaming path is covered. Patch A deliberately scopes the fix to OpenAIChatCompletionsModel via an opt-in raise_on_length_truncation flag and leaves LiteLLM and AnyLLM returning the indistinguishable empty turn, which the independent LiteLLM test confirms (FAIL-ASSERT on base and on A, PASS on B). A's stated rationale that those providers' behavior 'must remain unchanged' contradicts the issue's own framing that the same reasoning applies to them word for word. On tests, B covers non-streaming and streaming for all three model classes, partial-text and empty-stop controls, buffered streaming, a reasoning-only variant, and end-to-end Runner.run / run_streamed assertions; A's tests are thorough for the OpenAI path (including span usage preservation) but add a test that locks in the unfixed behavior for the shared handler default. On craft, B centralizes the error in ChatCmplHelpers.empty_length_truncation_error with an actionable message and marks it to drain stream events, which matters for the streamed runner; A adds useful tracing-usage preservation and a _build_response_usage refactor, but that extra work does not compensate for leaving half the named paths broken. B is the better fix.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

13m 32s

Wall-clock time in the container

Full test suite

No new failures

423 passed · 0 skipped · 423 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

404vs 586 merged

Changed lines, ours vs merged

A completion truncated at zero visible tokens (finish_reason="length", empty content) converts to zero output items, and ModelResponse does not carry finish_reason, so the caller cannot tell it apart from a model that genuinely returned nothing.

This is the same failure the project already fixed for content_filter, and the reasoning written next to that fix applies here word for word.

Measured

Driving OpenAIChatCompletionsModel.get_response against a mocked completion with empty assistant content, varying only finish_reason:

finish_reason      output items   what the caller sees
content_filter     1              ['message'] -> ResponseOutputRefusal
length             0              []
stop               0              []

ModelResponse exposes output, usage, response_id, request_id. finish_reason is read once for a logger.debug line and then discarded, so nothing downstream can recover it.

Mechanism

Converter.message_to_output_items drops an empty message in two steps:

if message.content:            # "" is falsy, no text part is appended
    message_item.content.append(ResponseOutputText(...))
...
if message_item.content:       # still empty, so the message itself is never appended
    items.append(message_item)

That is correct for content=None. For a truncated turn it discards the one thing that distinguishes it.

Why I think this is in scope rather than by design

The content_filter guard in openai_chatcompletions.py:304, and its siblings in litellm_model.py, any_llm_model.py and the stream handler, exist because of exactly this shape. The comment says so:

Without this, message converts to zero output items and the caller sees an indistinguishable "empty turn", which drives agent loops into fruitless retries.

and the test for it is explicit:

A content-filter block (empty message, finish_reason=content_filter) must become an explicit ResponseOutputRefusal, not zero output items.

So "zero output items for an empty message" is already treated as a bug when the provider had something to say. Truncation is the other case where the provider had something to say.

I am not arguing that every empty message should produce an item. finish_reason="stop" with empty content is genuinely an empty turn and returning nothing for it seems right. The asymmetry is that length carries information and it is dropped.

Reachability

Any max_tokens small enough to be exhausted before the first visible token produces this shape. It is easiest to hit by accident with reasoning models, where the budget is consumed by reasoning tokens and the visible content can be empty while the request is nowhere near "finished". An agent loop then sees an empty turn, has no signal that the budget was the cause, and retries into the same wall.

Options

  1. Surface it on ModelResponse. Carry finish_reason (or an incomplete_reason) through, and let callers decide. Most general, and it also serves cases beyond this one.
  2. Mirror the content_filter treatment. Synthesize a refusal-like item when finish_reason == "length" and the message is otherwise empty. Smallest diff, consistent with the existing fix, but it borrows refusal for something that is not a refusal.
  3. Raise. A truncated-to-nothing turn is arguably a configuration error worth failing on rather than returning an empty result.

I lean towards 1 with 2 as a fallback, but this is a design call and I did not want to presume, especially as it touches four model paths.

Happy to send the PR for whichever shape you prefer, with tests covering the three finish reasons on both the streaming and non-streaming paths.

Environment

openai-agents 0.21.1, main at ebb746d, Python 3.12.12.

The two patches

Our patch+398 −6
src/agents/extensions/models/any_llm_model.pyindex a737bb83..ff12db2b 100644@@ -667,6 +667,18 @@ class AnyLLMModel(Model):                 "output_tokens_details": usage.output_tokens_details.model_dump(),             } +            # A completion truncated by the token limit before any visible output would+            # otherwise convert to zero output items, indistinguishable from an empty turn.+            if (+                message is not None+                and first_choice is not None+                and first_choice.finish_reason == "length"+                and not message.content+                and not message.refusal+                and not message.tool_calls+            ):+                raise ChatCmplHelpers.empty_length_truncation_error()+             provider_data: dict[str, Any] = {"model": self.model}             if message is not None and hasattr(response, "id"):                 provider_data["response_id"] = response.idsrc/agents/extensions/models/litellm_model.pyindex e64a8967..7a2072c3 100644@@ -315,6 +315,18 @@ class LitellmModel(Model):                 "output_tokens_details": usage.output_tokens_details.model_dump(),             } +            # A completion truncated by the token limit before any visible output would+            # otherwise convert to zero output items, indistinguishable from an empty turn.+            if (+                message is not None+                and first_choice is not None+                and getattr(first_choice, "finish_reason", None) == "length"+                and not message.content+                and not getattr(message, "tool_calls", None)+                and not (getattr(message, "provider_specific_fields", None) or {}).get("refusal")+            ):+                raise ChatCmplHelpers.empty_length_truncation_error()+             # Surface content-filter refusals explicitly. Some providers (e.g.             # Anthropic on Amazon Bedrock) signal a safety block only via             # ``finish_reason == "content_filter"`` with an empty message and nosrc/agents/models/chatcmpl_helpers.pyindex d28ddcec..e438af2f 100644@@ -18,6 +18,7 @@ from openai.types.responses.response_text_delta_event import ( ) from pydantic import ValidationError +from ..exceptions import ModelBehaviorError, _mark_error_to_drain_stream_events from ..logger import log_model_action_debug, logger from ..model_settings import ModelSettings from ..version import __version__@@ -40,6 +41,23 @@ HEADERS_OVERRIDE: ContextVar[dict[str, str] | None] = ContextVar(   class ChatCmplHelpers:+    @classmethod+    def empty_length_truncation_error(cls) -> ModelBehaviorError:+        """Error for a completion truncated by the token limit before any visible output.++        A choice with finish_reason="length" and no content, refusal or tool calls would+        otherwise convert to zero output items, indistinguishable from a genuinely empty turn,+        and an agent loop would retry into the same token limit. The Responses API reports the+        same condition as an incomplete response, which the SDK raises on, so match that.+        """+        error = ModelBehaviorError(+            "Chat Completions response was truncated by the token limit "+            "(finish_reason='length') before producing any output. Increase max_tokens; "+            "reasoning models can spend the whole budget on reasoning tokens."+        )+        _mark_error_to_drain_stream_events(error)+        return error+     @classmethod     def is_openai(cls, client: AsyncOpenAI) -> bool:         return is_official_openai_client(client)src/agents/models/chatcmpl_stream_handler.pyindex 51dcfe82..7fc36359 100644@@ -445,11 +445,12 @@ class ChatCmplStreamHandler:                  if has_passthrough_output:                     passthrough_choices.append(choice)-                elif choice.finish_reason == "content_filter":-                    # A content-filtered choice ends the stream with an empty delta, so it-                    # would otherwise be dropped here and the handler would never see the-                    # finish_reason it needs to synthesize the refusal. Forward a-                    # delta-stripped copy so buffering semantics are unchanged.+                elif choice.finish_reason in ("content_filter", "length"):+                    # A content-filtered or token-limit-truncated choice can end the stream+                    # with an empty delta, so it would otherwise be dropped here and the+                    # handler would never see the finish_reason it needs to surface the+                    # refusal or truncation. Forward a delta-stripped copy so buffering+                    # semantics are unchanged.                     passthrough_choices.append(choice.model_copy(update={"delta": ChoiceDelta()}))              if passthrough_choices or chunk.usage is not None:@@ -631,6 +632,9 @@ class ChatCmplStreamHandler:         # empty delta and no refusal field. Track it so we can synthesize an         # explicit refusal after the stream if nothing else was emitted.         saw_content_filter = False+        # A turn truncated by the token limit before any visible output would otherwise+        # complete as an indistinguishable empty turn; raise instead (see below).+        saw_length = False         async for chunk in stream:             if not state.started:                 state.started = True@@ -675,6 +679,8 @@ class ChatCmplStreamHandler:              if choice.finish_reason == "content_filter":                 saw_content_filter = True+            elif choice.finish_reason == "length":+                saw_length = True              if not choice.delta:                 continue@@ -1117,6 +1123,14 @@ class ChatCmplStreamHandler:                             sequence_number=sequence_number.get_and_increment(),                         ) +        if (+            saw_length+            and state.text_content_index_and_output is None+            and state.refusal_content_index_and_output is None+            and not state.function_calls+        ):+            raise ChatCmplHelpers.empty_length_truncation_error()+         # Content-filter refusal with no emitted output: synthesize a refusal so         # the completed response carries a ResponseOutputRefusal rather than an         # empty turn. Only when nothing else was produced (text / refusal / toolsrc/agents/models/openai_chatcompletions.pyindex c5e45091..7bf5afa7 100644@@ -326,6 +326,18 @@ class OpenAIChatCompletionsModel(Model):                 "output_tokens_details": usage.output_tokens_details.model_dump(),             } +            # A completion truncated by the token limit before any visible output would+            # otherwise convert to zero output items, indistinguishable from an empty turn.+            if (+                message is not None+                and first_choice is not None+                and first_choice.finish_reason == "length"+                and not message.content+                and not message.refusal+                and not message.tool_calls+            ):+                raise ChatCmplHelpers.empty_length_truncation_error()+             # Build provider_data for provider_specific_fields             provider_data = {"model": self.model}             if message is not None and hasattr(response, "id"):tests/models/test_any_llm_model.pyindex b80f7c03..81e616a8 100644@@ -583,6 +583,59 @@ async def test_any_llm_chat_path_surfaces_content_filter_refusal(monkeypatch) ->     assert refusals[0].refusal  +@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+@pytest.mark.parametrize("content", [None, ""])+async def test_any_llm_chat_path_raises_on_empty_length_truncation(+    monkeypatch, content: str | None+) -> None:+    """A turn truncated by the token limit before any output must not become an empty output."""+    completion = _chat_completion("")+    completion.choices[0].message.content = content+    completion.choices[0].finish_reason = "length"+    provider = FakeAnyLLMProvider(supports_responses=False, chat_response=completion)+    module, _create_calls = _import_any_llm_module(monkeypatch, provider)++    model = module.AnyLLMModel(model="openrouter/openai/gpt-5.4-mini")+    with pytest.raises(ModelBehaviorError, match="finish_reason='length'"):+        await model.get_response(+            system_instructions=None,+            input="hi",+            model_settings=ModelSettings(),+            tools=[],+            output_schema=None,+            handoffs=[],+            tracing=ModelTracing.DISABLED,+            previous_response_id=None,+            conversation_id=None,+            prompt=None,+        )+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_any_llm_chat_stream_raises_on_empty_length_truncation(monkeypatch) -> None:+    stream = _ClosableStream(+        [+            _chat_chunk(""),+            ChatCompletionChunk(+                id="chunk_123",+                created=0,+                model="fake-model",+                object="chat.completion.chunk",+                choices=[ChunkChoice(index=0, delta=ChoiceDelta(), finish_reason="length")],+            ),+        ]+    )+    provider = FakeAnyLLMProvider(supports_responses=False, chat_response=stream)+    module, _create_calls = _import_any_llm_module(monkeypatch, provider)+    model = module.AnyLLMModel(model="openrouter/openai/gpt-5.4-mini")++    with pytest.raises(ModelBehaviorError, match="finish_reason='length'"):+        async for _event in _stream_events(model):+            pass++ @pytest.mark.allow_call_model_methods @pytest.mark.asyncio async def test_any_llm_chat_path_content_filter_keeps_real_content(monkeypatch) -> None:tests/models/test_litellm_chatcompletions_stream.pyindex ece71165..5310daf4 100644@@ -27,6 +27,7 @@ from openai.types.responses import (     ResponseRefusalDeltaEvent, ) +from agents.exceptions import ModelBehaviorError from agents.extensions.models.litellm_model import LitellmModel from agents.extensions.models.litellm_provider import LitellmProvider from agents.items import TResponseStreamEvent@@ -1037,3 +1038,61 @@ async def test_stream_response_lets_in_flight_close_finish_after_cancellation(     finally:         release.set()         task.cancel()+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_stream_response_raises_on_empty_length_truncation(monkeypatch) -> None:+    """A stream truncated by the token limit before any visible output must not+    complete as an indistinguishable empty turn."""+    chunks = [+        ChatCompletionChunk(+            id="chunk-id",+            created=1,+            model="fake",+            object="chat.completion.chunk",+            choices=[Choice(index=0, delta=ChoiceDelta(role="assistant", content=""))],+        ),+        ChatCompletionChunk(+            id="chunk-id",+            created=1,+            model="fake",+            object="chat.completion.chunk",+            choices=[Choice(index=0, delta=ChoiceDelta(), finish_reason="length")],+            usage=CompletionUsage(completion_tokens=16, prompt_tokens=7, total_tokens=23),+        ),+    ]++    async def fake_stream() -> AsyncIterator[ChatCompletionChunk]:+        for c in chunks:+            yield c++    async def patched_fetch_response(self, *args, **kwargs):+        resp = Response(+            id="resp-id",+            created_at=0,+            model="fake-model",+            object="response",+            output=[],+            tool_choice="none",+            tools=[],+            parallel_tool_calls=False,+        )+        return resp, fake_stream()++    monkeypatch.setattr(LitellmModel, "_fetch_response", patched_fetch_response)+    model = LitellmProvider().get_model("gpt-4")+    with pytest.raises(ModelBehaviorError, match="finish_reason='length'"):+        async for _event in model.stream_response(+            system_instructions=None,+            input="",+            model_settings=ModelSettings(),+            tools=[],+            output_schema=None,+            handoffs=[],+            tracing=ModelTracing.DISABLED,+            previous_response_id=None,+            conversation_id=None,+            prompt=None,+        ):+            passtests/models/test_litellm_content_filter.pyindex 7b77a955..7431e8c7 100644@@ -3,6 +3,7 @@ import pytest from litellm.types.utils import Choices, Message, ModelResponse, Usage from openai.types.responses import ResponseOutputMessage, ResponseOutputRefusal +from agents.exceptions import ModelBehaviorError from agents.extensions.models.litellm_model import LitellmModel from agents.model_settings import ModelSettings from agents.models.interface import ModelTracing@@ -87,3 +88,30 @@ async def test_normal_stop_is_unaffected(monkeypatch):         if isinstance(content, ResponseOutputRefusal)     ]     assert not refusals+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+@pytest.mark.parametrize("content", [None, ""])+async def test_empty_length_truncation_raises(monkeypatch, content):+    """A completion truncated by the token limit before any visible output must not+    come back as zero output items, indistinguishable from an empty turn."""+    with pytest.raises(ModelBehaviorError, match="finish_reason='length'"):+        await _get_response(monkeypatch, finish_reason="length", content=content)+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_partial_length_truncation_keeps_content(monkeypatch):+    resp = await _get_response(monkeypatch, finish_reason="length", content="partial")++    assert isinstance(resp.output[0], ResponseOutputMessage)+    assert resp.output[0].content[0].text == "partial"  # type: ignore[union-attr]+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_empty_stop_is_an_empty_turn(monkeypatch):+    resp = await _get_response(monkeypatch, finish_reason="stop", content="")++    assert resp.output == []tests/models/test_openai_chatcompletions.pyindex b8198e6e..4c0d395a 100644@@ -50,7 +50,7 @@ from agents import (     generation_span,     trace, )-from agents.exceptions import UserError+from agents.exceptions import ModelBehaviorError, UserError from agents.models._retry_runtime import provider_managed_retries_disabled from agents.models.chatcmpl_helpers import HEADERS_OVERRIDE, ChatCmplHelpers from agents.models.fake_id import FAKE_RESPONSES_ID@@ -426,6 +426,85 @@ async def test_get_response_preserves_empty_nonfiltered_output(monkeypatch) -> N     assert resp.output == []  +@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+@pytest.mark.parametrize("content", [None, ""])+async def test_get_response_raises_on_empty_length_truncation(+    monkeypatch, content: str | None+) -> None:+    """A completion truncated by the token limit before any visible output must not+    come back as zero output items, indistinguishable from an empty turn."""+    with pytest.raises(ModelBehaviorError, match="finish_reason='length'"):+        await _get_response_for_choice(+            monkeypatch,+            Choice(+                index=0,+                finish_reason="length",+                message=ChatCompletionMessage(role="assistant", content=content),+            ),+        )+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_get_response_preserves_partial_length_truncated_output(monkeypatch) -> None:+    resp = await _get_response_for_choice(+        monkeypatch,+        Choice(+            index=0,+            finish_reason="length",+            message=ChatCompletionMessage(role="assistant", content="partial"),+        ),+    )++    assert len(resp.output) == 1+    assert isinstance(resp.output[0], ResponseOutputMessage)+    assert isinstance(resp.output[0].content[0], ResponseOutputText)+    assert resp.output[0].content[0].text == "partial"+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_get_response_preserves_empty_string_stop_output(monkeypatch) -> None:+    resp = await _get_response_for_choice(+        monkeypatch,+        Choice(+            index=0,+            finish_reason="stop",+            message=ChatCompletionMessage(role="assistant", content=""),+        ),+    )++    assert resp.output == []+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_runner_raises_on_empty_length_truncation(monkeypatch) -> None:+    chat = ChatCompletion(+        id="resp-id",+        created=0,+        model="fake",+        object="chat.completion",+        choices=[+            Choice(+                index=0,+                finish_reason="length",+                message=ChatCompletionMessage(role="assistant", content=""),+            )+        ],+    )++    async def patched_fetch_response(self, *args, **kwargs):+        return chat++    monkeypatch.setattr(OpenAIChatCompletionsModel, "_fetch_response", patched_fetch_response)+    agent = Agent(name="test", model=OpenAIProvider(use_responses=False).get_model("gpt-4"))++    with pytest.raises(ModelBehaviorError, match="finish_reason='length'"):+        await Runner.run(agent, "hi")++ @pytest.mark.allow_call_model_methods @pytest.mark.asyncio @pytest.mark.parametrize(tests/models/test_openai_chatcompletions_stream.pyindex 1cced004..892725cc 100644@@ -3780,6 +3780,111 @@ async def test_buffer_tool_call_stream_forwards_content_filter_finish_reason() -     assert not ChatCmplStreamHandler._delta_has_passthrough_output(terminal_choices[0].delta)  +@pytest.mark.asyncio+async def test_buffer_tool_call_stream_forwards_length_finish_reason() -> None:+    chunks = [+        _chunk_with([Choice(index=0, delta=ChoiceDelta(content=""))]),+        _chunk_with([Choice(index=0, delta=ChoiceDelta(), finish_reason="length")]),+    ]++    async def source() -> AsyncIterator[ChatCompletionChunk]:+        for chunk in chunks:+            yield chunk++    buffered = [c async for c in ChatCmplStreamHandler.buffer_tool_call_stream(source())]++    terminal_choices = [+        choice for chunk in buffered for choice in chunk.choices if choice.finish_reason == "length"+    ]+    assert len(terminal_choices) == 1+    assert not ChatCmplStreamHandler._delta_has_passthrough_output(terminal_choices[0].delta)+++def _empty_length_truncated_chunks(*, with_reasoning: bool) -> list[ChatCompletionChunk]:+    chunks: list[ChatCompletionChunk] = []+    if with_reasoning:+        reasoning_delta = ChoiceDelta(role="assistant", content=None)+        # reasoning_content is a provider extra field the handler reads via hasattr.+        reasoning_delta.reasoning_content = "thinking..."  # type: ignore[attr-defined]+        chunks.append(_chunk_with([Choice(index=0, delta=reasoning_delta)]))+    chunks.append(_chunk_with([Choice(index=0, delta=ChoiceDelta(role="assistant", content=""))]))+    chunks.append(+        _chunk_with(+            [Choice(index=0, delta=ChoiceDelta(), finish_reason="length")],+            usage=CompletionUsage(completion_tokens=16, prompt_tokens=7, total_tokens=23),+        )+    )+    return chunks+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+@pytest.mark.parametrize("with_reasoning", [False, True])+async def test_stream_handler_raises_on_empty_length_truncation(with_reasoning: bool) -> None:+    """A stream truncated by the token limit before any visible output must not+    complete as an indistinguishable empty turn."""+    with pytest.raises(ModelBehaviorError, match="finish_reason='length'"):+        await _collect_handler_events(+            *_empty_length_truncated_chunks(with_reasoning=with_reasoning)+        )+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_buffered_stream_raises_on_empty_length_truncation(monkeypatch) -> None:+    with pytest.raises(ModelBehaviorError, match="finish_reason='length'"):+        await _buffered_stream_events(+            monkeypatch, _empty_length_truncated_chunks(with_reasoning=False)+        )+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_stream_handler_preserves_partial_length_truncated_text() -> None:+    events = await _collect_handler_events(+        _chunk_with([Choice(index=0, delta=ChoiceDelta(content="partial"))]),+        _chunk_with([Choice(index=0, delta=ChoiceDelta(), finish_reason="length")]),+    )++    completed_event = events[-1]+    assert isinstance(completed_event, ResponseCompletedEvent)+    assistant_msg = completed_event.response.output[0]+    assert isinstance(assistant_msg, ResponseOutputMessage)+    text_part = assistant_msg.content[0]+    assert isinstance(text_part, ResponseOutputText)+    assert text_part.text == "partial"+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_stream_handler_completes_empty_stop_turn() -> None:+    events = await _collect_handler_events(+        _chunk_with([Choice(index=0, delta=ChoiceDelta(role="assistant", content=""))]),+        _chunk_with([Choice(index=0, delta=ChoiceDelta(), finish_reason="stop")]),+    )++    completed_event = events[-1]+    assert isinstance(completed_event, ResponseCompletedEvent)+    assert completed_event.response.output == []+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_streamed_runner_raises_on_empty_length_truncation(monkeypatch) -> None:+    async def patched_fetch_response(self, *args, **kwargs):+        return _empty_response(), _completion_stream(+            *_empty_length_truncated_chunks(with_reasoning=False)+        )++    monkeypatch.setattr(OpenAIChatCompletionsModel, "_fetch_response", patched_fetch_response)+    agent = Agent(name="test", model=OpenAIProvider(use_responses=False).get_model("gpt-4"))++    result = Runner.run_streamed(agent, "hi")+    with pytest.raises(ModelBehaviorError, match="finish_reason='length'"):+        async for _ in result.stream_events():+            pass++ @pytest.mark.asyncio async def test_buffer_tool_call_stream_does_not_duplicate_tool_calls_finish() -> None:     """finish_reason == "tool_calls" is still emitted only by the synthesized 
Merged fix · PR #4513+550 −36
src/agents/models/chatcmpl_stream_handler.pyindex 51dcfe825..289e116b0 100644@@ -445,11 +445,11 @@ class ChatCmplStreamHandler:                  if has_passthrough_output:                     passthrough_choices.append(choice)-                elif choice.finish_reason == "content_filter":-                    # A content-filtered choice ends the stream with an empty delta, so it-                    # would otherwise be dropped here and the handler would never see the-                    # finish_reason it needs to synthesize the refusal. Forward a-                    # delta-stripped copy so buffering semantics are unchanged.+                elif choice.finish_reason in {"content_filter", "length"}:+                    # A content-filtered or truncated choice ends the stream with an empty+                    # delta, so it would otherwise be dropped here and the handler would+                    # never see the finish_reason it needs to act on.+                    # Forward a delta-stripped copy so buffering semantics are unchanged.                     passthrough_choices.append(choice.model_copy(update={"delta": ChoiceDelta()}))              if passthrough_choices or chunk.usage is not None:@@ -609,6 +609,7 @@ class ChatCmplStreamHandler:         model: str | None = None,         strict_feature_validation: bool = False,         preserve_raw_usage: bool = False,+        raise_on_length_truncation: bool = False,     ) -> AsyncIterator[TResponseStreamEvent]:         """         Handle a streaming chat completion response and yield response events.@@ -620,6 +621,11 @@ class ChatCmplStreamHandler:                 provider-specific stream processing.             preserve_raw_usage: Whether to retain the last provider usage payload before                 converting it to the Responses usage shape.+            raise_on_length_truncation: Whether to raise ModelBehaviorError when the+                stream terminates with finish_reason == "length" and no visible output.+                This is an internal option enabled only by OpenAIChatCompletionsModel:+                the shared handler also serves LiteLLM and AnyLLM, whose streaming+                behavior must remain unchanged.         """         usage: CompletionUsage | None = None         raw_usage: dict[str, Any] | None = None@@ -630,7 +636,11 @@ class ChatCmplStreamHandler:         # safety block only through finish_reason == "content_filter" with an         # empty delta and no refusal field. Track it so we can synthesize an         # explicit refusal after the stream if nothing else was emitted.+        # A completion truncated before any visible token (finish_reason ==+        # "length") has the same shape; track it too so we can surface a model+        # behavior error instead of collapsing into an empty turn.         saw_content_filter = False+        saw_length = False         async for chunk in stream:             if not state.started:                 state.started = True@@ -675,6 +685,8 @@ class ChatCmplStreamHandler:              if choice.finish_reason == "content_filter":                 saw_content_filter = True+            elif choice.finish_reason == "length":+                saw_length = True              if not choice.delta:                 continue@@ -1174,6 +1186,34 @@ class ChatCmplStreamHandler:                 sequence_number=sequence_number.get_and_increment(),             ) +        # A completion truncated before any visible token (finish_reason ==+        # "length") is a token- or reasoning-budget exhaustion, not a policy+        # refusal. Surface it as a model behavior error rather than manufacturing+        # a refusal that would route through model_refusal handlers. This is an+        # internal behavior enabled only by OpenAIChatCompletionsModel: the+        # shared handler also serves LiteLLM and AnyLLM, whose streaming behavior+        # must remain unchanged.+        if (+            saw_length+            and raise_on_length_truncation+            and state.text_content_index_and_output is None+            and state.refusal_content_index_and_output is None+            and not state.function_calls+        ):+            # Preserve the request and any reported token usage on the response+            # so the caller can attach it to the generation span before the+            # error propagates.+            if usage is not None:+                response.usage = cls._build_response_usage(usage)+            else:+                _mark_request_completed_without_usage(response)+            if preserve_raw_usage and raw_usage is not None:+                _attach_raw_usage_snapshot(response, raw_usage)+            raise ModelBehaviorError(+                "Chat Completions stream terminated with finish_reason='length' "+                "but produced no assistant text, tool call, or refusal."+            )+         cls._finalize_thinking_blocks(state)         for event in cls._finish_reasoning_item(state, sequence_number):             yield event@@ -1294,27 +1334,7 @@ class ChatCmplStreamHandler:         final_response = response.model_copy()         final_response.output = outputs -        final_response.usage = (-            ResponseUsage(-                input_tokens=usage.prompt_tokens or 0,-                output_tokens=usage.completion_tokens or 0,-                total_tokens=usage.total_tokens or 0,-                output_tokens_details=OutputTokensDetails(-                    reasoning_tokens=usage.completion_tokens_details.reasoning_tokens-                    if usage.completion_tokens_details-                    and usage.completion_tokens_details.reasoning_tokens-                    else 0-                ),-                input_tokens_details=_make_input_tokens_details(-                    cached_tokens=usage.prompt_tokens_details.cached_tokens-                    if usage.prompt_tokens_details and usage.prompt_tokens_details.cached_tokens-                    else 0,-                    cache_write_tokens=_cache_write_tokens(usage.prompt_tokens_details),-                ),-            )-            if usage-            else None-        )+        final_response.usage = cls._build_response_usage(usage)         if preserve_raw_usage:             _attach_raw_usage_snapshot(final_response, raw_usage)         if usage is None:@@ -1328,3 +1348,26 @@ class ChatCmplStreamHandler:             type="response.completed",             sequence_number=sequence_number.get_and_increment(),         )++    @staticmethod+    def _build_response_usage(usage: CompletionUsage | None) -> ResponseUsage | None:+        """Convert the streamed provider usage into a Responses usage payload."""+        if usage is None:+            return None+        return ResponseUsage(+            input_tokens=usage.prompt_tokens or 0,+            output_tokens=usage.completion_tokens or 0,+            total_tokens=usage.total_tokens or 0,+            output_tokens_details=OutputTokensDetails(+                reasoning_tokens=usage.completion_tokens_details.reasoning_tokens+                if usage.completion_tokens_details+                and usage.completion_tokens_details.reasoning_tokens+                else 0+            ),+            input_tokens_details=_make_input_tokens_details(+                cached_tokens=usage.prompt_tokens_details.cached_tokens+                if usage.prompt_tokens_details and usage.prompt_tokens_details.cached_tokens+                else 0,+                cache_write_tokens=_cache_write_tokens(usage.prompt_tokens_details),+            ),+        )src/agents/models/openai_chatcompletions.pyindex c5e450912..83d261826 100644@@ -300,6 +300,18 @@ class OpenAIChatCompletionsModel(Model):                 else Usage(requests=1)             ) +            # Record the request and token usage on the span before the terminal+            # branches below, so a ModelBehaviorError raised for a truncated empty+            # completion still leaves the request and usage accounted for.+            span_generation.span_data.usage = {+                "requests": usage.requests,+                "input_tokens": usage.input_tokens,+                "output_tokens": usage.output_tokens,+                "total_tokens": usage.total_tokens,+                "input_tokens_details": usage.input_tokens_details.model_dump(),+                "output_tokens_details": usage.output_tokens_details.model_dump(),+            }+             # Some providers signal a filtered non-streaming completion only through             # finish_reason="content_filter" and an otherwise empty message. Preserve             # that terminal signal as a refusal instead of returning an empty output.@@ -313,18 +325,27 @@ class OpenAIChatCompletionsModel(Model):             ):                 message.refusal = "Response withheld by the provider's content filter." +            # A completion truncated before any visible token (finish_reason="length")+            # is a token- or reasoning-budget exhaustion, not a policy refusal.+            # Surface it as a model behavior error rather than manufacturing a+            # refusal that would route through model_refusal handlers.+            if (+                message is not None+                and first_choice is not None+                and first_choice.finish_reason == "length"+                and not message.content+                and not message.refusal+                and not message.tool_calls+            ):+                raise ModelBehaviorError(+                    "Chat Completions response terminated with finish_reason='length' "+                    "but produced no assistant text, tool call, or refusal."+                )+             if tracing.include_data():                 span_generation.span_data.output = (                     [message.model_dump()] if message is not None else []                 )-            span_generation.span_data.usage = {-                "requests": usage.requests,-                "input_tokens": usage.input_tokens,-                "output_tokens": usage.output_tokens,-                "total_tokens": usage.total_tokens,-                "input_tokens_details": usage.input_tokens_details.model_dump(),-                "output_tokens_details": usage.output_tokens_details.model_dump(),-            }              # Build provider_data for provider_specific_fields             provider_data = {"model": self.model}@@ -470,6 +491,7 @@ class OpenAIChatCompletionsModel(Model):                     cast(AsyncStream[ChatCompletionChunk], stream_for_handler),                     model=self.model,                     strict_feature_validation=self._strict_feature_validation,+                    raise_on_length_truncation=True,                     **raw_usage_options,                 ):                     if chunk.type == "response.completed":@@ -483,6 +505,12 @@ class OpenAIChatCompletionsModel(Model):                         )                      yield chunk+            except ModelBehaviorError:+                # The handler preserves the request and any reported token usage on the+                # base response before raising (e.g. a token-budget-exhausted empty+                # completion). Attach it to the span before the error surfaces.+                self._populate_stream_generation_span(span_generation, response, tracing)+                raise             except asyncio.CancelledError:                 close_stream_in_background = True                 self._schedule_async_iterator_close(stream)tests/models/test_openai_chatcompletions.pyindex b8198e6ec..c3d196b59 100644@@ -50,7 +50,7 @@ from agents import (     generation_span,     trace, )-from agents.exceptions import UserError+from agents.exceptions import ModelBehaviorError, UserError from agents.models._retry_runtime import provider_managed_retries_disabled from agents.models.chatcmpl_helpers import HEADERS_OVERRIDE, ChatCmplHelpers from agents.models.fake_id import FAKE_RESPONSES_ID@@ -426,6 +426,171 @@ async def test_get_response_preserves_empty_nonfiltered_output(monkeypatch) -> N     assert resp.output == []  +@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_get_response_raises_on_truncated_empty_turn(monkeypatch) -> None:+    with pytest.raises(ModelBehaviorError, match="finish_reason='length'"):+        await _get_response_for_choice(+            monkeypatch,+            Choice(+                index=0,+                finish_reason="length",+                message=ChatCompletionMessage(role="assistant", content=None),+            ),+        )+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_get_response_raises_on_truncated_empty_string_turn(monkeypatch) -> None:+    with pytest.raises(ModelBehaviorError, match="finish_reason='length'"):+        await _get_response_for_choice(+            monkeypatch,+            Choice(+                index=0,+                finish_reason="length",+                message=ChatCompletionMessage(role="assistant", content=""),+            ),+        )+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_get_response_traces_error_on_truncated_empty_turn(monkeypatch) -> None:+    with trace(workflow_name="truncation-error"):+        with pytest.raises(ModelBehaviorError, match="finish_reason='length'"):+            await _get_response_for_choice(+                monkeypatch,+                Choice(+                    index=0,+                    finish_reason="length",+                    message=ChatCompletionMessage(role="assistant", content=None),+                ),+                tracing=ModelTracing.ENABLED,+            )++    generation_spans = [+        span for span in fetch_ordered_spans() if span.span_data.type == "generation"+    ]+    assert len(generation_spans) == 1+    generation = generation_spans[0]+    exported_span = generation.export()+    assert exported_span is not None+    assert exported_span["error"] is not None+    # The request (and any reported tokens) must be preserved on the span even though+    # the call raised, so the run's usage accounting does not lose the request.+    assert generation.span_data.usage is not None+    assert generation.span_data.usage["requests"] == 1+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_get_response_traces_usage_on_truncated_empty_turn(monkeypatch) -> None:+    """The token usage reported before a truncated empty completion raises must be+    preserved on the generation span, alongside the request."""+    chat = ChatCompletion(+        id="resp-id",+        created=0,+        model="fake",+        object="chat.completion",+        choices=[+            Choice(+                index=0,+                finish_reason="length",+                message=ChatCompletionMessage(role="assistant", content=None),+            )+        ],+        usage=CompletionUsage(+            completion_tokens=0,+            prompt_tokens=7,+            total_tokens=7,+            prompt_tokens_details=PromptTokensDetails(cached_tokens=2),+        ),+    )++    async def patched_fetch_response(self, *args, **kwargs):+        return chat++    monkeypatch.setattr(OpenAIChatCompletionsModel, "_fetch_response", patched_fetch_response)+    model = OpenAIProvider(use_responses=False).get_model("gpt-4")++    with trace(workflow_name="truncation-usage"):+        with pytest.raises(ModelBehaviorError, match="finish_reason='length'"):+            await model.get_response(+                system_instructions=None,+                input="",+                model_settings=ModelSettings(),+                tools=[],+                output_schema=None,+                handoffs=[],+                tracing=ModelTracing.ENABLED,+                previous_response_id=None,+                conversation_id=None,+                prompt=None,+            )++    generation = next(span for span in fetch_ordered_spans() if span.span_data.type == "generation")+    assert generation.span_data.usage is not None+    assert generation.span_data.usage["requests"] == 1+    assert generation.span_data.usage["input_tokens"] == 7+    assert generation.span_data.usage["total_tokens"] == 7+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+@pytest.mark.parametrize(+    ("message", "expected_output_type", "expected_content_type"),+    [+        (+            ChatCompletionMessage(role="assistant", content="partial"),+            ResponseOutputMessage,+            ResponseOutputText,+        ),+        (+            ChatCompletionMessage(role="assistant", content=None, refusal="provider refusal"),+            ResponseOutputMessage,+            ResponseOutputRefusal,+        ),+        (+            ChatCompletionMessage(+                role="assistant",+                content=None,+                tool_calls=[+                    ChatCompletionMessageFunctionToolCall(+                        id="call-1",+                        type="function",+                        function=Function(name="do_thing", arguments="{}"),+                    )+                ],+            ),+            ResponseFunctionToolCall,+            None,+        ),+    ],+)+async def test_get_response_preserves_nonempty_truncated_output(+    monkeypatch,+    message: ChatCompletionMessage,+    expected_output_type: type[object],+    expected_content_type: type[object] | None,+) -> None:+    resp = await _get_response_for_choice(+        monkeypatch,+        Choice(index=0, finish_reason="length", message=message),+    )++    assert len(resp.output) == 1+    assert isinstance(resp.output[0], expected_output_type)+    if expected_content_type is not None:+        assert isinstance(resp.output[0], ResponseOutputMessage)+        assert len(resp.output[0].content) == 1+        assert isinstance(resp.output[0].content[0], expected_content_type)+    if isinstance(resp.output[0], ResponseOutputMessage) and isinstance(+        resp.output[0].content[0], ResponseOutputRefusal+    ):+        assert resp.output[0].content[0].refusal == "provider refusal"++ @pytest.mark.allow_call_model_methods @pytest.mark.asyncio @pytest.mark.parametrize(tests/models/test_openai_chatcompletions_stream.pyindex 1cced004e..5c7949376 100644@@ -83,11 +83,15 @@ async def _completion_stream( async def _collect_handler_events(     *chunks: ChatCompletionChunk,     model: str | None = None,+    raise_on_length_truncation: bool = False, ) -> list[Any]:     return [         event         async for event in ChatCmplStreamHandler.handle_stream(-            _empty_response(), cast(Any, _completion_stream(*chunks)), model=model+            _empty_response(),+            cast(Any, _completion_stream(*chunks)),+            model=model,+            raise_on_length_truncation=raise_on_length_truncation,         )     ] @@ -3780,6 +3784,149 @@ async def test_buffer_tool_call_stream_forwards_content_filter_finish_reason() -     assert not ChatCmplStreamHandler._delta_has_passthrough_output(terminal_choices[0].delta)  +@pytest.mark.asyncio+async def test_handler_stream_raises_on_length_truncation() -> None:+    """A stream that terminates with finish_reason == "length" and no emitted+    content must raise ModelBehaviorError instead of manufacturing a refusal,+    matching the non-streaming path, when the internal option is enabled."""+    terminal = ChatCompletionChunk(+        id="chunk-id",+        created=1,+        model="fake",+        object="chat.completion.chunk",+        choices=[Choice(index=0, delta=ChoiceDelta(), finish_reason="length")],+    )++    with pytest.raises(ModelBehaviorError, match="finish_reason='length'"):+        await _collect_handler_events(terminal, raise_on_length_truncation=True)+++@pytest.mark.asyncio+async def test_handler_stream_length_is_ignored_without_internal_option() -> None:+    """Without the internal option, a length terminal with no output must keep the+    released behavior: no error and no refusal synthesis (an empty turn).++    The shared handler also serves LiteLLM and AnyLLM, so the length-error behavior+    must be opt-in rather than changing those providers' streaming output.+    """+    terminal = ChatCompletionChunk(+        id="chunk-id",+        created=1,+        model="fake",+        object="chat.completion.chunk",+        choices=[Choice(index=0, delta=ChoiceDelta(), finish_reason="length")],+    )++    output_events = await _collect_handler_events(terminal)++    assert "response.refusal.delta" not in [e.type for e in output_events]+    completed_event = output_events[-1]+    assert isinstance(completed_event, ResponseCompletedEvent)+    assert completed_event.response.output == []+++@pytest.mark.asyncio+async def test_handler_stream_length_does_not_clobber_text() -> None:+    """A length finish_reason arriving after real text was streamed must not+    synthesize a refusal."""+    chunk1 = _chunk_with([Choice(index=0, delta=ChoiceDelta(content="answer"))])+    chunk2 = _chunk_with([Choice(index=0, delta=ChoiceDelta(), finish_reason="length")])++    output_events = await _collect_handler_events(chunk1, chunk2, raise_on_length_truncation=True)++    assert "response.refusal.delta" not in [e.type for e in output_events]+    completed_event = output_events[-1]+    assert isinstance(completed_event, ResponseCompletedEvent)+    assistant_msg = completed_event.response.output[0]+    assert isinstance(assistant_msg, ResponseOutputMessage)+    text_part = assistant_msg.content[0]+    assert isinstance(text_part, ResponseOutputText)+    assert text_part.text == "answer"+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_buffered_stream_raises_on_length_truncation(monkeypatch) -> None:+    """With tool-call buffering enabled, a stream that terminates with+    finish_reason == "length" and no emitted content must still raise+    ModelBehaviorError, mirroring the non-buffered behavior."""+    chunk1 = ChatCompletionChunk(+        id="chunk-id",+        created=1,+        model="fake",+        object="chat.completion.chunk",+        choices=[Choice(index=0, delta=ChoiceDelta(role="assistant", content=""))],+    )+    chunk2 = ChatCompletionChunk(+        id="chunk-id",+        created=1,+        model="fake",+        object="chat.completion.chunk",+        choices=[Choice(index=0, delta=ChoiceDelta(), finish_reason="length")],+        usage=CompletionUsage(completion_tokens=0, prompt_tokens=7, total_tokens=7),+    )++    with pytest.raises(ModelBehaviorError, match="finish_reason='length'"):+        await _buffered_stream_events(monkeypatch, [chunk1, chunk2])+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_buffered_stream_length_does_not_clobber_text(monkeypatch) -> None:+    """A length finish_reason arriving after real text was streamed must not+    synthesize a refusal, even with buffering enabled."""+    chunk1 = ChatCompletionChunk(+        id="chunk-id",+        created=1,+        model="fake",+        object="chat.completion.chunk",+        choices=[Choice(index=0, delta=ChoiceDelta(content="answer"))],+    )+    chunk2 = ChatCompletionChunk(+        id="chunk-id",+        created=1,+        model="fake",+        object="chat.completion.chunk",+        choices=[Choice(index=0, delta=ChoiceDelta(), finish_reason="length")],+        usage=CompletionUsage(completion_tokens=1, prompt_tokens=7, total_tokens=8),+    )++    output_events = await _buffered_stream_events(monkeypatch, [chunk1, chunk2])++    assert "response.refusal.delta" not in [e.type for e in output_events]+    completed_event = output_events[-1]+    assert isinstance(completed_event, ResponseCompletedEvent)+    assistant_msg = completed_event.response.output[0]+    assert isinstance(assistant_msg, ResponseOutputMessage)+    text_part = assistant_msg.content[0]+    assert isinstance(text_part, ResponseOutputText)+    assert text_part.text == "answer"+++@pytest.mark.asyncio+async def test_buffer_tool_call_stream_forwards_length_finish_reason() -> None:+    """The buffering layer must forward a length-truncated terminal choice even+    though its delta is empty, so the finish_reason reaches the handler instead+    of being swallowed (mirrors the content_filter forwarding behavior)."""+    chunks = [+        _chunk_with([Choice(index=0, delta=ChoiceDelta(content=""))]),+        _chunk_with([Choice(index=0, delta=ChoiceDelta(), finish_reason="length")]),+    ]++    async def source() -> AsyncIterator[ChatCompletionChunk]:+        for chunk in chunks:+            yield chunk++    buffered = [c async for c in ChatCmplStreamHandler.buffer_tool_call_stream(source())]++    terminal_choices = [+        choice for chunk in buffered for choice in chunk.choices if choice.finish_reason == "length"+    ]+    assert len(terminal_choices) == 1+    # The forwarded copy carries no delta output.+    assert not ChatCmplStreamHandler._delta_has_passthrough_output(terminal_choices[0].delta)++ @pytest.mark.asyncio async def test_buffer_tool_call_stream_does_not_duplicate_tool_calls_finish() -> None:     """finish_reason == "tool_calls" is still emitted only by the synthesized@@ -4214,3 +4361,134 @@ async def test_stream_span_is_recorded_for_a_consumer_that_stops_at_the_terminal     generation = next(s for s in fetch_ordered_spans() if s.span_data.type == "generation")     assert generation.span_data.usage is not None     assert generation.span_data.usage["requests"] == 1+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_streamed_span_preserves_usage_on_length_truncation(monkeypatch) -> None:+    """A stream terminated by finish_reason == "length" with no output raises+    ModelBehaviorError, but the request and reported token usage must still be+    preserved on the generation span before the error surfaces."""++    def _length_stream_patch():+        chunk = ChatCompletionChunk(+            id="chunk-id",+            created=1,+            model="fake",+            object="chat.completion.chunk",+            choices=[Choice(index=0, delta=ChoiceDelta(), finish_reason="length")],+            usage=CompletionUsage(+                completion_tokens=0,+                prompt_tokens=7,+                total_tokens=7,+                prompt_tokens_details=PromptTokensDetails(cached_tokens=2),+            ),+        )++        async def fake_stream() -> AsyncIterator[ChatCompletionChunk]:+            yield chunk++        async def patched_fetch_response(self, *args, **kwargs):+            resp = Response(+                id="resp-id",+                created_at=0,+                model="fake-model",+                object="response",+                output=[],+                tool_choice="none",+                tools=[],+                parallel_tool_calls=False,+            )+            return resp, fake_stream()++        return patched_fetch_response++    monkeypatch.setattr(OpenAIChatCompletionsModel, "_fetch_response", _length_stream_patch())+    model = OpenAIProvider(use_responses=False).get_model("gpt-4")++    with trace(workflow_name="stream-length-truncation"):+        with pytest.raises(ModelBehaviorError, match="finish_reason='length'"):+            async for _ in model.stream_response(+                system_instructions=None,+                input="",+                model_settings=ModelSettings(),+                tools=[],+                output_schema=None,+                handoffs=[],+                tracing=ModelTracing.ENABLED,+                previous_response_id=None,+                conversation_id=None,+                prompt=None,+            ):+                pass++    generation = next(s for s in fetch_ordered_spans() if s.span_data.type == "generation")+    exported_span = generation.export()+    assert exported_span is not None+    assert exported_span["error"] is not None+    assert generation.span_data.usage is not None+    assert generation.span_data.usage["requests"] == 1+    assert generation.span_data.usage["input_tokens"] == 7+    assert generation.span_data.usage["total_tokens"] == 7+++@pytest.mark.allow_call_model_methods+@pytest.mark.asyncio+async def test_streamed_span_counts_request_on_length_truncation_without_usage(+    monkeypatch,+) -> None:+    """When the provider omits usage, a length-truncated empty completion still+    counts the request on the generation span (tokens stay at zero), mirroring+    the non-streaming path."""++    def _length_stream_patch():+        chunk = ChatCompletionChunk(+            id="chunk-id",+            created=1,+            model="fake",+            object="chat.completion.chunk",+            choices=[Choice(index=0, delta=ChoiceDelta(), finish_reason="length")],+            usage=None,+        )++        async def fake_stream() -> AsyncIterator[ChatCompletionChunk]:+            yield chunk++        async def patched_fetch_response(self, *args, **kwargs):+            resp = Response(+                id="resp-id",+                created_at=0,+                model="fake-model",+                object="response",+                output=[],+                tool_choice="none",+                tools=[],+                parallel_tool_calls=False,+            )+            return resp, fake_stream()++        return patched_fetch_response++    monkeypatch.setattr(OpenAIChatCompletionsModel, "_fetch_response", _length_stream_patch())+    model = OpenAIProvider(use_responses=False).get_model("gpt-4")++    with trace(workflow_name="stream-length-no-usage"):+        with pytest.raises(ModelBehaviorError, match="finish_reason='length'"):+            async for _ in model.stream_response(+                system_instructions=None,+                input="",+                model_settings=ModelSettings(),+                tools=[],+                output_schema=None,+                handoffs=[],+                tracing=ModelTracing.ENABLED,+                previous_response_id=None,+                conversation_id=None,+                prompt=None,+            ):+                pass++    generation = next(s for s in fetch_ordered_spans() if s.span_data.type == "generation")+    assert generation.span_data.usage is not None+    assert generation.span_data.usage["requests"] == 1+    assert generation.span_data.usage["total_tokens"] == 0