Backtestsopenai-agents-pythonrun-20261024-184640#5275

Issue #5275 Merged fix PR #5276 Run 24 Oct 2026 · run-20261024-184640 openai/openai-agents-python

gpt-4o-transcribe is being retired (config.py)

Better

Our patch was judged better than the merged fix.

In this judgment, Patch A = our agent, Patch B = the merged fix.

Both patches make the same core change, switching DEFAULT_STT_MODEL in src/agents/voice/models/openai_model_provider.py from gpt-4o-transcribe to gpt-transcribe, and all four independent tests go from FAIL-ASSERT on base to PASS on both, so correctness on the issue's ask is equal. The difference is in scope and risk. Patch A adds a focused test that asserts the default STT model name is gpt-transcribe and that an explicit model still overrides it; it would fail before and pass after, and nothing else changes. Patch B bundles two things the issue never asks for: (1) an unrelated pyproject.toml change to the sqlalchemy extra (SQLAlchemy[asyncio]), which is pure dependency churn in a model-rename fix, and (2) a behavioural change in openai_stt.py that, via a hard-coded string compare on self.model == 'gpt-transcribe', drops the language kwarg and re-sends it as extra_body={'languages': [...]}. Nothing in the issue supports that the replacement model rejects the language parameter, and the independent test with language set passes identically on A and B, so this remap earns no credit and adds risk: it is a brittle special case that would not apply to any other model name, and it is applied only to the non-streamed path while the streamed session code is untouched (yet B's streamed test asserts a 'languages' key, so that test's validity depends on unchanged code already behaving that way). B's wire-level tests are more thorough than A's, but thoroughness in tests does not offset unrelated dependency edits and speculative API-shape changes mixed into a one-line default swap. A resolves the issue at its root with no extraneous risk, so A is the better fix.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

3m

Wall-clock time in the container

Full test suite

No new failures

852 passed · 3 skipped · 855 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

13vs 99 merged

Changed lines, ours vs merged

Hi, quick heads-up in case this isn't already on your radar.

OpenAI is shutting down gpt-4o-transcribe on February 26, 2027.

It looks like openai-agents-python still uses it, where it's DEFAULT_STT_MODEL. It's still there on the default branch as of October 1:

0

OpenAI's replacement is gpt-transcribe.

There's still time, but the provider sets the date and it won't move.

How I found it: I maintain DocsWatcher, an open source scanner that checks code against the deprecation notices providers publish. I ran it over the 50 most-starred repos tagged openai. Most of what it turns up is model names sitting in lists and price tables, which don't break anything, so I set those aside and read every hit in real code myself before opening anything. This one looked like a real call to me, but you know your code and I don't, so if I've got it wrong just say so and I'll close this.

Nothing to adopt here. I just thought you'd want to know.

The two patches

Our patch+12 −1
src/agents/voice/models/openai_model_provider.pyindex d662664b..8b2b4fab 100644@@ -28,7 +28,7 @@ def shared_http_client() -> httpx2.AsyncClient:     return _http_client  -DEFAULT_STT_MODEL = "gpt-4o-transcribe"+DEFAULT_STT_MODEL = "gpt-transcribe" DEFAULT_TTS_MODEL = "gpt-4o-mini-tts"  tests/voice/test_openai_model_provider.pyindex b27d3750..52f106b0 100644@@ -86,3 +86,14 @@ def test_voice_provider_explicit_options_override_default_client(      assert provider._get_client() is created_client     assert captured_kwargs[option_name] == option_value+++def test_voice_provider_default_stt_model_is_not_retired_gpt_4o_transcribe():+    # gpt-4o-transcribe is retired on 2027-02-26; the default must use its replacement.+    client = openai.AsyncOpenAI(api_key="test_key")+    provider = OpenAIVoiceModelProvider(openai_client=client)++    stt_model = provider.get_stt_model(None)++    assert stt_model.model_name == "gpt-transcribe"+    assert provider.get_stt_model("whisper-1").model_name == "whisper-1" 
Merged fix · PR #5276+93 −6
pyproject.tomlindex d810e410f..5c3027623 100644@@ -49,7 +49,7 @@ any-llm = [     "pygments>=2.20.0; python_version >= '3.11'", ] realtime = ["websockets>=15.0, <17"]-sqlalchemy = ["SQLAlchemy>=2.0", "asyncpg>=0.29.0"]+sqlalchemy = ["SQLAlchemy[asyncio]>=2.0", "asyncpg>=0.29.0"] encrypt = ["cryptography>=45.0,<51"] redis = ["redis>=7"] dapr = [src/agents/voice/models/openai_model_provider.pyindex d662664bd..8b2b4fabd 100644@@ -28,7 +28,7 @@ def shared_http_client() -> httpx2.AsyncClient:     return _http_client  -DEFAULT_STT_MODEL = "gpt-4o-transcribe"+DEFAULT_STT_MODEL = "gpt-transcribe" DEFAULT_TTS_MODEL = "gpt-4o-mini-tts"  src/agents/voice/models/openai_stt.pyindex 977594746..4632ccd85 100644@@ -9,7 +9,7 @@ from time import monotonic from typing import Any, cast from uuid import uuid4 -from openai import AsyncOpenAI+from openai import AsyncOpenAI, omit  from ... import _debug from ...exceptions import AgentsException, UserError@@ -644,7 +644,16 @@ class OpenAISTTModel(STTModel):                     model=self.model,                     file=input.to_audio_file(),                     prompt=self._non_null_or_not_given(settings.prompt),-                    language=self._non_null_or_not_given(settings.language),+                    language=(+                        omit+                        if self.model == "gpt-transcribe"+                        else self._non_null_or_not_given(settings.language)+                    ),+                    extra_body=(+                        {"languages": [settings.language]}+                        if self.model == "gpt-transcribe" and settings.language is not None+                        else None+                    ),                     temperature=self._non_null_or_not_given(settings.temperature),                 )                 if trace_include_sensitive_data:tests/voice/test_openai_model_provider.pyindex b27d3750f..51898293b 100644@@ -1,15 +1,88 @@ # Tests for the OpenAI voice model provider (OpenAIVoiceModelProvider). +import json+from email.parser import BytesParser+from email.policy import default from typing import Any, cast+from unittest.mock import AsyncMock  import httpx2+import numpy as np import openai import pytest  from agents.exceptions import UserError from agents.models import _openai_shared+from agents.voice import AudioInput, StreamedAudioInput, STTModelSettings from agents.voice.models import openai_model_provider from agents.voice.models.openai_model_provider import OpenAIVoiceModelProvider, shared_http_client+from agents.voice.models.openai_stt import OpenAISTTTranscriptionSession+++@pytest.mark.asyncio+@pytest.mark.parametrize("model_name", [None, "gpt-4o-transcribe"])+@pytest.mark.parametrize("language", [None, "fr"])+async def test_voice_provider_transcription_model_and_language_on_wire(+    model_name: str | None, language: str | None+) -> None:+    captured: dict[str, bytes] = {}++    async def handle(request: httpx2.Request) -> httpx2.Response:+        assert request.url.path == "/v1/audio/transcriptions"+        message = BytesParser(policy=default).parsebytes(+            f"Content-Type: {request.headers['content-type']}\r\n\r\n".encode()+            + await request.aread()+        )+        for part in message.iter_parts():+            captured[part.get_param("name", header="content-disposition")] = part.get_payload(+                decode=True+            )+        return httpx2.Response(200, json={"text": "Bonjour"})++    async with openai.AsyncOpenAI(+        api_key="test-key",+        http_client=httpx2.AsyncClient(transport=httpx2.MockTransport(handle)),+    ) as client:+        model = OpenAIVoiceModelProvider(openai_client=client).get_stt_model(model_name)+        transcript = await model.transcribe(+            AudioInput(buffer=np.zeros(240, dtype=np.int16)),+            STTModelSettings(language=language, prompt="A greeting", temperature=0.2),+            False,+            False,+        )++    assert transcript == "Bonjour"+    assert captured["model"] == (b"gpt-transcribe" if model_name is None else b"gpt-4o-transcribe")+    assert captured["prompt"] == b"A greeting"+    assert captured["temperature"] == b"0.2"+    if language is None:+        assert "language" not in captured+        assert "languages[]" not in captured+    elif model_name is None:+        assert captured["languages[]"] == b"fr"+        assert "language" not in captured+    else:+        assert captured["language"] == b"fr"+        assert "languages[]" not in captured+++@pytest.mark.asyncio+async def test_voice_provider_streamed_default_transcription_config() -> None:+    async with openai.AsyncOpenAI(api_key="test-key") as client:+        model = OpenAIVoiceModelProvider(openai_client=client).get_stt_model(None)+        session = await model.create_session(+            StreamedAudioInput(), STTModelSettings(language="fr"), False, False+        )+        assert isinstance(session, OpenAISTTTranscriptionSession)+        websocket = AsyncMock()+        session._websocket = websocket+        await session._configure_session()+        payload = json.loads(websocket.send.await_args.args[0])+        assert payload["session"]["audio"]["input"]["transcription"] == {+            "model": "gpt-transcribe",+            "languages": ["fr"],+        }+        await session.close()   @pytest.mark.parametrize(uv.lockindex 5f1442dd8..9bd91fffd 100644@@ -2751,7 +2751,7 @@ s3 = [ ] sqlalchemy = [     { name = "asyncpg" },-    { name = "sqlalchemy" },+    { name = "sqlalchemy", extra = ["asyncio"] }, ] temporal = [     { name = "protobuf" },@@ -2858,7 +2858,7 @@ requires-dist = [     { name = "redis", marker = "extra == 'redis'", specifier = ">=7" },     { name = "requests", specifier = ">=2.0,<3" },     { name = "runloop-api-client", marker = "extra == 'runloop'", specifier = ">=1.16.0,<2.0.0" },-    { name = "sqlalchemy", marker = "extra == 'sqlalchemy'", specifier = ">=2.0" },+    { name = "sqlalchemy", extras = ["asyncio"], marker = "extra == 'sqlalchemy'", specifier = ">=2.0" },     { name = "starlette", specifier = ">=1.3.1" },     { name = "temporalio", marker = "extra == 'temporal'", specifier = "==1.33.0" },     { name = "textual", marker = "extra == 'temporal'", specifier = ">=8.2.3,<8.3" },@@ -4184,6 +4184,11 @@ wheels = [     { url = "https://files.pythonhosted.org/packages/b3/3f/3582293d1e185e71d19d7c731c3e2ee20ba21981c4a1115c0806c1f62120/sqlalchemy-2.0.52-py3-none-any.whl", hash = "sha256:3b81b8363a919ce53453591cdb93702e6bd54ade6c4fa2f468fc053baee5ed89", size = 1950700, upload-time = "2026-08-11T20:47:21.603Z" }, ] +[package.optional-dependencies]+asyncio = [+    { name = "greenlet" },+]+ [[package]] name = "sse-starlette" version = "3.0.2"