Backtestsopenai-agents-pythonrun-20261024-184640#618

Issue #618 Merged fix PR # Run 24 Oct 2026 · run-20261024-184640 openai/openai-agents-python

Resource tracker warning (leaked semaphores) with MCPServerStdio

Better

Our patch was judged better than the merged fix.

In this judgment, Patch B = our agent, Patch A = the merged fix.

Patch A only adds a warnings.filterwarnings call in the SDK's own process. The leaked-semaphore warning is emitted by the multiprocessing resource_tracker, which runs in a separate helper process spawned by the MCP server child, so a filter installed in the agent's interpreter cannot reach it. Even if it could, it would be masking a symptom (the server being SIGKILLed before it can clean up) rather than fixing it, and it would hide a genuinely useful warning for unrelated leaks in user code. The independent tests confirm this: both scenarios still FAIL-ASSERT under Patch A, identical to base. Patch B identifies the actual mechanism: upstream stdio_client runs its close-stdin/wait/terminate shutdown inside the transport task group, so when those tasks are cancelled (asyncio.run teardown or owner-task cancellation) the graceful sequence is skipped and anyio SIGKILLs the child, which then cannot release its semaphores. B vendors a near-copy of stdio_client that runs the shutdown under a shielded CancelScope, still bounded by the same termination timeout and falling back to _terminate_process_tree. Both independent tests pass under B. B also adds three regression tests that spawn a real FastMCP server and verify via a marker file that the server reached post-mcp.run() code (i.e., exited gracefully) under transport-task cancellation, owner-task cancellation, and asyncio.run shutdown; these would fail on base and pass with the fix. Scope-wise B is larger (duplicating ~150 lines of the upstream transport and relying on private mcp helpers like _create_platform_compatible_process and _terminate_process_tree, which is a maintenance risk if upstream changes), but that risk is justified by a working fix and is clearly documented in the module docstring. A delivers no working fix and no tests, so B wins decisively on correctness and tests.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

16m 23s

Wall-clock time in the container

Full test suite

No new failures

218 passed · 0 skipped · 218 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

316vs 17 merged

Changed lines, ours vs merged

Debug information

  • Agents SDK version: 0.0.12
  • Python 3.13.1

Describe the bug

When using an external MCP plugin with the OpenAI Agents SDK (MCPServerStdio in this case), I consistently receive a resource_tracker: There appear to be XX leaked semaphore objects to clean up at shutdown warning message during the program's shutdown. The exact number of leaked objects (e.g., 24) varies.

Steps to reproduce

Use the following code snippet which initializes a custom OpenAI client and then attempts to run an agent configured with an MCPServerStdio plugin.

Code Snippet

import asyncio
from agents import Agent, Runner
from agents import set_default_openai_client, set_default_openai_api, set_tracing_disabled
from agents.mcp import MCPServerStdio
from openai import AsyncOpenAI

# --- Replace with your actual values ---
OPENAI_BASE_URL = "YOUR_CUSTOM_BASE_URL" # e.g., "http://localhost:11434/v1" or similar
OPENAI_API_KEY = "YOUR_API_KEY" # Can be "ollama" or similar if using local endpoint
MODEL_NAME = "YOUR_MODEL_NAME" # e.g., "llama3" or "gpt-4o"
# --- End of replacements ---

async def main():
    # Create custom OpenAI client
    custom_client = AsyncOpenAI(base_url=OPENAI_BASE_URL, api_key=OPENAI_API_KEY)

    # Required configuration for using a custom OpenAI API endpoint
    set_default_openai_api("chat_completions") # Or other API type if needed
    set_default_openai_client(custom_client)
    set_tracing_disabled(True) # Disable tracing if not needed

    # Get Python executable path
    current_dir = os.path.dirname(os.path.abspath(__file__))
    python_executable = sys.executable
    soc_push_script = os.path.join(current_dir, "plugins", "soc_push_script.py") # Assuming this script exists and works as an MCP plugin

    # Initialize the MCP server plugin using MCPServerStdio
    # Make sure soc_push_script.py implements the MCP protocol correctly
    mcp_soc_push = MCPServerStdio(
        name="soc_push_script",
        params={
            "command": python_executable,
            "args": [soc_push_script],
        })

    # Run the agent with the MCP plugin
    try:
        async with mcp_soc_push as push_tool: # Use the plugin as a tool
            agent = Agent(
                name="test",
                instructions='test',
                model=MODEL_NAME,
                mcp_servers=[push_tool], # Add the MCP tool to the agent's servers
            )
            print("Starting agent run...")
            result = await Runner.run(starting_agent=agent, input='')
            print("Agent run finished.")
            print("Final Output:", result.final_output)
    finally:
         # Explicitly attempt cleanup if necessary, though the warning suggests leakage
         print("Attempting cleanup (may not prevent warning)...")
         # Specific MCP cleanup methods might be needed depending on implementation
         pass # Add cleanup logic here if known

# Assuming this script is run directly
if __name__ == "__main__":
    asyncio.run(main())

Note: The soc_push_script.py file is not provided, but it is assumed to be a functional MCP plugin script that interacts via standard I/O.

Console Output

/usr/local/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/multiprocessing/resource_tracker.py:276: UserWarning: resource_tracker: There appear to be 24 leaked semaphore objects to clean up at shutdown: {'/mp-t9hi4oa_', '/mp-pbo_pje7', '/mp-_1jipfl5', '/mp-04d4uokn', '/mp-jw_gfhjk', '/mp-8122t3qm', '/mp-tjg6llji', '/mp-75j5swhj', '/mp-oprtjvq9', '/mp-_9c_6tt4', '/mp-o6ld1gx6', '/mp-vfh4ncjt', '/mp-6pf__aj0', '/mp-h8x7h4uu', '/mp-_heqxtvk', '/mp-uvzgo0ft', '/mp-gqm7a9dj', '/mp-j09dabgf', '/mp-w30c1pxu', '/mp-4gzlcmxm', '/mp-b5004bcn', '/mp-d1l4chll', '/mp-we2hfv61', '/mp-t1hr16ma'}
  warnings.warn(

The two patches

Our patch+315 −1
src/agents/mcp/server.pyindex 4c0b7b5..e1f820f 100644@@ -15,7 +15,7 @@ import httpx if sys.version_info < (3, 11):     from exceptiongroup import BaseExceptionGroup  # pyright: ignore[reportMissingImports] from anyio.streams.memory import MemoryObjectReceiveStream, MemoryObjectSendStream-from mcp import ClientSession, StdioServerParameters, Tool as MCPTool, stdio_client+from mcp import ClientSession, StdioServerParameters, Tool as MCPTool from mcp.client.session import MessageHandlerFnT from mcp.client.sse import sse_client from mcp.client.streamable_http import GetSessionIdCallback, streamablehttp_client@@ -28,6 +28,7 @@ from ..logger import logger from ..run_context import RunContextWrapper from ..tool import ToolErrorFunction from ..util._types import MaybeAwaitable+from .stdio import stdio_client from .util import (     HttpClientFactory,     MCPToolMetaResolver,src/agents/mcp/stdio.pynew file mode 100644index 0000000..f02d710@@ -0,0 +1,156 @@+"""Stdio transport for MCP servers whose shutdown survives task cancellation.++This mirrors `mcp.client.stdio.stdio_client`, with one difference. Upstream, the MCP stdio+shutdown sequence (close the server's stdin, wait for it to exit, then terminate) runs inside+the transport's task group. When the reader/writer tasks of that task group are cancelled from+the outside, the shutdown sequence is skipped. This happens, for example, when `asyncio.run()`+cancels every remaining task at interpreter shutdown, or when the task that owns the server is+cancelled. AnyIO then SIGKILLs the server process. A server killed that way cannot run its own+cleanup, which surfaces as warnings such as+"resource_tracker: There appear to be N leaked semaphore objects to clean up at shutdown".++Here the shutdown sequence runs in a shielded cancel scope, so the server always gets a chance+to exit gracefully. It stays bounded by the same termination timeouts as upstream.+"""++from __future__ import annotations++import sys+from collections.abc import AsyncIterator+from contextlib import asynccontextmanager+from typing import TextIO++import anyio+import anyio.lowlevel+from anyio.abc import Process+from anyio.streams.memory import MemoryObjectReceiveStream, MemoryObjectSendStream+from anyio.streams.text import TextReceiveStream+from mcp import StdioServerParameters, types+from mcp.client.stdio import (+    PROCESS_TERMINATION_TIMEOUT,+    _create_platform_compatible_process,+    _get_executable_command,+    _terminate_process_tree,+    get_default_environment,+)+from mcp.os.win32.utilities import FallbackProcess+from mcp.shared.message import SessionMessage++from ..logger import logger+++async def _shutdown_process(process: Process | FallbackProcess) -> None:+    """Run the MCP stdio shutdown sequence: close stdin, wait for exit, then terminate."""+    if process.stdin:+        try:+            await process.stdin.aclose()+        except Exception as e:+            # stdin may already be closed (e.g. the server exited on its own).+            logger.debug(f"Error closing stdin of MCP stdio server process: {e}")++    try:+        with anyio.fail_after(PROCESS_TERMINATION_TIMEOUT):+            await process.wait()+    except TimeoutError:+        # The server did not exit after its stdin was closed; escalate to SIGTERM -> SIGKILL.+        await _terminate_process_tree(process)+    except ProcessLookupError:+        # The process already exited.+        pass+++@asynccontextmanager+async def stdio_client(+    server: StdioServerParameters, errlog: TextIO = sys.stderr+) -> AsyncIterator[+    tuple[+        MemoryObjectReceiveStream[SessionMessage | Exception],+        MemoryObjectSendStream[SessionMessage],+    ]+]:+    """Connect to an MCP server by spawning it and communicating over stdin/stdout."""+    read_stream_writer: MemoryObjectSendStream[SessionMessage | Exception]+    read_stream: MemoryObjectReceiveStream[SessionMessage | Exception]+    write_stream: MemoryObjectSendStream[SessionMessage]+    write_stream_reader: MemoryObjectReceiveStream[SessionMessage]++    read_stream_writer, read_stream = anyio.create_memory_object_stream(0)+    write_stream, write_stream_reader = anyio.create_memory_object_stream(0)++    try:+        process = await _create_platform_compatible_process(+            command=_get_executable_command(server.command),+            args=server.args,+            env=(+                {**get_default_environment(), **server.env}+                if server.env is not None+                else get_default_environment()+            ),+            errlog=errlog,+            cwd=server.cwd,+        )+    except OSError:+        await read_stream.aclose()+        await write_stream.aclose()+        await read_stream_writer.aclose()+        await write_stream_reader.aclose()+        raise++    async def stdout_reader() -> None:+        assert process.stdout, "Opened process is missing stdout"++        try:+            async with read_stream_writer:+                buffer = ""+                async for chunk in TextReceiveStream(+                    process.stdout,+                    encoding=server.encoding,+                    errors=server.encoding_error_handler,+                ):+                    lines = (buffer + chunk).split("\n")+                    buffer = lines.pop()++                    for line in lines:+                        try:+                            message = types.JSONRPCMessage.model_validate_json(line)+                        except Exception as exc:+                            logger.exception("Failed to parse JSONRPC message from MCP server")+                            await read_stream_writer.send(exc)+                            continue++                        await read_stream_writer.send(SessionMessage(message))+        except anyio.ClosedResourceError:+            await anyio.lowlevel.checkpoint()++    async def stdin_writer() -> None:+        assert process.stdin, "Opened process is missing stdin"++        try:+            async with write_stream_reader:+                async for session_message in write_stream_reader:+                    json = session_message.message.model_dump_json(by_alias=True, exclude_none=True)+                    await process.stdin.send(+                        (json + "\n").encode(+                            encoding=server.encoding,+                            errors=server.encoding_error_handler,+                        )+                    )+        except anyio.ClosedResourceError:+            await anyio.lowlevel.checkpoint()++    async with anyio.create_task_group() as tg, process:+        tg.start_soon(stdout_reader)+        tg.start_soon(stdin_writer)+        try:+            yield read_stream, write_stream+        finally:+            # Shield the shutdown so that cancellation of this task, or of the reader/writer+            # tasks, cannot skip it and leave the server to be SIGKILLed.+            with anyio.CancelScope(shield=True):+                try:+                    await _shutdown_process(process)+                finally:+                    await read_stream.aclose()+                    await write_stream.aclose()+                    await read_stream_writer.aclose()+                    await write_stream_reader.aclose()tests/mcp/test_stdio_shutdown.pynew file mode 100644index 0000000..6962fce@@ -0,0 +1,157 @@+"""Regression tests: MCPServerStdio must shut its server process down gracefully (close stdin and+let it exit) even when cleanup runs under task cancellation, instead of SIGKILLing it. A killed+server cannot run its own cleanup, which surfaces as e.g. "resource_tracker: There appear to be N+leaked semaphore objects to clean up at shutdown".+"""++from __future__ import annotations++import asyncio+import os+import subprocess+import sys+import textwrap+from pathlib import Path++import pytest++import agents+from agents.mcp import MCPServerStdio++SERVER_SCRIPT = textwrap.dedent(+    """+    import sys+    from mcp.server.fastmcp import FastMCP++    mcp = FastMCP("graceful")++    @mcp.tool()+    def add(a: int, b: int) -> int:+        return a + b++    mcp.run()+    # Only reached when the client closes our stdin; a SIGKILL never gets here.+    with open(sys.argv[1], "w") as f:+        f.write("graceful")+    """+)+++def _write_server(tmp_path: Path) -> tuple[Path, Path]:+    script = tmp_path / "server.py"+    script.write_text(SERVER_SCRIPT)+    return script, tmp_path / "marker.txt"+++@pytest.mark.asyncio+async def test_cleanup_waits_for_graceful_exit_when_transport_tasks_are_cancelled(+    tmp_path: Path,+):+    """Cancelling the owning task together with the transport's reader/writer tasks (what+    `asyncio.run()` does at shutdown) must still let the server exit on its own."""+    script, marker = _write_server(tmp_path)+    server = MCPServerStdio(+        params={"command": sys.executable, "args": [str(script), str(marker)]},+        client_session_timeout_seconds=30,+    )+    connected = asyncio.Event()+    cleaned_up = asyncio.Event()++    async def owner() -> None:+        try:+            await server.connect()+            tools = await server.list_tools()+            assert [tool.name for tool in tools] == ["add"]+            connected.set()+            await asyncio.sleep(3600)+        finally:+            await server.cleanup()+            cleaned_up.set()++    current = asyncio.current_task()+    tasks_before = asyncio.all_tasks()+    owner_task = asyncio.create_task(owner())+    await asyncio.wait_for(connected.wait(), timeout=30)++    spawned = [t for t in asyncio.all_tasks() - tasks_before if t is not current]+    assert owner_task in spawned+    assert len(spawned) > 1, "expected the stdio transport to run its own tasks"+    for task in spawned:+        task.cancel()+    await asyncio.wait_for(asyncio.gather(*spawned, return_exceptions=True), timeout=30)++    assert cleaned_up.is_set()+    assert server.session is None+    assert marker.exists(), "MCP server process was killed instead of exiting gracefully"+    assert marker.read_text() == "graceful"+++@pytest.mark.asyncio+async def test_cleanup_waits_for_graceful_exit_when_owner_task_is_cancelled(tmp_path: Path):+    script, marker = _write_server(tmp_path)+    server = MCPServerStdio(+        params={"command": sys.executable, "args": [str(script), str(marker)]},+        client_session_timeout_seconds=30,+    )+    connected = asyncio.Event()++    async def owner() -> None:+        async with server:+            await server.list_tools()+            connected.set()+            await asyncio.sleep(3600)++    owner_task = asyncio.create_task(owner())+    await asyncio.wait_for(connected.wait(), timeout=30)+    owner_task.cancel()+    with pytest.raises(asyncio.CancelledError):+        await asyncio.wait_for(owner_task, timeout=30)++    assert server.session is None+    assert marker.read_text() == "graceful"+++def test_asyncio_run_shutdown_lets_server_exit_gracefully(tmp_path: Path):+    """A server still connected when `asyncio.run()` tears down the loop must not be killed."""+    script, marker = _write_server(tmp_path)+    driver = tmp_path / "driver.py"+    driver.write_text(+        textwrap.dedent(+            f"""+            import asyncio+            import sys++            from agents.mcp import MCPServerStdio++            async def owner(connected):+                args = [{str(script)!r}, {str(marker)!r}]+                server = MCPServerStdio(+                    params={{"command": sys.executable, "args": args}},+                    client_session_timeout_seconds=30,+                )+                async with server:+                    await server.list_tools()+                    connected.set()+                    await asyncio.sleep(3600)++            async def main():+                connected = asyncio.Event()+                asyncio.create_task(owner(connected))+                await asyncio.wait_for(connected.wait(), timeout=30)+                # Return while the server is still connected; asyncio.run() cancels the rest.++            asyncio.run(main())+            """+        )+    )+    src_dir = str(Path(agents.__file__).resolve().parent.parent)+    pythonpath = [src_dir, os.environ.get("PYTHONPATH", "")]+    env = {**os.environ, "PYTHONPATH": os.pathsep.join(p for p in pythonpath if p)}++    result = subprocess.run(+        [sys.executable, str(driver)], env=env, capture_output=True, text=True, timeout=60+    )++    assert result.returncode == 0, result.stderr+    assert marker.exists(), f"MCP server process was killed at shutdown. stderr:\n{result.stderr}"+    assert marker.read_text() == "graceful" 
Merged fix · PR #+17 −0
src/agents/mcp/server.pyindex 4c0b7b5fe..72f7ca0b1 100644@@ -2,8 +2,11 @@ from __future__ import annotations  import abc import asyncio+import atexit import inspect+import multiprocessing import sys+import warnings from collections.abc import Awaitable from contextlib import AbstractAsyncContextManager, AsyncExitStack from datetime import timedelta@@ -65,6 +68,20 @@ class _UnsetType:     pass  +# Suppress Python's multiprocessing resource tracker semaphore leak warnings.+# These warnings occur when MCPServerStdio spawns subprocesses that don't get+# properly registered with the resource tracker. This is a known issue with+# stdio-based MCP servers and is not a bug in the user's code.+# We suppress this warning at import time since it's a known limitation of the+# MCP stdio transport and doesn't indicate a real problem.+warnings.filterwarnings(+    "ignore",+    message=r"There appear to be \d+ leaked semaphore objects to clean up at shutdown",+    category=UserWarning,+    module=r"multiprocessing\.resource_tracker",+)++ _UNSET = _UnsetType()  if TYPE_CHECKING: