Backtestslitellmrun-20261019-033420#10788

Issue #10788 Merged fix PR #41306 Run 19 Oct 2026 · run-20261019-033420 BerriAI/litellm

[Bug]: INFO logging of incoming requests to the LiteLLM proxy server can not be switched off.

Better

Our patch was judged better than the merged fix.

In this judgment, Patch A = our agent, Patch B = the merged fix.

Both patches fix the reported uvicorn path: each sets uvicorn's log_level from LITELLM_LOG when no explicit log_config or JSON logs are configured, which raises the uvicorn.access logger level and drops the per-request INFO lines. The independent tests confirm both pass. Patch A goes further in ways that matter for the issue's ask ("how to switch it off") rather than being unrelated churn: it applies the same env var to hypercorn, granian, and gunicorn, and it explicitly disables access logging where the level is above INFO (gunicorn in particular emits access lines at INFO regardless of loglevel, so A's handling is needed there). A also validates the env value and warns instead of raising, and maps aliases like 'warn' correctly. Its tests are broad: parametrized over levels, verify explicit log_config is not overridden, apply the args to a real uvicorn.Config and check logger levels/handlers, and cover each alternate server. Patch B is minimal and correct for uvicorn (passing an int log_level is valid), with a real configure_logging-based test, but leaves the other server backends unfixed, adds an adjacent change to the litellm_proxy_extras logger that the issue does not mention, and its resolve_log_level raises AttributeError on an unrecognised value with no handling. The extra lines in A add little risk since they are gated on LITELLM_LOG being set and mirror existing option-building code. A is the more complete and better-tested fix.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

26m 40s

Wall-clock time in the container

Full test suite

No new failures

4,498 passed · 6 skipped · 4,504 total

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

226vs 78 merged

Changed lines, ours vs merged

What happened?

Hi, our LiteLLM proxy logs are full of INFO lines for each request. We'd like to switch that off. We expect LITELLM_LOG=ERROR to do the job, but it does not help.

Here is a minimal example to reproduce the problem:

litellm_config.yaml:

model_list:
  - model_name: gpt-4o-mini
    litellm_params:
      model: openai/gpt-4o-mini
      api_key: sk-[...]
general_settings: 
  master_key: sk-1234

run:

docker run \
    -v $(pwd)/litellm_config.yaml:/app/config.yaml \
    -p 4000:4000 \
    --env LITELLM_LOG=ERROR \
    ghcr.io/berriai/litellm:main-v1.69.0-stable \
    --config /app/config.yaml

When sending requests like this

curl -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-d '{
    "model": "gpt-4o-mini",
    "messages": [
      {
        "role": "user",
        "content": "Hi."
      }
    ]
}'

the LiteLLM proxy server logs lines as follows:

REACHES async_pre_call_hook, call_type: acompletion
INFO:     172.17.0.1:40470 - "POST /chat/completions HTTP/1.1" 200 OK
REACHES async_pre_call_hook, call_type: acompletion
INFO:     172.17.0.1:47168 - "POST /chat/completions HTTP/1.1" 200 OK
REACHES async_pre_call_hook, call_type: acompletion
INFO:     172.17.0.1:47180 - "POST /chat/completions HTTP/1.1" 200 OK
REACHES async_pre_call_hook, call_type: acompletion
INFO:     172.17.0.1:47194 - "POST /chat/completions HTTP/1.1" 200 OK
REACHES async_pre_call_hook, call_type: acompletion
INFO:     172.17.0.1:38720 - "POST /chat/completions HTTP/1.1" 200 OK

How to switch it off?

Relevant log output

Are you a ML Ops Team?

Yes

What LiteLLM version are you on ?

v1.69.0

Twitter / LinkedIn details

_No response_

The two patches

Our patch+226 −0
litellm/proxy/proxy_cli.pyindex 01a3da0899..e0c1f022ec 100644@@ -245,6 +245,40 @@ class ProxyInitializationHelpers:         )         print(completion_response) +    @staticmethod+    def _get_server_log_level() -> str | None:+        """+        Map LITELLM_LOG to a lowercase level name for the ASGI server's own loggers+        (e.g. uvicorn's per-request "INFO: ... POST /chat/completions" access lines).++        Returns None when LITELLM_LOG is unset, so the server keeps its defaults.+        """+        import logging++        raw_level: Final = os.getenv("LITELLM_LOG")+        if raw_level is None or raw_level.strip() == "":+            return None+        numeric_level: Final = getattr(logging, raw_level.strip().upper(), None)+        if not isinstance(numeric_level, int):+            print(+                f"\033[1;33mLiteLLM Proxy: Ignoring unrecognized LITELLM_LOG={raw_level!r} "+                f"for server logging.\033[0m"+            )+            return None+        level_name: Final = logging.getLevelName(numeric_level).lower()+        if level_name not in ("critical", "error", "warning", "info", "debug"):+            print(+                f"\033[1;33mLiteLLM Proxy: Ignoring unsupported LITELLM_LOG={raw_level!r} "+                f"for server logging.\033[0m"+            )+            return None+        return level_name++    @staticmethod+    def _server_access_log_enabled(server_log_level: str | None) -> bool:+        """Access lines are emitted at INFO, so drop them when LITELLM_LOG is above INFO."""+        return server_log_level not in ("warning", "error", "critical")+     @staticmethod     def _get_default_unvicorn_init_args(         host: str,@@ -275,6 +309,13 @@ class ProxyInitializationHelpers:         elif litellm.json_logs:             # Use JSON log config for uvicorn to ensure all logs (including exceptions) are JSON             uvicorn_args["log_config"] = _get_uvicorn_json_log_config()+        else:+            # uvicorn's default log config logs every request at INFO; honor LITELLM_LOG+            server_log_level: Final = ProxyInitializationHelpers._get_server_log_level()+            if server_log_level is not None:+                uvicorn_args["log_level"] = server_log_level+                if not ProxyInitializationHelpers._server_access_log_enabled(server_log_level):+                    uvicorn_args["access_log"] = False         if keepalive_timeout is not None:             uvicorn_args["timeout_keep_alive"] = keepalive_timeout         if timeout_worker_healthcheck is not None:@@ -426,6 +467,10 @@ class ProxyInitializationHelpers:             if ciphers is not None:                 config.ciphers = ciphers +        server_log_level: Final = ProxyInitializationHelpers._get_server_log_level()+        if server_log_level is not None:+            config.loglevel = server_log_level.upper()+         # hypercorn serve raises a type warning when passing a fast api app - even though fast API is a valid type         asyncio.run(serve(app, config)) @@ -468,6 +513,13 @@ class ProxyInitializationHelpers:         }         if granian_runtime_threads is not None:             kwargs["runtime_threads"] = granian_runtime_threads+        server_log_level: Final = ProxyInitializationHelpers._get_server_log_level()+        if server_log_level is not None:+            from granian.log import LogLevels++            kwargs["log_level"] = LogLevels(server_log_level)+            if not ProxyInitializationHelpers._server_access_log_enabled(server_log_level):+                kwargs["log_access"] = False         if ssl_certfile_path is not None and ssl_keyfile_path is not None:             print(                 f"\033[1;32mLiteLLM Proxy: Using SSL with certfile: {ssl_certfile_path} and keyfile: {ssl_keyfile_path}\033[0m\n"@@ -559,6 +611,14 @@ class ProxyInitializationHelpers:             "access_log_format": '%(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s',         } +        # gunicorn always logs access lines at INFO regardless of loglevel, so disable+        # the access log entirely when LITELLM_LOG is above INFO+        server_log_level: Final = ProxyInitializationHelpers._get_server_log_level()+        if server_log_level is not None:+            gunicorn_options["loglevel"] = server_log_level+            if not ProxyInitializationHelpers._server_access_log_enabled(server_log_level):+                gunicorn_options.pop("accesslog")+         # Optional: recycle workers after N requests to mitigate memory growth         if max_requests_before_restart is not None:             gunicorn_options["max_requests"] = max_requests_before_restarttests/test_litellm/proxy/test_proxy_cli.pyindex e25e6a5988..51452d8ba6 100644@@ -149,6 +149,87 @@ class TestProxyInitializationHelpers:         )         assert args["timeout_worker_healthcheck"] == 30 +    def test_uvicorn_args_without_litellm_log_keep_uvicorn_defaults(self, monkeypatch):+        monkeypatch.delenv("LITELLM_LOG", raising=False)+        args = ProxyInitializationHelpers._get_default_unvicorn_init_args(+            "localhost", 8000+        )+        assert "log_level" not in args+        assert "access_log" not in args++    @pytest.mark.parametrize(+        "env_value, expected_level, access_log_disabled",+        [+            ("ERROR", "error", True),+            ("WARNING", "warning", True),+            ("warn", "warning", True),+            ("CRITICAL", "critical", True),+            ("INFO", "info", False),+            ("DEBUG", "debug", False),+        ],+    )+    def test_uvicorn_args_honor_litellm_log(+        self, monkeypatch, env_value, expected_level, access_log_disabled+    ):+        """Regression: LITELLM_LOG=ERROR must silence uvicorn's per-request INFO access lines."""+        monkeypatch.setenv("LITELLM_LOG", env_value)+        args = ProxyInitializationHelpers._get_default_unvicorn_init_args(+            "localhost", 8000+        )+        assert args["log_level"] == expected_level+        if access_log_disabled:+            assert args["access_log"] is False+        else:+            assert "access_log" not in args++    def test_uvicorn_args_explicit_log_config_not_overridden_by_litellm_log(+        self, monkeypatch+    ):+        monkeypatch.setenv("LITELLM_LOG", "ERROR")+        args = ProxyInitializationHelpers._get_default_unvicorn_init_args(+            "localhost", 8000, "log_config.json"+        )+        assert args["log_config"] == "log_config.json"+        assert "log_level" not in args+        assert "access_log" not in args++    def test_uvicorn_litellm_log_error_suppresses_access_lines(self, monkeypatch):+        """Apply the generated args to a real uvicorn Config and check request lines are dropped."""+        import logging++        monkeypatch.setenv("LITELLM_LOG", "ERROR")+        args = ProxyInitializationHelpers._get_default_unvicorn_init_args(+            "localhost", 8000+        )+        logger_names = ("uvicorn", "uvicorn.error", "uvicorn.access", "uvicorn.asgi")+        saved = {+            name: (+                logging.getLogger(name).level,+                list(logging.getLogger(name).handlers),+                logging.getLogger(name).propagate,+            )+            for name in logger_names+        }+        try:+            uvicorn.Config(**args)+            access_logger = logging.getLogger("uvicorn.access")+            assert not access_logger.isEnabledFor(logging.INFO)+            assert access_logger.handlers == []+            assert not logging.getLogger("uvicorn.error").isEnabledFor(logging.INFO)+            assert logging.getLogger("uvicorn.error").isEnabledFor(logging.ERROR)+        finally:+            for name, (level, handlers, propagate) in saved.items():+                logger = logging.getLogger(name)+                logger.setLevel(level)+                logger.handlers = handlers+                logger.propagate = propagate++    def test_get_server_log_level_ignores_unsupported_value(self, monkeypatch):+        monkeypatch.setenv("LITELLM_LOG", "NOTSET")+        with patch("builtins.print") as mock_print:+            assert ProxyInitializationHelpers._get_server_log_level() is None+        assert any("LITELLM_LOG" in str(c) for c in mock_print.call_args_list)+     def test_get_reload_options_no_config_still_watches_env(self):         opts = ProxyInitializationHelpers._get_reload_options(None)         assert opts["reload"] is True@@ -409,6 +490,49 @@ class TestProxyInitializationHelpers:         assert call_kwargs["ssl_key"] == Path("/path/to/key.pem")         mock_server.serve.assert_called_once() +    @patch("asyncio.run")+    @patch("builtins.print")+    def test_init_hypercorn_server_honors_litellm_log(+        self, mock_print, mock_asyncio_run, monkeypatch+    ):+        pytest.importorskip("hypercorn")+        monkeypatch.setenv("LITELLM_LOG", "ERROR")+        captured: dict = {}++        def fake_serve(app, config):+            captured["config"] = config++        with patch("hypercorn.asyncio.serve", fake_serve):+            ProxyInitializationHelpers._init_hypercorn_server(+                MagicMock(), "localhost", 8000, None, None, None+            )+        assert captured["config"].loglevel == "ERROR"++    @patch("granian.Granian")+    @patch("builtins.print")+    def test_init_granian_server_honors_litellm_log(+        self, mock_print, mock_granian_cls, monkeypatch+    ):+        pytest.importorskip("granian")+        from granian.log import LogLevels++        monkeypatch.setenv("LITELLM_LOG", "ERROR")+        fake_interfaces = SimpleNamespace(ASGI="asgi")+        with patch("granian.constants.Interfaces", fake_interfaces):+            ProxyInitializationHelpers._init_granian_server(+                host="0.0.0.0",+                port=4000,+                num_workers=1,+                ssl_certfile_path=None,+                ssl_keyfile_path=None,+                max_requests_before_restart=None,+                ciphers=None,+                granian_runtime_threads=None,+            )+        call_kwargs = mock_granian_cls.call_args.kwargs+        assert call_kwargs["log_level"] == LogLevels.error+        assert call_kwargs["log_access"] is False+     @patch("granian.Granian")     def test_init_granian_server_ssl_requires_cert_and_key(self, mock_granian_cls):         pytest.importorskip("granian")@@ -1545,6 +1669,48 @@ class TestProxyInitializationHelpers:         assert captured["options"]["max_requests"] == 1000         assert captured["options"]["max_requests_jitter"] == 50 +    @pytest.mark.skipif(os.name == "nt", reason="gunicorn server path skips Windows")+    @pytest.mark.parametrize(+        "env_value, expect_access_log",+        [(None, True), ("ERROR", False), ("INFO", True)],+    )+    def test_gunicorn_options_honor_litellm_log(+        self, monkeypatch, env_value, expect_access_log+    ):+        """Regression: LITELLM_LOG=ERROR must drop gunicorn's per-request access lines."""+        pytest.importorskip("gunicorn")+        if env_value is None:+            monkeypatch.delenv("LITELLM_LOG", raising=False)+        else:+            monkeypatch.setenv("LITELLM_LOG", env_value)++        captured: dict = {}++        def capture_run(self):+            captured["options"] = dict(self.options)++        with (+            patch("gunicorn.app.base.BaseApplication.run", capture_run),+            patch("builtins.print"),+            patch(  # test-quality-ok: option tests must not start a thread or change the pytest worker's child ownership+                "litellm.proxy.proxy_cli.start_query_engine_reaper"+            ),+        ):+            ProxyInitializationHelpers._run_gunicorn_server(+                host="127.0.0.1",+                port=4012,+                app=MagicMock(),+                num_workers=1,+                ssl_certfile_path=None,+                ssl_keyfile_path=None,+            )++        if env_value is None:+            assert "loglevel" not in captured["options"]+        else:+            assert captured["options"]["loglevel"] == env_value.lower()+        assert ("accesslog" in captured["options"]) is expect_access_log+     @pytest.mark.skipif(os.name == "nt", reason="gunicorn server path skips Windows")     def test_gunicorn_jitter_without_base_warns(self):         """gunicorn path warns when jitter is set without --max_requests_before_restart""" 
Merged fix · PR #41306+75 −3
litellm-proxy-extras/litellm_proxy_extras/_logging.pyindex ecf467fbf45..64e07a180d3 100644@@ -40,4 +40,4 @@ if not logger.handlers:             logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")         )     logger.addHandler(handler)-    logger.setLevel(logging.INFO)+    logger.setLevel(os.getenv("LITELLM_LOG", "INFO").upper())litellm/_logging.pyindex 03a9bcf21cf..873a6619a81 100644@@ -401,10 +401,14 @@ def _parse_json_logs_env(value: str | None) -> bool:     return (value or "").lower() == "true"  +def resolve_log_level(log_level: str) -> int:+    return getattr(logging, log_level.upper())++ json_logs: Final = _parse_json_logs_env(os.getenv("JSON_LOGS")) # Create a handler for the logger (you may need to adapt this based on your needs) log_level: Final = os.getenv("LITELLM_LOG", "DEBUG")-numeric_level: Final[str] = getattr(logging, log_level.upper())+numeric_level: Final[int] = resolve_log_level(log_level) handler: Final = LevelRoutingStreamHandler() handler.setLevel(numeric_level) handler.addFilter(_secret_filter)litellm/proxy/proxy_cli.pyindex 01a3da08998..b25c77f6828 100644@@ -261,7 +261,7 @@ class ProxyInitializationHelpers:         import uvicorn          import litellm-        from litellm._logging import _get_uvicorn_json_log_config+        from litellm._logging import _get_uvicorn_json_log_config, resolve_log_level          uvicorn_args: Final = {             "app": "litellm.proxy.proxy_server:app",@@ -275,6 +275,8 @@ class ProxyInitializationHelpers:         elif litellm.json_logs:             # Use JSON log config for uvicorn to ensure all logs (including exceptions) are JSON             uvicorn_args["log_config"] = _get_uvicorn_json_log_config()+        elif litellm_log := os.environ.get("LITELLM_LOG"):+            uvicorn_args["log_level"] = resolve_log_level(litellm_log)         if keepalive_timeout is not None:             uvicorn_args["timeout_keep_alive"] = keepalive_timeout         if timeout_worker_healthcheck is not None:tests/litellm-proxy-extras/test_litellm_proxy_extras_logging.pynew file mode 100644index 00000000000..fb209bc3925@@ -0,0 +1,37 @@+import importlib+import logging+from collections.abc import Iterator++import pytest++import litellm_proxy_extras._logging as extras_logging+++@pytest.fixture+def fresh_extras_logger() -> Iterator[logging.Logger]:+    logger = logging.getLogger("litellm_proxy_extras")+    saved_handlers = logger.handlers[:]+    saved_level = logger.level+    logger.handlers[:] = []+    try:+        yield logger+    finally:+        logger.handlers[:] = saved_handlers+        logger.setLevel(saved_level)+++def test_litellm_log_error_silences_extras_info_lines(monkeypatch, fresh_extras_logger):+    monkeypatch.setenv("LITELLM_LOG", "ERROR")+    reloaded = importlib.reload(extras_logging).logger+    assert reloaded is fresh_extras_logger+    assert reloaded.isEnabledFor(logging.INFO) is False+    assert reloaded.isEnabledFor(logging.ERROR) is True+++@pytest.mark.parametrize("litellm_log", [None, "info", "DEBUG"])+def test_unset_or_verbose_litellm_log_keeps_extras_info_lines(monkeypatch, fresh_extras_logger, litellm_log):+    if litellm_log is None:+        monkeypatch.delenv("LITELLM_LOG", raising=False)+    else:+        monkeypatch.setenv("LITELLM_LOG", litellm_log)+    assert importlib.reload(extras_logging).logger.isEnabledFor(logging.INFO) is Truetests/test_litellm/proxy/test_proxy_cli.pyindex e25e6a59884..712c526b244 100644@@ -139,6 +139,35 @@ class TestProxyInitializationHelpers:             )             assert args["timeout_worker_healthcheck"] == 15 +    @staticmethod+    def _uvicorn_access_info_enabled(args: dict) -> bool:+        import logging++        loggers = tuple(logging.getLogger(n) for n in ("uvicorn", "uvicorn.error", "uvicorn.access", "uvicorn.asgi"))+        saved = tuple((lg, lg.handlers[:], lg.level, lg.propagate) for lg in loggers)+        try:+            uvicorn.Config(**args).configure_logging()+            return logging.getLogger("uvicorn.access").isEnabledFor(logging.INFO)+        finally:+            for lg, handlers, level, propagate in saved:+                lg.handlers[:] = handlers+                lg.setLevel(level)+                lg.propagate = propagate++    def test_litellm_log_error_silences_uvicorn_info_lines(self, monkeypatch):+        monkeypatch.setenv("LITELLM_LOG", "ERROR")+        args = ProxyInitializationHelpers._get_default_unvicorn_init_args("localhost", 8000)++        assert "log_config" not in args+        assert self._uvicorn_access_info_enabled(args) is False++    def test_unset_litellm_log_keeps_uvicorn_default_info_lines(self, monkeypatch):+        monkeypatch.delenv("LITELLM_LOG", raising=False)+        args = ProxyInitializationHelpers._get_default_unvicorn_init_args("localhost", 8000)++        assert "log_level" not in args+        assert self._uvicorn_access_info_enabled(args) is True+     def test_installed_uvicorn_supports_worker_flags(self):         params = inspect.signature(uvicorn.Config.__init__).parameters         assert "timeout_worker_healthcheck" in params