Backtestslitellmrun-20261018-182052#10788
[Bug]: INFO logging of incoming requests to the LiteLLM proxy server can not be switched off.
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Both patches fix the core bug the same way: pass LITELLM_LOG through as uvicorn's log_level so uvicorn.access/uvicorn.error stop emitting INFO request lines, and the independent tests pass on both. B is the stronger fix. Its level parsing (_get_server_log_level) is tolerant of blank, whitespace-padded, lowercase, and invalid values and returns None instead of raising; A's resolve_log_level uses getattr(logging, ...) and will raise AttributeError on an invalid value, and its litellm_proxy_extras change calls logger.setLevel with a raw env string that raises ValueError at import time on a typo. A only applies the level in an elif branch, so with JSON_LOGS=true the fix silently does nothing, while B applies log_level alongside the JSON log_config (uvicorn applies log_level after dictConfig, so this works) and has a test for it. B also covers the hypercorn, granian, and gunicorn launch paths the same proxy CLI exposes, including disabling gunicorn's pinned-at-INFO access log, which is the same user-facing problem under a different server flag; this is adjacent work but low-risk and tested rather than churn. A's extras logger change is tangential to the issue (the request lines come from uvicorn, not litellm_proxy_extras) and adds import-time failure risk. B's tests are more comprehensive (parsing table, json mode, user log_config not overridden, each server path), and A's tests are solid but narrower. Minor nit against B: slightly larger diff and granian/hypercorn string conversions are untested for unusual levels, but nothing that outweighs A's json_logs gap and unsafe parsing.
Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.
Solve
25m 33s
Wall-clock time in the container
Full test suite
No new failures
4,499 passed · 6 skipped · 4,506 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
192vs 78 merged
Changed lines, ours vs merged
The issue
View on GitHubWhat happened?
Hi, our LiteLLM proxy logs are full of INFO lines for each request. We'd like to switch that off. We expect LITELLM_LOG=ERROR to do the job, but it does not help.
Here is a minimal example to reproduce the problem:
litellm_config.yaml:
model_list:
- model_name: gpt-4o-mini
litellm_params:
model: openai/gpt-4o-mini
api_key: sk-[...]
general_settings:
master_key: sk-1234
run:
docker run \
-v $(pwd)/litellm_config.yaml:/app/config.yaml \
-p 4000:4000 \
--env LITELLM_LOG=ERROR \
ghcr.io/berriai/litellm:main-v1.69.0-stable \
--config /app/config.yaml
When sending requests like this
curl -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-d '{
"model": "gpt-4o-mini",
"messages": [
{
"role": "user",
"content": "Hi."
}
]
}'
the LiteLLM proxy server logs lines as follows:
REACHES async_pre_call_hook, call_type: acompletion
INFO: 172.17.0.1:40470 - "POST /chat/completions HTTP/1.1" 200 OK
REACHES async_pre_call_hook, call_type: acompletion
INFO: 172.17.0.1:47168 - "POST /chat/completions HTTP/1.1" 200 OK
REACHES async_pre_call_hook, call_type: acompletion
INFO: 172.17.0.1:47180 - "POST /chat/completions HTTP/1.1" 200 OK
REACHES async_pre_call_hook, call_type: acompletion
INFO: 172.17.0.1:47194 - "POST /chat/completions HTTP/1.1" 200 OK
REACHES async_pre_call_hook, call_type: acompletion
INFO: 172.17.0.1:38720 - "POST /chat/completions HTTP/1.1" 200 OK
How to switch it off?
Relevant log output
Are you a ML Ops Team?
Yes
What LiteLLM version are you on ?
v1.69.0
Twitter / LinkedIn details
_No response_
The two patches
litellm/_logging.pyindex 03a9bcf21c..4dc169cd8f 100644@@ -744,6 +744,21 @@ def _initialize_loggers_with_handler(handler: logging.Handler): lg.propagate = False # prevent bubbling to parent/root +def _get_server_log_level() -> int | None:+ """+ Numeric level for the ASGI server's own loggers (e.g. uvicorn.access), taken from+ an explicitly set LITELLM_LOG.++ Returns None when LITELLM_LOG is unset or blank, so the server keeps its own+ defaults (INFO access lines) instead of inheriting litellm's DEBUG fallback.+ """+ raw: Final = os.getenv("LITELLM_LOG")+ if raw is None or not raw.strip():+ return None+ level: Final = logging.getLevelName(raw.strip().upper())+ return level if isinstance(level, int) else None++ def _get_uvicorn_json_log_config(): """ Generate a uvicorn log_config dictionary that applies JSON formatting to all loggers.litellm/proxy/proxy_cli.pyindex 01a3da0899..ccaec1162c 100644@@ -1,6 +1,7 @@ # ruff: noqa: T201 import importlib import json+import logging import os import random import re@@ -17,6 +18,7 @@ from dotenv import load_dotenv from pydantic import BaseModel, ConfigDict import litellm+from litellm._logging import _get_server_log_level from litellm.constants import DEFAULT_NUM_WORKERS_LITELLM_PROXY from litellm.proxy.db.pgbouncer import ( PgBouncerError,@@ -275,6 +277,12 @@ class ProxyInitializationHelpers: elif litellm.json_logs: # Use JSON log config for uvicorn to ensure all logs (including exceptions) are JSON uvicorn_args["log_config"] = _get_uvicorn_json_log_config()+ if log_config is None:+ # uvicorn's default config logs every request at INFO; honour LITELLM_LOG+ # (e.g. LITELLM_LOG=ERROR) for uvicorn.error, uvicorn.access and uvicorn.asgi.+ server_log_level = _get_server_log_level()+ if server_log_level is not None:+ uvicorn_args["log_level"] = server_log_level if keepalive_timeout is not None: uvicorn_args["timeout_keep_alive"] = keepalive_timeout if timeout_worker_healthcheck is not None:@@ -416,6 +424,9 @@ class ProxyInitializationHelpers: print(f"\033[1;32mLiteLLM Proxy: Starting server on {host}:{port} using Hypercorn\033[0m\n") config: Final = Config() config.bind = [f"{host}:{port}"]+ server_log_level: Final = _get_server_log_level()+ if server_log_level is not None:+ config.loglevel = logging.getLevelName(server_log_level) if ssl_certfile_path is not None and ssl_keyfile_path is not None: print(@@ -468,6 +479,11 @@ class ProxyInitializationHelpers: } if granian_runtime_threads is not None: kwargs["runtime_threads"] = granian_runtime_threads+ server_log_level: Final = _get_server_log_level()+ if server_log_level is not None:+ from granian.log import LogLevels++ kwargs["log_level"] = LogLevels(logging.getLevelName(server_log_level).lower()) if ssl_certfile_path is not None and ssl_keyfile_path is not None: print( f"\033[1;32mLiteLLM Proxy: Using SSL with certfile: {ssl_certfile_path} and keyfile: {ssl_keyfile_path}\033[0m\n"@@ -562,6 +578,13 @@ class ProxyInitializationHelpers: # Optional: recycle workers after N requests to mitigate memory growth if max_requests_before_restart is not None: gunicorn_options["max_requests"] = max_requests_before_restart+ server_log_level: Final = _get_server_log_level()+ if server_log_level is not None:+ gunicorn_options["loglevel"] = logging.getLevelName(server_log_level).lower()+ # gunicorn pins its access logger (which UvicornWorker hands to uvicorn.access)+ # at INFO, so loglevel alone can't silence per-request lines.+ if server_log_level > logging.INFO:+ gunicorn_options["accesslog"] = None if max_requests_before_restart_jitter is not None: if max_requests_before_restart is None: print(tests/test_litellm/proxy/test_proxy_cli.pyindex e25e6a5988..9397bdc9de 100644@@ -1,4 +1,5 @@ import inspect+import logging import os from pathlib import Path from types import SimpleNamespace@@ -149,6 +150,122 @@ class TestProxyInitializationHelpers: ) assert args["timeout_worker_healthcheck"] == 30 + @pytest.mark.parametrize(+ "raw, expected",+ [+ ("ERROR", logging.ERROR),+ ("error", logging.ERROR),+ (" WARN ", logging.WARNING),+ ("CRITICAL", logging.CRITICAL),+ ("DEBUG", logging.DEBUG),+ ("", None),+ ("not-a-level", None),+ ],+ )+ def test_get_server_log_level_from_litellm_log(self, monkeypatch, raw, expected):+ from litellm._logging import _get_server_log_level++ monkeypatch.setenv("LITELLM_LOG", raw)+ assert _get_server_log_level() == expected++ def test_get_server_log_level_unset_keeps_server_default(self, monkeypatch):+ from litellm._logging import _get_server_log_level++ monkeypatch.delenv("LITELLM_LOG", raising=False)+ assert _get_server_log_level() is None++ def test_litellm_log_error_silences_uvicorn_access_log(self, monkeypatch):+ """LITELLM_LOG=ERROR must drop uvicorn's per-request INFO access lines."""+ monkeypatch.setenv("LITELLM_LOG", "ERROR")+ uvicorn_logger_names = ("uvicorn", "uvicorn.error", "uvicorn.access", "uvicorn.asgi")+ saved = {+ name: (+ logging.getLogger(name).level,+ list(logging.getLogger(name).handlers),+ logging.getLogger(name).propagate,+ )+ for name in uvicorn_logger_names+ }+ try:+ with patch("litellm.json_logs", False):+ args = ProxyInitializationHelpers._get_default_unvicorn_init_args(+ "localhost", 8000+ )+ assert args["log_level"] == logging.ERROR+ # uvicorn.Config applies logging configuration on construction, exactly as uvicorn.run does.+ uvicorn.Config(**args)+ access_logger = logging.getLogger("uvicorn.access")+ assert not access_logger.isEnabledFor(logging.INFO)+ assert access_logger.isEnabledFor(logging.ERROR)+ assert not logging.getLogger("uvicorn.error").isEnabledFor(logging.INFO)+ finally:+ for name, (level, handlers, propagate) in saved.items():+ lg = logging.getLogger(name)+ lg.setLevel(level)+ lg.handlers = handlers+ lg.propagate = propagate++ def test_litellm_log_applies_in_json_log_mode(self, monkeypatch):+ monkeypatch.setenv("LITELLM_LOG", "ERROR")+ with patch("litellm.json_logs", True):+ args = ProxyInitializationHelpers._get_default_unvicorn_init_args(+ "localhost", 8000+ )+ assert args["log_level"] == logging.ERROR+ assert isinstance(args["log_config"], dict)++ def test_unset_litellm_log_keeps_uvicorn_default_level(self, monkeypatch):+ monkeypatch.delenv("LITELLM_LOG", raising=False)+ with patch("litellm.json_logs", False):+ args = ProxyInitializationHelpers._get_default_unvicorn_init_args(+ "localhost", 8000+ )+ assert "log_level" not in args++ def test_user_log_config_is_not_overridden_by_litellm_log(self, monkeypatch):+ monkeypatch.setenv("LITELLM_LOG", "ERROR")+ args = ProxyInitializationHelpers._get_default_unvicorn_init_args(+ "localhost", 8000, "log_config.json"+ )+ assert args["log_config"] == "log_config.json"+ assert "log_level" not in args++ def test_litellm_log_error_sets_hypercorn_loglevel(self, monkeypatch):+ pytest.importorskip("hypercorn")+ monkeypatch.setenv("LITELLM_LOG", "ERROR")+ with (+ patch("hypercorn.asyncio.serve") as mock_serve,+ patch("asyncio.run"),+ patch("builtins.print"),+ ):+ ProxyInitializationHelpers._init_hypercorn_server(+ MagicMock(), "localhost", 8000, None, None, None+ )+ config = mock_serve.call_args.args[1]+ assert config.loglevel == "ERROR"++ @patch("granian.Granian")+ @patch("builtins.print")+ def test_litellm_log_error_sets_granian_log_level(+ self, mock_print, mock_granian_cls, monkeypatch+ ):+ pytest.importorskip("granian")+ from granian.log import LogLevels++ monkeypatch.setenv("LITELLM_LOG", "ERROR")+ fake_interfaces = SimpleNamespace(ASGI="asgi")+ with patch("granian.constants.Interfaces", fake_interfaces):+ ProxyInitializationHelpers._init_granian_server(+ host="0.0.0.0",+ port=4000,+ num_workers=1,+ ssl_certfile_path=None,+ ssl_keyfile_path=None,+ max_requests_before_restart=None,+ ciphers=None,+ )+ assert mock_granian_cls.call_args.kwargs["log_level"] == LogLevels.error+ def test_get_reload_options_no_config_still_watches_env(self): opts = ProxyInitializationHelpers._get_reload_options(None) assert opts["reload"] is True@@ -1545,6 +1662,43 @@ class TestProxyInitializationHelpers: assert captured["options"]["max_requests"] == 1000 assert captured["options"]["max_requests_jitter"] == 50 + @pytest.mark.skipif(os.name == "nt", reason="gunicorn server path skips Windows")+ @pytest.mark.parametrize(+ "litellm_log, expected_loglevel, access_log_on",+ [("ERROR", "error", False), ("WARNING", "warning", False), ("INFO", "info", True)],+ )+ def test_gunicorn_options_honour_litellm_log(+ self, monkeypatch, litellm_log, expected_loglevel, access_log_on+ ):+ """LITELLM_LOG above INFO turns off gunicorn's per-request access lines"""+ pytest.importorskip("gunicorn")+ monkeypatch.setenv("LITELLM_LOG", litellm_log)++ captured: dict = {}++ def capture_run(self):+ captured["options"] = dict(self.options)+ captured["accesslog"] = self.cfg.accesslog+ captured["loglevel"] = self.cfg.loglevel++ with (+ patch("gunicorn.app.base.BaseApplication.run", capture_run),+ patch( # test-quality-ok: option tests must not start a thread or change the pytest worker's child ownership+ "litellm.proxy.proxy_cli.start_query_engine_reaper"+ ),+ ):+ ProxyInitializationHelpers._run_gunicorn_server(+ host="127.0.0.1",+ port=4012,+ app=MagicMock(),+ num_workers=1,+ ssl_certfile_path=None,+ ssl_keyfile_path=None,+ )++ assert captured["loglevel"] == expected_loglevel+ assert (captured["accesslog"] is not None) is access_log_on+ @pytest.mark.skipif(os.name == "nt", reason="gunicorn server path skips Windows") def test_gunicorn_jitter_without_base_warns(self): """gunicorn path warns when jitter is set without --max_requests_before_restart"""
litellm-proxy-extras/litellm_proxy_extras/_logging.pyindex ecf467fbf45..64e07a180d3 100644@@ -40,4 +40,4 @@ if not logger.handlers: logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s") ) logger.addHandler(handler)- logger.setLevel(logging.INFO)+ logger.setLevel(os.getenv("LITELLM_LOG", "INFO").upper())litellm/_logging.pyindex 03a9bcf21cf..873a6619a81 100644@@ -401,10 +401,14 @@ def _parse_json_logs_env(value: str | None) -> bool: return (value or "").lower() == "true" +def resolve_log_level(log_level: str) -> int:+ return getattr(logging, log_level.upper())++ json_logs: Final = _parse_json_logs_env(os.getenv("JSON_LOGS")) # Create a handler for the logger (you may need to adapt this based on your needs) log_level: Final = os.getenv("LITELLM_LOG", "DEBUG")-numeric_level: Final[str] = getattr(logging, log_level.upper())+numeric_level: Final[int] = resolve_log_level(log_level) handler: Final = LevelRoutingStreamHandler() handler.setLevel(numeric_level) handler.addFilter(_secret_filter)litellm/proxy/proxy_cli.pyindex 01a3da08998..b25c77f6828 100644@@ -261,7 +261,7 @@ class ProxyInitializationHelpers: import uvicorn import litellm- from litellm._logging import _get_uvicorn_json_log_config+ from litellm._logging import _get_uvicorn_json_log_config, resolve_log_level uvicorn_args: Final = { "app": "litellm.proxy.proxy_server:app",@@ -275,6 +275,8 @@ class ProxyInitializationHelpers: elif litellm.json_logs: # Use JSON log config for uvicorn to ensure all logs (including exceptions) are JSON uvicorn_args["log_config"] = _get_uvicorn_json_log_config()+ elif litellm_log := os.environ.get("LITELLM_LOG"):+ uvicorn_args["log_level"] = resolve_log_level(litellm_log) if keepalive_timeout is not None: uvicorn_args["timeout_keep_alive"] = keepalive_timeout if timeout_worker_healthcheck is not None:tests/litellm-proxy-extras/test_litellm_proxy_extras_logging.pynew file mode 100644index 00000000000..fb209bc3925@@ -0,0 +1,37 @@+import importlib+import logging+from collections.abc import Iterator++import pytest++import litellm_proxy_extras._logging as extras_logging+++@pytest.fixture+def fresh_extras_logger() -> Iterator[logging.Logger]:+ logger = logging.getLogger("litellm_proxy_extras")+ saved_handlers = logger.handlers[:]+ saved_level = logger.level+ logger.handlers[:] = []+ try:+ yield logger+ finally:+ logger.handlers[:] = saved_handlers+ logger.setLevel(saved_level)+++def test_litellm_log_error_silences_extras_info_lines(monkeypatch, fresh_extras_logger):+ monkeypatch.setenv("LITELLM_LOG", "ERROR")+ reloaded = importlib.reload(extras_logging).logger+ assert reloaded is fresh_extras_logger+ assert reloaded.isEnabledFor(logging.INFO) is False+ assert reloaded.isEnabledFor(logging.ERROR) is True+++@pytest.mark.parametrize("litellm_log", [None, "info", "DEBUG"])+def test_unset_or_verbose_litellm_log_keeps_extras_info_lines(monkeypatch, fresh_extras_logger, litellm_log):+ if litellm_log is None:+ monkeypatch.delenv("LITELLM_LOG", raising=False)+ else:+ monkeypatch.setenv("LITELLM_LOG", litellm_log)+ assert importlib.reload(extras_logging).logger.isEnabledFor(logging.INFO) is Truetests/test_litellm/proxy/test_proxy_cli.pyindex e25e6a59884..712c526b244 100644@@ -139,6 +139,35 @@ class TestProxyInitializationHelpers: ) assert args["timeout_worker_healthcheck"] == 15 + @staticmethod+ def _uvicorn_access_info_enabled(args: dict) -> bool:+ import logging++ loggers = tuple(logging.getLogger(n) for n in ("uvicorn", "uvicorn.error", "uvicorn.access", "uvicorn.asgi"))+ saved = tuple((lg, lg.handlers[:], lg.level, lg.propagate) for lg in loggers)+ try:+ uvicorn.Config(**args).configure_logging()+ return logging.getLogger("uvicorn.access").isEnabledFor(logging.INFO)+ finally:+ for lg, handlers, level, propagate in saved:+ lg.handlers[:] = handlers+ lg.setLevel(level)+ lg.propagate = propagate++ def test_litellm_log_error_silences_uvicorn_info_lines(self, monkeypatch):+ monkeypatch.setenv("LITELLM_LOG", "ERROR")+ args = ProxyInitializationHelpers._get_default_unvicorn_init_args("localhost", 8000)++ assert "log_config" not in args+ assert self._uvicorn_access_info_enabled(args) is False++ def test_unset_litellm_log_keeps_uvicorn_default_info_lines(self, monkeypatch):+ monkeypatch.delenv("LITELLM_LOG", raising=False)+ args = ProxyInitializationHelpers._get_default_unvicorn_init_args("localhost", 8000)++ assert "log_level" not in args+ assert self._uvicorn_access_info_enabled(args) is True+ def test_installed_uvicorn_supports_worker_flags(self): params = inspect.signature(uvicorn.Config.__init__).parameters assert "timeout_worker_healthcheck" in params