When Your LLM Proxy Steals Your Keys: The litellm Supply Chain Attack

Supply chain attacks against Python packages are not new, but the litellm incident that Simon Willison documented is worth studying in detail because of how it combines a clever execution mechanism with a high-value target. litellm is one of the most popular unified LLM proxy libraries in the Python ecosystem. It sits between your application code and a dozen or more AI provider APIs, which means the developers who depend on it tend to have a lot of valuable credentials sitting in their environment.

Version 1.82.8 of litellm shipped with a file called litellm_init.pth. The name sounds innocuous, almost like legitimate package initialization code. The .pth extension is where the real danger lives.

What .pth Files Actually Do

Python’s .pth file mechanism is a path configuration feature that has existed since Python 2. When the interpreter starts, it scans each directory in sys.path for files ending in .pth and processes them line by line. Lines that start with import are executed as Python statements at interpreter startup, before your application code runs, before any virtual environment activation hooks fire, and before you have any meaningful opportunity to intercept them.

Here is what that looks like in a simplified form:

# site-packages/litellm_init.pth
import os; exec(__import__('base64').b64decode(b'aW1wb3J0IG9z...'))

The obfuscated payload decodes and executes immediately when Python starts. You do not have to import litellm. You do not have to run your application. Any Python process that uses the same site-packages directory where litellm installed will trigger the payload.

This is one of the most powerful persistence and execution mechanisms available to a malicious Python package, and it is largely overlooked in discussions about PyPI security. Most attention goes to setup.py execution during installation, which pip install --no-build-isolation and similar flags can sometimes mitigate. The .pth trick bypasses all of that: it fires on every subsequent Python invocation in the affected environment.

Why AI Developers Are a High-Value Target

litellm’s whole value proposition is that it gives you a single interface to call OpenAI, Anthropic, Google, Cohere, Azure OpenAI, Mistral, and roughly a hundred other providers. That means a typical litellm user has API keys for multiple services stored in their environment, often in a .env file or exported shell variables.

These are not cheap credentials. An OpenAI API key with a generous spending limit, an Anthropic API key attached to a production account, Azure credentials with broader IAM scope, all of these are worth real money to an attacker. They can be used directly to run inference at the victim’s expense, or sold, or used to exfiltrate model outputs from proprietary fine-tuned deployments.

The attack payload in 1.82.8 targeted exactly this surface: environment variables containing API keys and tokens. The malicious code collected credentials from os.environ and exfiltrated them to an external endpoint. Given that litellm is heavily used in agentic frameworks, CI pipelines, and development tooling, a single compromised install could expose credentials for an entire organization.

The Discovery and Timeline

The malicious version appears to have been caught relatively quickly, though “relatively quickly” in supply chain attack terms still means an unknown number of developers ran pip install litellm or had automated dependency updates pull in 1.82.8 before the issue was identified. PyPI does not publish real-time download statistics granular enough to know exactly how many installs occurred before the package was flagged.

This is a recurring problem with the PyPI incident response model. Detection relies on researchers, automated scanners like Phylum or Socket, or vigilant users noticing anomalous behavior. The turnaround from upload to takedown has improved, but there is always a window.

The litellm maintainers themselves appear to have been victims here rather than perpetrators. The most likely explanation is a compromised publishing credential, a maintainer account with a weak or reused password, or a token stored insecurely that an attacker harvested and used to push a poisoned release. This is how most legitimate-package supply chain attacks work: the attacker does not need to compromise the codebase or trick maintainers into merging malicious code. They just need the PyPI token.

How to Audit Your Environment

If you installed litellm recently or have automated dependency updates, checking your installed version is the first step:

pip show litellm

But that only tells you the current version. If 1.82.8 was installed and then updated, the malicious .pth file may still be present in your site-packages directory. You need to check directly:

python -c "import site; print(site.getsitepackages())"

Then look in those directories for any .pth files that do not belong:

find /path/to/site-packages -name "*.pth" -newer /path/to/some/reference -exec cat {} \;

More broadly, auditing all .pth files in your environment is good hygiene regardless of this incident:

find $(python -c "import site; print(' '.join(site.getsitepackages()))") -name "*.pth" | xargs grep -l 'import'

Any .pth file with an import line that you did not put there deliberately warrants scrutiny. Legitimate packages occasionally use this mechanism for namespace packages and path manipulation, but executable import lines are much rarer and should be explainable.

If your environment was exposed, rotate all API keys immediately. Treat any credentials that were present in os.environ or .env files during the window when 1.82.8 was installed as compromised. Check your provider billing dashboards for unusual usage.

The Broader Problem

The AI and machine learning Python ecosystem has accumulated an enormous attack surface over the past three years. The libraries that make LLM development convenient, litellm, LangChain, LlamaIndex, Hugging Face’s various packages, are also libraries with wide privilege. They handle credentials, they make network requests, they often run in environments with access to sensitive data and internal services.

The pip install workflow that the community has normalized, pulling packages directly from PyPI with minimal verification, is a liability at this scale. Tools like pip-audit can catch known vulnerabilities but not novel malicious packages. Private package mirrors with explicit version pinning and hash verification (--require-hashes in pip) are the most effective mitigation, though they add operational overhead that most projects avoid.

For packages where you genuinely need to track upstream versions, tools like Dependabot or Renovate can help, but they need to be paired with a review step rather than auto-merging every version bump. A single .pth line is invisible in a diff that otherwise contains only version number changes.

There is also a credential hygiene angle here. Environment variables are a convenient way to pass secrets to applications, and they are also trivially readable by any code running in the same process. Secret stores with runtime injection, short-lived tokens, and scoped credentials with spending limits reduce the blast radius when something like this happens. An API key that can spend ten dollars before hitting a hard limit is a much less valuable target than one attached to a production account with no ceiling.

The litellm incident is a clean example of how the concentration of valuable credentials in AI development environments makes the ecosystem a worthwhile target. The .pth mechanism gave the attacker reliable execution. The user base gave them a high probability of finding something worth stealing. Both of those facts are useful to keep in mind the next time you run an unreviewed pip install in an environment where your production API keys live.