Simon Willison put a useful frame on something that has been building for a few years: cybersecurity now looks like proof of work, but the work falls entirely on the wrong side of the ledger.
In Bitcoin, proof of work means the cost to produce a valid block is high while the cost to verify one is trivially low. The asymmetry is intentional. It makes attacks expensive. The security analogy runs in the opposite direction: generating attacks, vulnerable code, and exploit scaffolding has gotten dramatically cheaper, while finding, triaging, and patching vulnerabilities requires the same slow, expensive human attention it always did. The asymmetry now favors attackers.
The Cost Curve Shifted for Attackers, Not Defenders
This is not a hypothetical future concern. A 2021 study by Pearce et al. found that roughly 40% of code suggestions from GitHub Copilot contained security vulnerabilities. That study used an older model. Models have improved considerably since then, but so has adoption, and the two do not cancel out. More developers accepting more suggestions from more capable tools means the attack surface scales with the quality improvement, not against it.
The vulnerability is structural: AI coding assistants optimize for “code that looks correct and runs,” not for “code that is secure.” They interpolate from training data, and training data contains a lot of vulnerable patterns. SQL queries without parameterization, JWT tokens stored in localStorage, unbounded deserialization calls, overly permissive CORS headers. These patterns appear in tutorials, in StackOverflow answers, in production codebases that got scraped. The model learned them.
What this produces is a large and growing body of code where the vulnerabilities are subtle, scattered, and contextually plausible. A human skimming an AI-generated function often cannot distinguish “looks fine” from “actually fine” without slowing down to audit carefully. Most people do not slow down.
The CVE Backlog Problem Is Not Separate From This
The National Vulnerability Database published 28,902 CVEs in 2023, and volumes in 2024 exceeded that. In early 2025, MITRE’s funding for NVD enrichment hit a crisis point when federal support was threatened, creating a backlog of thousands of CVEs sitting without analysis or scoring. The timing was not coincidental: the disclosure pipeline was already straining against a rate of new findings it was not designed to handle.
The people who write CVE analyses, build mitigations, update package advisories, and track exploit chains are not becoming more numerous at the same rate that vulnerabilities are being discovered. Automated scanning tools surface issues faster than teams can address them. Patch velocity has not kept pace with disclosure velocity.
This is the proof-of-work dynamic made concrete. Every new CVE requires real effort from a real person somewhere in the chain. The effort does not compress. A critical RCE in a widely deployed library requires the same careful analysis whether it was found in 2015 or 2026. But the rate at which such findings arrive is accelerating.
Prompt Injection Is the New Category That Fits the Pattern
The security problems introduced by AI systems themselves follow the same asymmetric structure. Prompt injection is now the first item on the OWASP Top 10 for LLM Applications. The attack is cheap to execute: an attacker embeds adversarial instructions in content the AI will read, whether that is a web page, a document, an email, or a tool response. The AI follows those instructions. The user does not see it happen.
Defending against this is substantially harder than mounting the attack. There is no reliable technical control that prevents an LLM from following injected instructions, because the LLM cannot reliably distinguish “instructions from the user” from “instructions that appear in content the user asked me to process.” Mitigations exist, they are layered, and they require ongoing maintenance as models and attack patterns evolve.
The indirect prompt injection variant, where the attacker does not interact with the AI directly but poisons content in the environment the AI agent will encounter, extends the attack surface to anywhere the agent has read access. A calendar invite. A README. A Slack message in a channel the bot monitors. For anyone building agentic systems, including the Discord bots I spend time on, this is a design constraint that touches every data source the bot reads.
Small Teams Cannot Absorb This
Large organizations with dedicated security engineering teams can at least throw headcount at the problem. The economics are brutal for everyone else.
A solo developer shipping a SaaS product with AI-generated code has neither the time nor the expertise to audit every function for subtle vulnerabilities. A two-person startup using an AI coding assistant to move faster has essentially traded development speed for security review that is not happening. This is not irresponsible; it is a rational response to the incentives. The cost of the compromise is deferred and often lands on users rather than builders.
The pattern has a historical parallel in spam. When sending email became nearly free, spam volumes exploded because the cost of sending one million messages was no longer meaningfully different from the cost of sending ten. Defenders had to build spam filters. Filters got better. Spam did not go away; it became a continuous maintenance burden. The defenders were never ahead, only approximately keeping pace.
AI-generated attack tooling, phishing content, and vulnerable code are following the same curve. The unit cost of attack dropped. The unit cost of defense did not.
What Might Actually Help
Some of the responses being developed are technically interesting, even if none of them fully solve the problem.
Formal verification tools like Lean 4 and Dafny can provide machine-checked proofs of program correctness for specific properties, including security-relevant invariants. The overhead is real: writing formal specifications is expensive, and most developers lack the training. But for critical infrastructure code, the cost is becoming defensible in a way it was not a decade ago.
Memory-safe languages reduce the surface for an entire class of vulnerabilities. Rust eliminates buffer overflows, use-after-free bugs, and data races at the language level. The Android team has published data showing that memory safety vulnerabilities dropped from 76% of Android security issues in 2019 to roughly 24% as the Rust proportion of the codebase grew. The gains are real and they compound.
AI-assisted security review is an attempt to restore some symmetry. Tools that use the same models generating the code to review it for vulnerabilities can catch some categories of issues, though they share the same blind spots as the generators. Using Semgrep or CodeQL rules tuned for AI-generated patterns is a different approach: static analysis that does not depend on the model understanding its own output.
None of these are complete answers. The structural problem, that generating attacks is cheap and defending against them is not, is not solved by any of them individually.
The Treadmill Is the Point
Willison’s framing resonates because proof of work has a specific quality: the work does not accumulate toward a goal. You produce it continuously to maintain a state, not to reach one. That is what security operations increasingly feel like. You patch, you monitor, you triage, and the queue does not shrink. The rate of new work matches or exceeds the rate of completion.
For developers building systems with significant attack surface, the most useful adjustment is probably attitudinal before it is technical: treat security review as a continuous process that scales with output volume, not a gate at the end of a project. AI tools increase output volume significantly. The review cost follows.
The asymmetry is not going away. Working with it honestly is more useful than hoping it resolves.