There is a post making the rounds on Hacker News that argues AI coding is gambling, and the 300-plus upvotes and nearly 400 comments suggest it landed on something real. The comparison gets dismissed as hyperbole in a lot of the replies, but it is more technically precise than it first appears, and the reasons why are worth working through.
The Probabilistic Core
When you ask an LLM to write code, you are not querying a deterministic function. You are sampling from a probability distribution over token sequences. The model assigns a probability to each possible next token, and the sampling process introduces randomness at every step. The temperature parameter in most LLM APIs controls how peaked that distribution is. At temperature=0.0, the model greedily selects the highest-probability token at each step, producing more consistent outputs. Consistency is not correctness.
# OpenAI-compatible call with temperature set to "deterministic"
response = client.chat.completions.create(
model="gpt-4o",
temperature=0.0,
messages=[{
"role": "user",
"content": "Implement a thread-safe counter in Python"
}]
)
# Temperature=0 narrows the output distribution.
# It does not collapse it to a single guaranteed-correct answer.
# Different model versions, quantization configs, and inference
# hardware can still produce different outputs for the same prompt.Even at temperature=0, outputs are not fully stable across model versions, quantization configurations, or the distributed GPU clusters that run inference at scale. Floating-point arithmetic on many-GPU setups is not reproducible in the way that classical software is. This is not a bug to be patched. The same probabilistic mechanism that allows LLMs to generalize across domains is the one that makes their outputs vary. You are always sampling from a distribution. Sometimes you sample from a region near a correct implementation. Sometimes you do not, and the distance from correct can be large while remaining visually indistinguishable from right.
Why “Gambling” Captures This
The comparison is apt not just because the outcome is uncertain, but because of the specific shape of that uncertainty. B.F. Skinner’s work on reinforcement schedules established that variable ratio reinforcement, where rewards arrive unpredictably after a varying number of attempts, produces the most persistent behavior. This is the schedule slot machines use. It is also the schedule that governs prompting an AI coding assistant.
You write a prompt, the code looks reasonable, you run it and it works. You repeat this pattern several times and it keeps working. Then you write something slightly more complex and the output is wrong in a way that takes two hours to debug. The rational response is to update your model of what the tool does reliably. The common response is to rephrase the prompt and try again, because the last ten times it worked. The wins condition behavior more strongly than the losses discourage it; the rewards keep arriving, just not predictably.
This is not a character flaw. It is a well-studied behavioral response to a specific reward schedule. Recognizing it as such is the first step to working around it.
The Confidence Problem
What makes AI coding failure modes genuinely risky, rather than just frustrating, is their low visibility. A syntax error gives you immediate feedback. A hallucinated method that almost matches a real API, an off-by-one in a boundary condition, an incorrect assumption about concurrent access: these look like code. They pass casual review. They often pass linters. They fail at runtime, in production, or in the edge case that tests did not cover.
Consider a class of async bug that LLMs generate with surprising frequency:
# AI-generated version: correct structure, race condition
async def process_payment(user_id: int, amount: float, db):
user = await db.get_user(user_id)
if user.balance >= amount:
# Another coroutine may have modified user.balance
# between this read and the update below
await db.update_balance(user_id, user.balance - amount)
return True
return False
# Correct version: check and update within a transaction
async def process_payment_safe(user_id: int, amount: float, db):
async with db.transaction() as tx:
user = await tx.get_user_for_update(user_id)
if user.balance >= amount:
await tx.update_balance(user_id, user.balance - amount)
return True
return FalseThe first version uses the right API names, the right async patterns, the right conditional logic. It fails under concurrent load. LLMs generate this pattern frequently because the simpler, incorrect version appears more often in training data than the transactional alternative. The model is not guessing randomly; it is reproducing the most common pattern it has seen. The most common pattern is not always correct.
Research examining GitHub Copilot-generated code found that roughly 40% of generated programs in security-sensitive contexts contained at least one vulnerability, spread across injection flaws, out-of-bounds access, and cryptographic misuse. A separate 2023 study analyzing AI responses to programming questions found that incorrect answers were expressed more confidently than correct ones at a measurable rate. The confidence of an LLM output does not correlate with its correctness. Our instincts for evaluating code are calibrated on human-written programs, not on outputs from probabilistic samplers. The mismatch matters.
I run into this regularly building Discord bots. The Discord.js API is well-represented in training data, and AI assistants produce working bot code quickly for standard patterns. The moment you need something non-standard, say, composing an interaction collector with a modal submission and an async write to a database, the output uses the right API names and follows the right overall shape but fails on timing. The interaction expires before the database write completes. The reply fires before the state is committed. These bugs do not announce themselves in development; they appear under load or with slow storage.
The Skill Gap That Moves the Odds
The gambling metaphor breaks down in one meaningful way: the odds are not fixed. Experienced developers get better expected value from these tools because they can evaluate the output faster. They know the APIs well enough to spot a hallucinated method. They have enough domain context to see when the generated logic does not match the actual problem. They write tests that exercise the behavior they care about rather than the behavior the AI assumed they cared about.
For a developer who already knows how to write the code, AI assistance works more like a fast typist than a slot machine. The value is throughput, not knowledge. You are not gambling on whether the output is correct; you are sampling from a distribution you understand well enough to evaluate cheaply. The expected value calculation comes out positive because the evaluation cost is low.
For a developer using AI to work outside their expertise, that calculus reverses. Incorrect outputs are harder to identify. Debugging takes longer. The confidence the output projects is more misleading, not because the model is more confident on unfamiliar terrain, but because the developer has fewer tools for detecting incorrectness. This is where the gambling framing is most accurate, and it is the use case these tools are increasingly marketed toward.
Managing the Variance
Treating AI coding output as probabilistic rather than deterministic changes how you should integrate it into a workflow.
Tests are the clearest mitigation. A suite that exercises actual behavior converts the question “is this output correct” from a visual inspection problem to an empirical one. This matters more when the code under review was sampled from a distribution rather than written with explicit intent.
Scope control helps substantially. AI tools perform better on small, well-specified tasks with clear interfaces than on large, loosely-defined ones. A function with a documented input-output contract is a better prompt target than “implement the authentication layer.” Narrower problems correspond to tighter output distributions, and tighter distributions sample more reliably from correct implementations.
Understanding the training distribution gives you calibration. These models are stronger on well-documented, widely-used APIs with extensive public example code. They are weaker on newer APIs, internal libraries, unusual API compositions, and any pattern that appears rarely in public repositories. The weaker regions are also where the high-confidence-but-wrong failure mode is most dangerous.
Maintaining the ability to evaluate the output is not optional. Using AI to code in domains where you cannot assess correctness removes the feedback loop entirely. The skill of knowing what correct looks like is not separable from the skill of programming; it is the central part of it.
The variable reward schedule makes these tools compelling to use. The losses are often invisible until later, and the wins keep arriving, just not predictably. That combination is worth being clear-eyed about. Not to avoid the tools, but to use them with the same risk management you would apply to any system whose outputs are probabilistic by design.