From Nasal Demons to Ghost Values: How C2Y Plans to Classify Undefined Behavior

C has lived with its undefined behavior problem since C89, and the community has mostly managed it through folklore, sanitizers, and a collective agreement not to look too directly at what compilers do with broken code. WG14 paper N3861, titled “Ghosts and Demons: Undefined Behavior in C2Y,” is part of a broader push in the upcoming C2Y standard to replace that folklore with something more formal and tractable.

The nasal demons problem

The term “nasal demons” traces back to a 1992 post on comp.std.c by John F. Woods, who responded to a question about what a particular undefined behavior would produce: it could, he wrote, make demons fly out of your nose. The joke stuck because it captures something real. When the C standard declares behavior undefined, it imposes zero constraints on the implementation. The compiler can assume that the undefined path never executes, which allows it to prove theorems about program state and eliminate entire branches of code.

The practical implication is that UB doesn’t just produce garbage at the site where it occurs. It allows the optimizer to modify code that runs before the UB point. John Regehr documented this in his Guide to Undefined Behavior in C and C++, and Chris Lattner wrote a three-part series on it for the LLVM blog in 2011 and 2012. Both are worth reading to understand why “just avoid UB” is harder than it sounds when the compiler is using UB to reason backward through your program.

The signed integer overflow case is the canonical example:

int always_increases(int x) {
    return x + 1 > x;
}

With optimization enabled, GCC and Clang compile this to return 1;. The compiler reasons that signed overflow is undefined, therefore x + 1 is always greater than x, and the function collapses to a constant. This is correct reasoning within the C abstract machine, but it regularly surprises programmers who think in terms of two’s complement hardware behavior.

Why one category is not enough

The trouble with the current standard is that “undefined behavior” is a single category covering at least five distinct situations with different origins, different implications, and different possible remedies.

Optimizable UB exists specifically to enable compiler analysis. Signed integer overflow is the primary example: by leaving it undefined, the compiler can assume loop induction variables don’t wrap and apply transformations that would otherwise require range proofs. Eliminating this UB would force the compiler to conservatively assume overflow is possible, which costs real optimization opportunities in numerical and loop-heavy code.

Hardware-mapped UB corresponds to operations that trap on real hardware: division by zero, misaligned access on strict-alignment architectures. The standard leaves these undefined because hardware behavior varies by target; each implementation defines them, but portably the behavior cannot be specified.

Memory safety UB covers out-of-bounds access, use-after-free, and data races. These are dangerous not primarily because of optimizer transformations but because they corrupt program state in ways that undermine all subsequent reasoning, and they are the origin of most exploitable vulnerabilities in production C code.

Portability UB arises from implementation differences: the width of int, the signedness of char, the behavior of right-shifting a negative integer. Each implementation has well-defined behavior; the standard just doesn’t mandate which.

Indeterminate value UB is what happens when you read an uninitialized variable. The value is neither a defined quantity nor a trap representation; it is genuinely unknowable within the abstract machine. This is what N3861 calls “ghost values,” and it has properties that make it worth treating separately from the rest.

Ghost values and erroneous behavior

The ghost value concept draws on a distinction the LLVM compiler infrastructure developed over years of representing uncertain values in its intermediate representation. LLVM IR has two “bad value” kinds: undef, which can be materialized as any bit pattern at each use site independently, and poison, which propagates through computations until it reaches a side-effectful operation. The difference matters because an undef in two places can produce two different values at runtime, while a poison value taints every computation that depends on it.

Uninitialized variables in C map roughly to poison in LLVM’s model. The consequence is that the compiler doesn’t simply read whatever bytes happen to be in the register or stack slot; it uses the undefined read to prove that certain execution paths are unreachable:

int x;
int y = x;
if (y == 42) {
    launch_missiles();
}

Under current C, the compiler can reason that y comes from an indeterminate read, that any specific value of y would require a defined program state that doesn’t exist here, and therefore this branch is unreachable and can be deleted. The function body silently disappears.

C2Y is looking at borrowing the “erroneous behavior” concept that C++26 adopted for this case, via WG21 paper P2795R5. Erroneous behavior means the compiler may produce any value for an uninitialized read but cannot use it to prove the execution path is unreachable. The if (y == 42) branch survives. The value of y is still arbitrary, but the code behaves like code rather than like a theorem the optimizer can refute. Sanitizers can detect erroneous behavior reliably and with well-defined output, because the optimizer can no longer erase the evidence before the sanitizer sees it.

The strict aliasing situation

Strict aliasing is a different kind of UB that C2Y is also examining. The rule says objects of incompatible types cannot alias each other, which allows the compiler to assume that writes through an int* don’t affect values read through a float*. This enables alias analysis that would otherwise require conservative assumptions across most pointer dereferences.

The problem is that a lot of real C code violates strict aliasing, particularly in networking stacks, binary serialization, and hardware register access:

float f = 3.14f;
int i = *(int *)&f; // UB: accessing float storage through int pointer

The standard-compliant version uses memcpy, which the optimizer recognizes and compiles to a single register move:

int i;
memcpy(&i, &f, sizeof(i)); // defined behavior; compiler elides the copy

The violation in the first version is silent. Compilers don’t warn on the pointer cast by default; the code compiles and appears to work until optimization is turned up, at which point the alias assumption allows the compiler to reorder or eliminate the read. C2Y discussions have included proposals to define certain type-punning patterns explicitly, carving them out of the “anything goes” UB category without eliminating alias analysis for cases that remain undefined.

Pointer provenance

One of the larger efforts feeding into C2Y is the formal treatment of pointer provenance. The Cerberus project at Cambridge, led by Peter Sewell, has produced a formal C semantics that treats every pointer as carrying metadata about which allocation it was derived from. Two pointers with the same address but different provenance are not interchangeable, and using a pointer outside its provenance is UB.

This formalizes what compilers already assume but what the standard doesn’t say clearly. The PNVI (Provenance Not Via Integers) model specifically addresses what happens when you cast a pointer to uintptr_t and back: the resulting pointer may have the same bit representation as the original but is no longer guaranteed to carry the same provenance. Code that round-trips pointers through integers, or does pointer arithmetic between unrelated allocations, sits in a space where the C abstract machine and hardware behavior diverge in ways that are hard to reason about without a formal model.

Papers in the N2577 series addressing provenance have been in active WG14 discussion since the C17 era, carried into C23, and remain a major topic for C2Y.

What other languages have done

C++20 required two’s complement representation for all signed integers, eliminating portability UB for integer representation while leaving overflow UB in place. C++23 added [[assume(expr)]] to make UB-based assumptions explicit in source code, and std::unreachable() to replace the __builtin_unreachable() idiom. C++26 added erroneous behavior for uninitialized reads, the same concept C2Y is now examining for C.

Rust takes a different approach: safe code has no undefined behavior by definition. Signed integer overflow wraps in release builds and panics in debug builds. Uninitialized memory requires explicit MaybeUninit<T> wrappers. The alias analysis guarantees that C gets from strict aliasing UB come instead from the borrow checker’s ownership and lifetime rules.

Zig makes the tradeoff explicit and tunable. Operations that would be UB in C are safety-checked in debug and safe release modes, producing well-defined trap output. In release-fast mode they revert to unchecked UB for performance. The programmer picks the tradeoff at build time rather than inferring it from optimization flags.

C cannot adopt Rust’s model wholesale: the existing codebase is too large, and the language is used in contexts where the programmer must control every aspect of memory layout and aliasing. But the C2Y taxonomy work is trying to give that control a formal basis, so the tradeoffs are visible and auditable rather than emergent properties of whatever optimizations the compiler happens to perform.

What N3861 represents

The paper title does real work. “Demons” invokes the nasal demons tradition: the acknowledgment that certain undefined behaviors represent genuine unbounded chaos in the abstract machine. “Ghosts” is the newer formal concept: values that exist and propagate through computation without being grounded in any specific representation, haunting later code in ways that are hard to reason about but are not, strictly speaking, infinite chaos.

The distinction matters because ghosts and demons require different remedies. A ghost value can potentially be tamed by reclassifying it as erroneous behavior, giving it a predictable interaction with sanitizers and a narrower set of permitted compiler exploitations. A demon, in this framing, is UB that cannot be constrained without abandoning the optimization or hardware guarantee it exists to enable, and it may need to stay exactly as undefined as it is.

If C2Y adopts even a partial version of this taxonomy, the visible effects would include better sanitizer output, more precise compiler documentation, and eventually the possibility of auditing which UB categories a given compilation is exploiting. For code running in kernels, firmware, and safety-critical systems, that audit capability is not a minor convenience. It is what separates “we think this is safe” from “we can demonstrate why this is safe.”