Splitting LLVM One Function at a Time: Zig's Path to Incremental Compilation

Incremental compilation is one of those problems that looks solved until you actually try to solve it through LLVM. The Zig project’s April 8 devlog entry documents progress on exactly this: making the LLVM backend participate in incremental rebuilds, so that a one-line change does not force the compiler to regenerate and re-optimize an entire program’s worth of IR.

To understand why this is worth a devlog entry, you need to understand what LLVM was built to do, and how that assumption shapes everything downstream.

LLVM’s Foundational Assumption

LLVM’s compilation model starts with a Module: a complete, self-contained IR graph representing a translation unit or, in link-time optimization mode, an entire program. The optimization pipeline operates on this Module. Passes like inlining, interprocedural constant propagation, and alias analysis all work by examining the full module at once. The backend then lowers the optimized Module to machine code.

This is a perfectly coherent design for release builds. The problem is that it treats compilation as a pure function from source to binary, with no memory of what came before. Every build is a cold build, as far as LLVM is concerned. If you change one function, you rebuild the module, re-run the optimization pipeline over everything, and emit a new object file. For a program with tens of thousands of functions, this gets expensive fast.

Clang addresses this at the translation unit level: each .c file produces its own Module, compiled independently, and the linker combines the object files. Change one .c file, and only its Module is recompiled. This is fast enough for C’s compilation model because .c files are already the natural unit of isolation. But for languages like Zig, Rust, or Swift, where a single source file can depend densely on types and functions defined in other files, translation-unit-level granularity is often too coarse to matter.

What the Self-Hosted Backend Already Does

Zig’s self-hosted compiler, written in Zig rather than C++, was designed around incremental compilation from the beginning. The x86_64, ARM, RISC-V, and WebAssembly backends all work at function granularity: each function is compiled independently, the compiler tracks which functions depend on which declarations, and a change propagates only as far as the dependency graph requires.

The mechanism is concrete. Functions are emitted into a memory-mapped output file. When a function changes, the compiler rewrites only that function’s machine code region and updates a fixup table for call targets and relocations. The InternPool, a global interning table for types, values, and declarations, allows structural sharing so that type changes can be precisely scoped. This is not a heuristic; it is the fundamental architecture of the compiler.

Lazy semantic analysis completes the picture. Declarations are not analyzed until they are referenced. A change to an unreferenced declaration costs nothing. The compiler builds and maintains a directed dependency graph, and on each rebuild, it walks only the subgraph reachable from changed declarations.

This works well and produces fast rebuilds. The catch is that the self-hosted backends do not yet produce code as good as LLVM for release builds. Users who need optimization still route through the LLVM backend, which historically provided no incremental compilation at all.

The LLVM Backend Problem

The challenge of making LLVM incremental is not primarily a missing API. LLVM’s Module type is mutable; you can add and remove functions. The challenge is that LLVM’s optimization passes assume a stable, consistent module. An inlining pass, for example, makes decisions based on what all the functions in the module look like. If you replace one function’s body and do not re-run the relevant passes, you may produce code that is internally inconsistent or that misses important optimizations.

More fundamentally, if two functions A and B are in the same Module and A was inlined into B during a previous compilation, and you now change A, you need to undo that inlining in B, recompile both, and re-run the inliner. There is no “undo” in LLVM’s optimization pipeline. The pipeline runs forward.

One approach to this is ThinLTO, which LLVM and Clang already support. Each translation unit is compiled to a bitcode file with a summary index. The linker uses the summary to identify cross-unit inlining candidates and imports only those. Unchanged modules skip re-optimization. This gives incremental behavior at module granularity, which for Clang means per-.c-file. For a language that compiles the whole program as one module, ThinLTO is not directly useful without restructuring how the module is partitioned.

Zig’s approach, visible in the devlog and in the design of the self-hosted compiler, is to create per-function LLVM Modules. Each function gets its own small Module, independently lowered and compiled to machine code by LLVM. Changed functions produce new small Modules; unchanged functions reuse their previous machine code. The results are stitched together by the linker.

This has real costs. LLVM cannot inline across Module boundaries without LTO, so per-function Modules effectively disable inlining by default for incremental builds. Type deduplication becomes a coordination problem: LLVM’s type system expects that structurally identical types within a compilation are the same object. When types span Modules, the compiler must declare them consistently across all the small Modules that reference them. Global variable references require external declarations in every Module that touches them. Debug info, which uses DWARF’s complex cross-reference model, must be managed carefully to avoid inconsistency.

These are solvable problems, and the devlog documents that progress is being made on them. The key insight driving the approach is that for debug and development builds, the loss of inlining is acceptable. Users rebuilding frequently to test a change do not need the fully optimized binary; they need a correct binary, fast. Release builds continue to use LLVM in its conventional whole-module mode with full optimization.

How Rust and Go Solve the Same Problem

Rust’s incremental compilation operates at a different level of the stack. The compiler divides a crate into codegen units (CGUs), each compiled to a separate LLVM Module. The query system tracks which computations are affected by a source change, and only CGUs containing affected functions are recompiled. The rest are reused from a disk cache.

This is effective and has been stable since around 2018. The tradeoff is that CGU granularity is coarser than function granularity. A widely-used internal type change can invalidate many CGUs, each of which must be fully recompiled and re-linked. The dependency tracking operates at the level of “which CGUs need recompiling,” not “which functions within a CGU.”

Go sidesteps the LLVM problem entirely by using its own backend. The gc compiler operates at package granularity: packages are compiled independently, export data describes each package’s public API in a compact binary format, and downstream packages only need to re-examine that data rather than re-parse source. The go build cache is content-addressed, so a package that has not changed and whose dependencies have not changed their exported API is simply not recompiled. This produces some of the fastest incremental build times in any compiled language, at the cost of whole-program optimization.

Zig’s approach is more ambitious than Go’s because it aims to preserve LLVM optimization quality for release builds while achieving fast incremental rebuilds for development. The self-hosted compiler handles the dependency tracking and the per-function module splitting; LLVM handles optimization within each small module. The incremental build uses the split-module path; the release build recombines everything into a conventional whole-program LLVM compilation.

What This Milestone Means

Getting incremental compilation to work through the LLVM backend is significant because it removes a real productivity cost that Zig users have lived with. The self-hosted backends, while improving, do not yet cover every target architecture or produce code quality sufficient for all production use cases. Users targeting those platforms, or who want LLVM’s optimization quality even in debug builds, have had to accept full rebuilds on every change.

The per-function Module approach also has implications for how LLVM is used beyond Zig. The idea of treating LLVM as a per-function code generator rather than a whole-module optimizer is a valid architecture for any compiler that wants fine-grained incremental compilation. LLVM’s ORC JIT framework already operates on this principle for JIT use cases; applying it to ahead-of-time compilation is a different engineering challenge but uses similar conceptual foundations from the ORC JIT design.

The fundamental tension between incremental compilation and cross-function optimization is real and does not disappear with this approach. A development build that skips inlining is a different artifact from a release build that enables it. Profiling a development build can be misleading; behavior under full optimization can differ. These tradeoffs are well-understood and unavoidable, and Zig’s design acknowledges them by treating the two build modes as genuinely different rather than trying to collapse them into one.

What the April 2026 devlog documents is that the infrastructure for making this split work correctly is coming together. That is a prerequisite for everything else: a compiler that rebuilds instantly during development and optimizes aggressively for release, with a single toolchain and no external build system required.