Tagged Words and Writable Microcode: Inside the Lisp Machine's Hardware Layer

The scheatkode post on hot-wiring the Lisp machine treats the Lisp machine as something you can reach into at a level most people never see. That framing is apt. Lisp machines were unusual not just because they ran Lisp natively, but because the entire hardware stack was designed so that the language environment and the machine’s lowest operational level were separated by a remarkably thin membrane. Understanding where that membrane sits explains what these machines were, and what getting beneath them actually involves.

Purpose-Built for One Language

The canonical Lisp machine was the MIT CADR, designed at the MIT AI Lab in the mid-1970s by Richard Greenblatt, Thomas Knight, and others. It was not a general-purpose processor with a Lisp compiler layered on top. The instruction set, word format, memory model, and hardware support structures were all designed to make Lisp primitives cheap to execute. Two commercial successors followed: Symbolics, which produced the 3600, 3640, 3670, and XL series, and Lisp Machine Inc (LMI), which produced the Lambda. Both were direct descendants of the CADR’s architectural choices.

The CADR processed 32-bit words, and every word in the system carried a 5-bit data type tag in its high bits, leaving 27 bits for the actual value or address:

 31       27 26                          0
 +----------+-----------------------------+
 | data type |        data/address        |
 +----------+-----------------------------+
       5 bits              27 bits

This gave 32 possible hardware-visible data types. The full list included DTP-FIX for fixnums, DTP-SYMBOL, DTP-CONS, DTP-ARRAY-POINTER, DTP-CLOSURE, DTP-STACK-GROUP, DTP-U-ENTRY for microcode entry points, and about two dozen more. When the processor fetched a word, it already knew its type without any additional memory load or metadata lookup. Type dispatch was structurally free at the hardware level.

The 27-bit address field meant the CADR could directly address 128 megawords, which in the late 1970s was generous. The Symbolics 3600 later extended this to a 40-bit word format with a larger tag field and a wider address space, but the fundamental commitment to per-word tagging carried through every generation.

CDR Coding

One hardware optimization that gets less attention than tagging is CDR coding. A cons cell normally occupies two machine words: the CAR and the CDR. If a list is laid out sequentially in memory, however, the CDR of each cell points to the immediately following address, which is trivially derivable. The CADR hardware understood this and could omit the CDR storage entirely for sequential cons pairs. Each word carried a 2-bit CDR code in addition to its type tag, indicating whether the CDR was stored explicitly, was the next sequential address, or terminated the list:

CDR-NORMAL   (0) - CDR is stored in the next word
CDR-NIL      (1) - this is the end of the list  
CDR-NEXT     (2) - CDR is implicitly the next memory location
CDR-ERROR    (3) - invalid

Lists that happened to be allocated sequentially consumed roughly half the memory of conventional cons cells. The garbage collector could also compact lists into CDR-coded form during collection. For Lisp code that creates and traverses many short lists, this was a meaningful memory win in an era when DRAM was expensive.

The Microcode Layer and the Spy

Beneath the Lisp instruction set sat the CADR’s writable control store: 1024 words of 48-bit-wide microcode, loaded from ROM at cold start and patchable at runtime. Each Lisp macro-instruction dispatched through a table into microcode. The microcode controlled the ALU, memory bus, register file, and next-instruction address directly. It was the real machine the hardware executed; the Lisp instruction set was a layer of abstraction above it.

This writable control store is the hinge of what “hot wiring” means in this context. Unlike a conventional CPU where the instruction set is fixed in silicon, the CADR’s control store could be modified after boot. The MIT team shipped a microcode-level debugger called the spy, which could halt the main processor mid-execution and step through individual microinstructions. From the spy, you could inspect every register, modify control store locations, and resume execution. Crucially, the spy ran on a separate minimal embedded processor with a direct view of the main machine’s internal state, not on top of the Lisp environment. It did not require the Lisp system to be functional, or even booted.

A typical CADR microinstruction encoded:

• Source and destination register selects from a 32-register file
• ALU operation (add, subtract, logical, shift)
• Memory bus direction and address source
• A next-address field for microcode branching
• Condition bits for conditional micro-branches
• A functional destination field for special-purpose hardware units

The CADR also had a hardware stack for function call frames, a dedicated register for the current function’s argument list, and specialized micro-operations for tail-call elimination. Tail calls that could be detected at compile time were compiled to micro-operations that reused the current stack frame rather than allocating a new one. This was not a compiler optimization added on top of a conventional calling convention; it was expressed directly in the microcode.

Cold Loading and the Boot Model

The running Lisp environment was stored on disk in a snapshot called a world or a band. A band was not an executable binary in the conventional sense. It was a raw memory image of the entire Lisp environment: all loaded code, all live objects on the heap, all compiled functions, and the complete system state at the moment the band was saved. Booting meant loading that band back into physical memory and resuming execution from a known restart entry point.

Before the band could load, the machine ran a cold load: a minimal Lisp interpreter just capable of reading the band off disk and restoring the heap. This cold load was stored in ROM or in a dedicated partition on disk. On Symbolics hardware, a separate Front End Processor (FEP) based on a Motorola 68000 managed this process entirely. The FEP ran its own minimal operating system, could list and load bands, could invoke the spy against the main processor, and could perform diagnostics independently of whether the main Lisp system was operable.

Accessing the FEP directly bypassed the Lisp environment completely. You could swap bands, patch microcode in the writable control store, halt the main CPU, and inspect its register state without any involvement from the Lisp system. This was not a back door or a workaround; it was the intended architecture for system maintenance and debugging.

Running These Machines Today

The CADR’s source code, including its microcode, was released by MIT under an open-source license and is part of the ITS source repository on GitHub. Brad Parker wrote a CADR emulator in C that models the hardware down to the memory bus and writable control store. Running the original MIT system software under emulation lets you load bands, invoke the spy, step through microinstructions, and observe the tagged word format at work, all things that would have required access to physical hardware forty years ago.

Symbolics took a different path with Open Genera, released in the late 1990s: a port of the Genera 8.3 system to DEC Alpha hardware. Rather than emulating the 3600’s physical hardware, Open Genera compiled the Lisp machine’s object code to run natively on the Alpha. The community later produced VLM (Virtual Lisp Machine), which runs Symbolics object code on modern x86 hardware under emulation. The complete Genera environment, including the Zmacs editor, the Inspector, the Lisp listener, and the CLIM GUI toolkit, is runnable today.

The two approaches answer different questions. The CADR emulator gives you the hardware layer and the spy; the Genera path gives you the highest-level environment. For exploring what the “hot wiring” metaphor actually means in practice, the CADR path is the one worth following, because the writable control store and the spy facility are present and accessible.

Ideas That Took the Long Route

Several things the Lisp machine designers built into hardware in the 1970s are being built into hardware again now, under different names and for different stated reasons.

Tagged memory is back. ARM MTE (Memory Tagging Extension) adds 4-bit tags to memory allocations for spatial and temporal safety. CHERI, the capability hardware from Cambridge, uses tagged pointers to enforce provenance. The observation that per-word type metadata is useful enough to embed in hardware was made first by Lisp machine designers; contemporary CPU architects are reaching the same conclusion from the direction of memory safety rather than language performance.

Writable microcode, which looked like a research facility in the CADR, is the mechanism Intel and AMD use to deploy errata fixes. When Intel shipped microcode updates for Spectre, they were using the same basic capability the CADR exposed to its users as a debugging tool, just with the user-facing modification interface removed.

The band-snapshot boot model maps directly to CRIU container checkpointing, to VM snapshots, and to the persistent-world model that Smalltalk implementations used. Treating the running system state as the primary artifact, with source code as a way to regenerate it, never became mainstream in Unix-derived systems. It keeps reappearing wherever fast startup and persistent state matter enough: serverless platforms that want sub-millisecond cold starts, language runtimes experimenting with heap serialization, and browser engines that cache compiled JavaScript across sessions.

Lisp machines were expensive, proprietary, and displaced by workstations running Common Lisp on commodity hardware. The architectural ideas took a longer route to influence. Most of them got there.