Three Allocators, Three Bets: What Makes jemalloc Worth Continued Investment
Source: hackernews
Meta’s renewed commitment to jemalloc arrives at an unusual moment for allocator development. Microsoft’s mimalloc benchmarks competitively against nearly everything. Google’s TCMalloc has been comprehensively rewritten around per-CPU caches and regularly posts favorable throughput numbers. Scudo ships in Android. If you were evaluating allocators from scratch today, you would have more credible options than at any previous point.
The case for investing in jemalloc anyway is not primarily about throughput. It is about what each of these systems is actually designed to be, and those designs differ enough that comparing benchmark numbers without that context produces misleading conclusions.
What Per-CPU Caching Optimizes
Modern TCMalloc, the version shipping since roughly 2020 and described in Google’s published architecture, centers its fast path on per-CPU caches. Each logical CPU has a dedicated slab for each size class. Allocation and deallocation access the current CPU’s slab directly; no lock is acquired, because no other thread is allowed to touch that CPU’s slab while the operation runs. The mechanism relies on restartable sequences (rseq) on recent Linux kernels, or an older sigaltstack trick on kernels without rseq support: begin an operation that must complete atomically, and if a context switch interrupts it, the kernel restarts it from a known entry point rather than resuming mid-operation.
The results are excellent throughput numbers, particularly on allocation-heavy benchmarks with many threads. Per-CPU caches eliminate lock contention by partitioning the allocator’s hot state by hardware rather than by thread.
What you trade away is configurability. TCMalloc does not expose an equivalent to jemalloc’s extent hooks. You cannot attach custom backing-memory logic to a TCMalloc allocation domain. The per-CPU slab layout is internal to the allocator. Runtime statistics are available via tcmalloc::MallocExtension, but the interface is substantially shallower than jemalloc’s mallctl namespace. The design optimizes the allocation throughput path and treats extensibility as secondary.
What Page-Local Free Lists Optimize
Microsoft published mimalloc in 2019 with an accompanying paper describing its core insight: partition free lists by page rather than maintaining global free lists per size class. Each thread gets its own heap. Within that heap, allocations are served from logical pages of 64 KB, each page handling one size class. Within a page there are three free lists: a thread-local list for the owning thread’s frees, a thread-free list that remote threads push to asynchronously, and a main list that serves allocation requests. Popping from the thread-local list requires no synchronization.
mimalloc’s fragmentation characteristics are generally strong because objects of the same size are collocated within pages, and pages reclaim cleanly once all their objects are freed. The implementation is compact compared to jemalloc’s, and throughput in Microsoft’s benchmarks compares favorably to both jemalloc and TCMalloc across a range of allocation patterns.
The limitation is structural: the library is not designed as a programmable substrate. There is no equivalent to custom arenas, no extent hooks interface, and no runtime configuration namespace approaching jemalloc’s depth. mimalloc is an excellent allocator in the traditional sense; it is not infrastructure that other systems customize to their memory topology.
What the Arena Model Optimizes
jemalloc’s design predates both TCMalloc’s per-CPU rewrite and mimalloc by a significant margin. Jason Evans wrote it for FreeBSD 7.0 in 2005, targeting SMP scalability at a time when the standard solution was a coarse-grained arena pool. The core insight was: create more arenas than CPUs (4 * ncpus by default), assign threads round-robin, and let each arena manage its own state independently so threads on different arenas never contend. The per-thread tcache then eliminates most arena locking on hot allocation paths.
What this model enables that per-CPU and page-local designs do not: explicit application control over allocation domains. You can create additional arenas via mallctl("arenas.create", ...), install custom extent hooks on each, route specific allocations through them using MALLOCX_ARENA(n), and collect per-arena statistics independently. This is not a minor feature; it is the architectural lever that makes NUMA-aware allocation, custom memory regions, and per-domain observability possible.
The extent hooks interface (introduced in jemalloc 5.0’s rewrite from fixed-size chunks to variable-size extents) exposes nine function pointers per arena: alloc, dalloc, destroy, commit, decommit, purge_lazy, purge_forced, split, and merge. Override any subset; leave the rest as NULL to use jemalloc’s defaults. A custom alloc hook can call mbind() to bind backing pages to a specific NUMA node before returning the extent; a custom merge hook prevents extents from different NUMA nodes from being coalesced into a region that spans the interconnect:
static void *numa_extent_alloc(extent_hooks_t *hooks, void *new_addr,
size_t size, size_t alignment,
bool *zero, bool *commit,
unsigned arena_ind) {
size_t map_size = size + alignment;
void *raw = mmap(new_addr, map_size,
PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (raw == MAP_FAILED) return NULL;
uintptr_t p = (uintptr_t)raw;
uintptr_t aligned = (p + alignment - 1) & ~(alignment - 1);
size_t leading = aligned - p;
size_t trailing = map_size - size - leading;
if (leading > 0) munmap(raw, leading);
if (trailing > 0) munmap((void *)(aligned + size), trailing);
unsigned long nodemask = 1UL << arena_to_numa_node[arena_ind];
if (mbind((void *)aligned, size, MPOL_BIND,
&nodemask, arena_to_numa_node[arena_ind] + 2, 0) != 0) {
munmap((void *)aligned, size);
return NULL;
}
*zero = true; *commit = true;
return (void *)aligned;
}The core slab and tcache machinery runs on top of these hooks unchanged. You get the allocator’s internal sophistication plus controlled OS-level memory placement. Neither TCMalloc nor mimalloc exposes a mechanism for this.
The mallctl Namespace as a Control Plane
The programmability argument extends beyond extent hooks. jemalloc’s mallctl interface is a hierarchical configuration and statistics namespace covering hundreds of keys. At runtime you can read per-arena allocation counts, size-class breakdowns, dirty-page counts, tcache fill and flush rates, decay state, and live allocation totals. You can enable and disable the heap profiler, adjust decay timers, dump profiles to files, and configure background thread behavior without restarting the process.
In Rust, tikv-jemalloc-ctl exposes this interface idiomatically. One detail that trips up first-time users: jemalloc caches statistics internally for performance, so you must advance the epoch before reading to get current values:
use tikv_jemalloc_ctl::{epoch, stats};
epoch::mib().unwrap().advance().unwrap();
let allocated = stats::allocated::mib().unwrap().read().unwrap();
let resident = stats::resident::mib().unwrap().read().unwrap();
println!("overhead: {} MB", (resident - allocated) / 1_000_000);The gap between resident and allocated is the most useful memory health signal in any long-running jemalloc process. When it trends upward, freed pages are accumulating faster than decay timers return them to the OS. When it spikes and recovers rhythmically, the batch interval is mismatched against the decay period. When it is stable, the allocator is in equilibrium with the workload. Neither TCMalloc nor mimalloc surfaces this signal at comparable granularity without instrumentation you build yourself.
What Meta’s Investment Actually Addresses
The four focus areas in Meta’s announcement correspond to what the arena model currently lacks relative to modern hardware.
NUMA-aware arena assignment formalizes what extent hooks make possible: one arena per NUMA node, extents bound to the correct node’s DRAM via mbind, threads assigned to arenas based on CPU affinity at startup. On a two-socket server, cross-node cache misses cost 30 to 60 nanoseconds more than local accesses; at the allocation rates of a busy C++ service, that compounds into measurable latency and reduced effective memory bandwidth. The announced work delivers this as a documented, tested feature rather than something each team hand-rolls with custom hooks.
Transparent huge page alignment ensures extents are 2 MB-aligned so the kernel’s THP machinery can back them with 2 MB physical pages. The TLB has a fixed number of entries; 2 MB pages give each entry 512 times the virtual address coverage of 4 KB pages, which matters for large heaps with scattered access patterns.
Per-arena decay configuration addresses a genuine operational gap. The dirty_decay_ms and muzzy_decay_ms parameters are currently global. A process hosting both latency-sensitive request handling and batch ML inference cannot apply different decay policies to each allocation domain today. The request path benefits from keeping dirty pages warm to avoid page fault latency; the batch workload benefits from returning pages quickly between batches. Per-arena decay parameters resolve this without forcing a process-wide compromise.
Profiling improvements aim to lower the expertise threshold for using jemalloc’s heap profiler in production. The profiler is already powerful: sampling-based, pprof-compatible output, low enough overhead at lg_prof_sample:19 that many teams run it continuously on a subset of production servers. The announced prof_recent ring buffer extends this toward always-on bounded-overhead sampling. The limiting factor today is discoverability. The distinction between stats.allocated and stats.resident, the epoch advance requirement, the existence of per-arena statistics, the prof_recent API, none of this appears in a malloc man page.
Which Allocator for Which Problem
For workloads that need raw allocation throughput without custom memory placement or runtime diagnostics, mimalloc and modern TCMalloc are credible choices. For workloads that require NUMA-aware physical placement, per-allocation-domain statistics, custom backing memory, runtime heap profiling, or per-arena decay control, jemalloc is the available option that covers all of those requirements in production.
Meta’s fleet uses jemalloc because the combination of programmability, observability, and operational hardening accumulated over twenty years has no equivalent in alternatives that optimize narrower targets. The upstream repository is where improvements land; FreeBSD ships it as the system allocator, Redis and RocksDB recommend it, and tikv-jemallocator packages it for the Rust ecosystem. The announced investment closes specific gaps between jemalloc’s design and modern server hardware, and the results flow through all of those downstream consumers once they land.