· 6 min read ·

Node.js fs Has No Interface: The Architectural Root of the VFS Problem

Source: hackernews

Node.js has been building out an extensibility story for years. The Module Customization Hooks API, stable in Node.js 22, lets you intercept every import and require call, resolve modules from alternative sources, and transform source before execution. You can write a custom loader that serves modules from a zip archive, a remote endpoint, or a database. The module system is genuinely hookable.

The fs module is not.

This is the architectural gap that Platformatic’s recent writeup is pointing at when it argues for a Node.js virtual filesystem. The proposal has 235 points and 200 comments on Hacker News because it names something real: the module hook system and the filesystem have been developed as if they were separate concerns. In practice, code loaded through a custom module hook often reads its configuration, templates, and data files via fs.readFile. The hook system stops at the module boundary. Everything that happens inside the module runs against the real filesystem.

What Go Got Right

Go added its io/fs.FS interface in Go 1.16, alongside the embed package. The design is worth examining because it is the correct approach to this problem.

fs.FS is an interface with a single method:

type FS interface {
    Open(name string) (File, error)
}

The Go team threaded this interface through everything in the standard library that reads files: http.FileServer, html/template, text/template, io/fs.ReadFile, io/fs.WalkDir. Any function that reads files can accept an fs.FS. Any type that can open files can implement fs.FS. The embed.FS type, the os.DirFS adapter for real directories, and any custom in-memory filesystem you build all satisfy the same interface and compose with everything that accepts it.

This is why //go:embed works seamlessly with the entire standard library and any third-party code written against fs.FS. You do not patch anything. You pass an embed.FS to the function, and the function reads from it:

//go:embed templates/*.html
var content embed.FS

// http.FileServer does not know or care whether it reads
// from disk or from an embedded archive
http.Handle("/", http.FileServer(http.FS(content)))

Node.js has no equivalent abstraction. The fs module is a concrete implementation backed by libuv, the C library that handles all I/O in Node.js. There is no FS interface above it. Libraries receive path strings and call fs.readFile(path, callback) directly. That path travels from the JavaScript call through the Node.js binding layer into libuv, which makes an OS syscall. There is no seam in that chain where an alternative source can be inserted without patching one of the layers.

This is the architectural root of the problem. The absence of an interface above fs means every VFS attempt has to patch the implementation rather than compose with an abstraction.

The Monkey-Patch Ecosystem

Because no composable seam exists at the language level, the Node.js ecosystem has filled the gap by patching the fs module object. The two most prominent tools for this are memfs and mock-fs.

memfs provides a complete in-memory filesystem implementing the same API surface as fs. For it to affect other modules, you either replace the entry in require.cache or use a module interception layer that swaps out fs before any user code loads. A typical test setup looks like:

import { fs as memFs } from 'memfs';
import { patchFs } from 'memfs/lib/node';

patchFs(memFs);
// Now require('fs').readFile reads from the in-memory filesystem

This works for a significant subset of cases. The problem is that Node.js exposes multiple pathways to the same underlying implementation. require('fs') and require('node:fs') both point to the same module, but patching the cache entry for 'fs' does not automatically patch 'node:fs' unless you handle both explicitly. Any library that imports with the node: prefix escapes the patch. This became more common after Node.js 14.18 made the node: prefix official.

The deeper limitation is that native addons calling into libuv directly bypass the JavaScript layer entirely. A .node module calling uv_fs_open() reaches the real filesystem regardless of what you have done to require('fs'). Monkey-patching is a JavaScript-layer abstraction over a C-layer implementation. The layers do not line up, and anything operating below the JavaScript boundary is invisible to the patch.

What Electron’s ASAR Did Right

Electron solved the same problem in 2014 with ASAR, and the solution targeted the correct layer.

ASAR patches Node.js at the binding layer, below the JavaScript fs module. Electron’s fork of Node.js intercepts file system calls before they reach libuv. When the binding layer receives a path containing .asar, it reads the archive header, finds the file’s offset and size, and serves it from the in-memory archive. This intercept covers the complete fs surface: readFile, readFileSync, createReadStream, stat, statSync, readdir, readdirSync. It also covers pathways that bypass JavaScript entirely, because the intercept is written in C, at the binding layer, not in JavaScript above it.

The cost is the fork. Electron maintains a patched Node.js binary specifically to have this capability. Node.js core has historically been reluctant to accept fs-level interception into the main distribution, partly because the right abstraction boundary is unclear and partly because intercepting every fs call has a non-trivial performance cost even when no virtual paths are registered.

ASAR is the existence proof that binding-level interception is tractable. The question is whether that belongs in a fork or in upstream Node.js.

The Multiple-Pathway Problem

A VFS design that operates at the JavaScript level faces a structural problem: Node.js exposes file system access through more pathways than a simple require('fs') patch can cover.

File reads can happen through:

  • fs.readFile / fs.readFileSync
  • fs.promises.readFile
  • fs.createReadStream
  • fs.open followed by fs.read
  • new Blob([...]) from a file path (Node.js 20+)
  • fetch('file://...'), which has a separate implementation from fs

Module loading happens through:

  • require() in CommonJS
  • import and dynamic import() in ESM
  • vm.runInNewContext() with a require function

The Module Hooks API handles the module loading pathways. It does not touch any of the file read pathways. Code loaded through a custom module hook can call fs.readFile('./config.json') the moment it starts executing, and that call goes straight to libuv with no opportunity for interception.

A binding-level VFS hook would catch all of these pathways. A JavaScript-level CustomFileSystem provider would catch most of them but miss native-addon I/O and the fetch('file://') pathway. For the SEA and edge-runtime use cases Platformatic is targeting, the JavaScript-layer coverage is sufficient for almost all real applications. But it is worth being precise about where the gap remains.

What a Proper Hook Would Need

The Node.js Module Hooks API provides a useful design template. It operates at a well-defined layer, requires no fork, and has been refined incrementally across several releases. A parallel design for fs is technically tractable.

A minimal fs provider API might look like:

import { registerFileSystemProvider } from 'node:fs';

registerFileSystemProvider({
  prefix: '/virtual/',
  readFile(path, options, callback) {
    const content = virtualStore.get(path.slice('/virtual/'.length));
    if (content === undefined) {
      const err = Object.assign(new Error('ENOENT'), { code: 'ENOENT' });
      callback(err);
    } else {
      callback(null, content);
    }
  },
  stat(path, options, callback) {
    // return synthetic stat
  },
  readdir(path, options, callback) {
    // enumerate virtual entries under prefix
  },
});

This would need to cover synchronous variants, stream variants, and promise variants without requiring three separate implementations from the provider, which points toward an internal dispatch layer that normalizes the call before handing it to the provider. That dispatch layer is precisely what Go’s fs.FS interface provides at the language level. Node.js would be building it after the fact, which is harder but not impossible.

The alternative to building it in core is accepting that userland solutions like memfs and ASAR remain the state of the art. Those solutions work, within their limitations, but they require every team to independently discover where the multiple-pathway problem bites them. A proper hook system would make the common case reliable and the edge cases explicit rather than silent failures.

Was this interesting?