· 5 min read ·

The Client-Server Tax: What You Pay When PostgreSQL Is Overkill

Source: hackernews

The dbpro.app article making the rounds on Hacker News asks whether you need a database at all. With 284 upvotes and nearly 300 comments, it clearly resonates. The piece stops short of getting specific about the mechanics, so let me try to add some.

The default behavior for many developers, myself included, is to reach for PostgreSQL the moment any persistence is involved. You write a docker-compose.yml, spin up a Postgres container, set up a connection pool, configure environment variables for the DSN, and move on. This takes about fifteen minutes. It feels professional. It is also, for a large class of applications, a significant amount of unnecessary infrastructure.

The Client-Server Tax

When you run PostgreSQL, every query crosses a process boundary. Even on localhost, this costs something. A TCP socket is established (or reused from a pool), the query is serialized, sent, parsed, planned, executed, and the result serialized back. On a modern machine, a simple indexed read on localhost typically costs 0.5 to 2 milliseconds in round-trip latency, which feels negligible in isolation.

At 50 concurrent users each triggering 10 queries per page load, you are making 500 queries per second. Two milliseconds per query means a full CPU second per second spent on network round-trips to localhost. For a typical web app where most queries are simple primary-key lookups, that overhead is pure waste.

Connection pooling helps with connection establishment but does not remove the IPC overhead. PgBouncer reduces connection churn; it does not make the data come back faster.

SQLite has none of this overhead. The database is a file. A query is a function call within the same process. For read-heavy single-node applications, SQLite routinely outperforms PostgreSQL on equivalent hardware precisely because it skips the entire client-server communication layer.

What SQLite Can Actually Handle

SQLite is routinely underestimated. Here are a few concrete details.

In WAL (Write-Ahead Logging) mode, SQLite supports unlimited concurrent readers alongside a single writer, with readers not blocking writers and writers not blocking readers. For most web applications, the single-writer constraint is not a practical bottleneck, because write operations are short and queue rapidly.

WAL mode is a single pragma:

PRAGMA journal_mode=WAL;

SQLite write throughput on a modern NVMe SSD is roughly 50,000 to 200,000 inserts per second, depending on transaction batch size. Wrapping inserts in explicit transactions closes most of the performance gap with PostgreSQL for bulk operations.

The maximum database size is 281 terabytes. SQLite handles databases in the hundreds of gigabytes routinely in production. The belief that SQLite is for small data is a misconception.

SQLite also ships with FTS5 for full-text search, a JSON1 extension for structured JSON queries, RTREE for spatial data, and STRICT tables (added in version 3.37.0, released in late 2021) for strong type enforcement. None of these are toy capabilities.

In Node.js, better-sqlite3 provides a synchronous, high-performance SQLite interface that integrates cleanly with Node’s event loop. In Bun, SQLite support is built directly into the runtime via bun:sqlite:

import { Database } from "bun:sqlite";

const db = new Database("data.db");
db.run("PRAGMA journal_mode=WAL");

const insert = db.prepare("INSERT INTO events (user_id, type) VALUES (?, ?)");
insert.run(userId, eventType);

There is no configuration, no connection pool, and no external process. The database is a file you can copy, inspect with the SQLite CLI, diff with standard tools, and back up with a single cp command.

When Flat Files Are Right

There is a class of data that does not benefit from a database at all. Configuration, static content, and small reference datasets often belong in flat files.

Hugo, Astro, and Jekyll treat the filesystem as the database, and this is the correct choice for static content. A directory of Markdown files with YAML frontmatter is more versionable, more diffable, and more directly editable than any database representation. Changes are a git diff away.

The risk with flat files is concurrent writes. Two processes writing to the same JSON file will corrupt it. The mitigation is straightforward: write atomically by writing to a temporary file, then renaming:

import { writeFileSync, renameSync } from "fs";
import { join } from "path";
import { tmpdir } from "os";

function atomicWriteJSON(targetPath, data) {
  const tmp = join(tmpdir(), `write-${Date.now()}-${Math.random()}`);
  writeFileSync(tmp, JSON.stringify(data, null, 2), "utf8");
  renameSync(tmp, targetPath); // atomic on POSIX systems
}

rename(2) on POSIX systems is atomic. If the process crashes after writeFileSync but before renameSync, the original file is untouched. If it crashes after, the new file is in place. There is no window where the file is partially written. SQLite uses an equivalent mechanism internally for its own durability guarantees.

The Replication Problem Is Solved

The strongest objection to SQLite in production has historically been replication and backup. A single-file database on one machine creates an obvious risk.

Litestream addresses this directly by continuously replicating a SQLite database to S3-compatible object storage with point-in-time recovery. It runs as a sidecar process and adds no application-level complexity. Restore time from a Litestream backup is typically under a minute for most database sizes.

Turso, built on libSQL (a SQLite fork maintained by Turso), goes further by providing hosted distributed SQLite with read replicas per region. The API stays compatible with SQLite, and the mental model is the same file-based database, now with geographic distribution.

Fly.io documented their own production use of SQLite with Litestream and found it reliable enough for their internal infrastructure. The architecture: a primary node running SQLite in WAL mode, continuous replication to object storage, and replicas that restore from object storage on startup. This is a complete, production-grade storage stack with significantly less operational complexity than a managed PostgreSQL cluster.

The Decision

The storage decision deserves more deliberate thought than it typically gets. A rough framework that holds up in practice:

  • Single-process application, single machine: SQLite in WAL mode, for most applications without exception.
  • Static content, versioned alongside code: Flat files. Markdown, TOML, or YAML as appropriate to the content type.
  • Multi-process, single machine: SQLite with careful concurrency design, or PostgreSQL if write volume genuinely warrants it.
  • Multi-machine writes, high concurrency: PostgreSQL, CockroachDB, or another distributed store.
  • Append-only logs and event streams: A flat log file or a purpose-built log store. PostgreSQL is frequently the wrong answer here.

The HN thread on this article includes the familiar counterpoint: “start with Postgres to avoid the migration later.” This is reasonable advice for teams confident they will scale, but it is poor advice for a solo project, an internal tool, or anything where the operational cost of a Postgres server is genuinely felt. The migration from SQLite to PostgreSQL is well-documented and mechanically straightforward. The time spent maintaining unnecessary infrastructure does not come back.

SQLite ships on every iPhone, every Android device, every Mac, every Windows machine, most Linux distributions, and inside Firefox, Chrome, and most Electron applications. It is the most deployed database engine in the world, by a wide margin. Treating it as a toy says more about developer habits than about the technology itself.

Was this interesting?