· 6 min read ·

SQLite's Devtools Problem Starts at Parse Time

Source: lobsters

SQLite is the most widely deployed database in the world. It runs inside every Android device, every iOS app, every Chromium-based browser, and most Electron applications. It powers local-first software, embedded systems, edge deployments, and server-side workloads from small prototypes to production services. By sheer number of installed instances, nothing else is close.

The tooling situation has never matched that ubiquity. Open a .sql file targeting SQLite in most editors and you get syntax highlighting, autocomplete, and error checking calibrated for PostgreSQL or generic ANSI SQL. The errors are wrong, the completions miss the interesting features, and the dialect-specific capabilities that make SQLite worth using are invisible to the tools supposed to help you use them.

Syntaqlite addresses this by building from SQLite’s grammar rather than from a lowest-common-denominator SQL parser with SQLite bolted on afterward. The distinction matters more than it might seem.

The anatomy of the mismatch

Most SQL databases extend the standard. SQLite diverges from it in a different way: it implements a permissive type system built around type affinity rather than strict types. A column declared INTEGER will store text if the value you insert can’t be coerced to an integer. This isn’t a deficiency; it’s documented behavior that enables schema flexibility and makes SQLite suitable for semi-structured data. But any type checker that assumes strict SQL semantics will produce constant false positives against real SQLite code.

Type affinity is just the start. SQLite has accumulated features with no analog in other SQL databases:

PRAGMA statements form a first-class configuration and introspection interface. PRAGMA journal_mode = WAL enables write-ahead logging. PRAGMA table_info(t) returns column metadata. PRAGMA foreign_keys = ON turns on foreign key enforcement, which is off by default. These have their own syntax, and no generic SQL parser models them at all.

Virtual tables are one of SQLite’s most powerful capabilities. The FTS5 module provides full-text search with a custom query language embedded in SQL string arguments. The rtree module handles spatial indexing. User-defined virtual tables expose arbitrary data sources as queryable tables. The syntax for creating and querying these, including the MATCH operator and auxiliary functions like snippet() and highlight() from FTS5, is specific to SQLite and requires dedicated grammar coverage.

Table modifiers change the storage semantics and constraint model. WITHOUT ROWID tables, added in SQLite 3.8.2, replace the default rowid B-tree with a primary-key-based structure and carry their own restrictions. STRICT tables, added in 3.37.0, flip the type enforcement model and require column types to be one of a fixed set: INTEGER, INT, REAL, TEXT, BLOB, or ANY. These modifiers interact with each other and with other SQLite features in ways no generic SQL tool models.

Recent versions have added features that are now common in SQLite codebases but absent from many tools’ understanding: the RETURNING clause (3.35.0), the upsert syntax with ON CONFLICT DO UPDATE (3.24.0), generated columns (3.31.0), and the -> and ->> JSON field operators (3.38.0). A tool trained on an older understanding of SQLite will flag valid modern code as invalid.

What existing tools actually cover

sqls, the most widely used SQL language server for editor integration, supports SQLite connections and provides schema-aware completion for table and column names. This is useful as far as it goes. But the underlying parser is a generic SQL implementation, meaning PRAGMA syntax is unrecognized, FTS5 queries produce errors, and STRICT table definitions don’t parse cleanly. The SQLite support is connection-level, not grammar-level.

sql-language-server covers similar ground. Schema introspection works when you have a live database connection. Dialect-specific syntax does not.

sqlfluff goes further than either by maintaining an explicit SQLite dialect with per-dialect linting rules. Its approach gives it better coverage of SQLite syntax than tools that treat it as generic SQL. The limitation is that sqlfluff is a linter rather than a language server, and its rule sets are always playing catch-up with SQLite releases. As SQLite has shipped significant features in every major version since 3.24.0, this gap compounds.

The tree-sitter-sql grammar provides syntax highlighting that works reasonably well for the SQL common subset. It doesn’t cover PRAGMA statements, virtual table syntax, or the FTS5 query language.

The pattern across all of these is the same: SQLite support is a thin layer over tooling designed for something else.

Why the grammar is the foundation

Building devtools that work well for SQLite requires two things: an accurate grammar that covers what SQLite actually accepts, and a schema model that reflects SQLite’s specific semantics.

The grammar problem is well-defined but extensive. SQLite’s complete grammar is documented in its railroad diagram specification, covering core SQL alongside all extensions. A tree-sitter grammar with full coverage would need dedicated rules for PRAGMA syntax, the virtual table CREATE syntax with module arguments, FTS5 query language embedded in MATCH expressions, table option modifiers, and the various special forms that SQLite supports but other databases don’t. None of these pieces are complicated in isolation, but getting them all right requires going through the specification carefully rather than adapting an existing SQL grammar.

The schema model problem is more interesting because of how SQLite is typically used. Unlike PostgreSQL or MySQL, SQLite is often embedded in applications rather than running as a standalone server. The schema may live in migration files, in CREATE TABLE statements scattered across test fixtures, or in an in-memory database that only exists at runtime. Tooling that requires a live database connection to provide meaningful completions and error checking covers only one of those cases.

A language server capable of deriving schema information from static analysis of the SQL files themselves would cover substantially more of real SQLite usage. This requires parsing DDL statements accurately, understanding generated columns, modeling virtual tables, and building a schema representation that can be queried for completions without a connection. The grammar has to come first because you can’t do that analysis without an accurate parse tree.

The Perfetto context

Lalit Maganti, the author of syntaqlite, works on Perfetto, Android’s system tracing infrastructure. Perfetto’s trace processor exposes trace data through a custom SQL dialect built on top of SQLite, extended with macro support, module imports, and materialized query views. This is a production SQLite-based environment used by performance engineers at Google and across the Android ecosystem for analyzing system traces.

Building syntaqlite from this context means the tool was designed for a case where generic SQL tooling is completely inadequate: a SQLite extension layer with its own constructs, where both the base SQLite semantics and the Perfetto-specific extensions need to be modeled correctly. Getting this right requires starting from accurate SQLite grammar coverage before adding anything on top.

The framing of “high-fidelity” in syntaqlite’s description reflects this origin. High fidelity in this context means the tooling reflects what the language actually is, not what a simpler model would assume it to be.

The broader ecosystem shift

SQLite is increasingly present in contexts where tooling investment is justified. Litestream enables continuous replication of SQLite databases, making small production deployments viable. libSQL, Turso’s SQLite fork, extends it with networked access and replica support. Cloudflare D1 runs SQLite at the edge with a familiar interface. LiteFS provides distributed SQLite for containerized environments.

Each of these deployments involves engineers writing SQL against SQLite in contexts where the schema is well-defined and editor tooling that understood the dialect would provide real value. The case for investing in SQLite devtools has grown alongside SQLite’s presence in server-side and edge contexts, not just embedded and mobile.

The challenge that syntaqlite addresses is a prerequisite for any of this tooling to work well. An accurate grammar and a schema model that reflects SQLite’s actual semantics are the foundation that everything from formatters to refactoring tools to CI linters can build on. Building it properly, from the specification rather than from an approximation, changes what the rest of the SQLite tooling ecosystem can do.

Was this interesting?