· 5 min read ·

The Disciplines a Million-Line Monolith Forces You to Learn

Source: lobsters

The industry spent most of the 2010s treating “monolith” as an insult. Microservices were the cure, and anyone still running a single deployable artifact at scale was either behind or naive. That narrative is now visibly fraying, and pieces like this one from semicolon&sons are part of why: 113 lessons distilled from scaling a monolith to 1 million lines of code, written by someone who lived the journey from tech lead to CTO.

Reading through the list, what strikes me isn’t any individual lesson but the shape of the whole. The lessons cluster around a handful of disciplines, and none of them are unique to monoliths. They’re the disciplines that make software workable at any scale, but that a monolith makes impossible to avoid.

The Boundary Problem

The central challenge of a large codebase is managing dependencies between parts. In a microservices architecture, you solve this with a network boundary: services can’t directly call each other’s internals because there are no shared internals. The cost is operational complexity, distributed system failure modes, and the latency that comes from network hops. In a monolith, you have none of those costs, but you also have no automatic enforcement. Nothing stops any file from importing anything else.

At small scale, this is fine. At 1M LOC with dozens of developers, it becomes the primary source of maintenance pain. The lesson that emerges repeatedly in large monoliths is that you have to create the discipline to enforce boundaries that the language and runtime won’t enforce for you.

Shopify solved this concretely with packwerk, a static analysis tool for Ruby/Rails that enforces package-level visibility rules. You declare which constants a package exposes, and packwerk fails CI if anything outside that package reaches into private internals. Their monolith sits somewhere around 3 million lines of Ruby, and the approach lets hundreds of engineers work on it without constant collision. The architectural unit isn’t the service; it’s the package. The enforcement mechanism is a linter running in CI rather than a network boundary.

This is domain-driven design in its most pragmatic form: not rewriting Eric Evans into architecture documents, but encoding bounded contexts into directory structure and import rules, then making violations fail the build. The package.yml files that packwerk reads declare ownership and privacy boundaries in under a dozen lines:

# components/orders/package.yml
enforce_privacy: true
enforce_dependencies: true
dependencies:
  - components/catalog
  - components/customers

Violate that dependency graph and your CI run fails. The discipline is in the tooling, not the team’s willpower.

What Actually Grows

One of the persistent surprises when looking at large codebases is what gets unwieldy. It’s not usually the application logic. Business logic can grow indefinitely if it’s well-organized. What grows problematically tends to cluster in three places: database schema, test suites, and the abstraction layers that accumulate over time.

Database schema is the hardest of these. At 1M LOC, you likely have hundreds of tables with relationships that evolved organically across years. Migrations become a source of anxiety because the schema is shared state across all of the code. There’s no package boundary at the database level; a table modification affects every query touching that table, and in a large codebase, finding every such query requires either naming discipline or good tooling.

Some teams respond to this with a “database per bounded context” approach, where different logical areas of the system own distinct schemas even within a monolith. You lose simple cross-context JOINs, but you gain the ability to evolve each schema without surveying the entire codebase for side effects. It’s a trade, not a cure.

Test suites are the other place where scale becomes painful in ways that aren’t obvious early. A test suite that runs in three minutes at 10,000 LOC might run in 45 minutes at 500,000 LOC. Full-suite feedback cycles of that length change how developers work in concrete ways: they stop running the full suite locally, they rely on CI for coverage they previously had locally, and slow feedback compounds review and integration latency. The response isn’t just “run tests in parallel”; it requires test architecture discipline. Tests that hit the database for every case, that spin up the full application stack for unit-level concerns, that share global state between cases, all accumulate a cost that’s acceptable early and crippling later.

The Leadership Gap

The article covers the technical side but also the organizational side, which is where most tech lead-to-CTO transitions become difficult. The skills that make someone effective as a tech lead, deep technical judgment, good code review, fast debugging, are not the skills that make someone effective as a CTO. The CTO role is primarily about organizational design, communication, and creating the conditions where good technical decisions get made by other people.

What’s interesting about scaling a monolith specifically is that it forces this transition earlier than microservices would. With microservices, you can delegate by service ownership: this team owns this service, they make the decisions, the boundary is the network. With a monolith, the codebase is shared, decisions about conventions and architecture affect everyone, and coordination is unavoidable. You have to build the organizational scaffolding to make that coordination work.

This means architecture decision records, explicit conventions documented and enforced in CI, code ownership at the package or directory level, regular cross-team calibration on patterns. None of this is glamorous work, but at 1M LOC with a growing team, the absence of it is what creates the spaghetti monolith that gives the architecture its bad reputation.

The Stack Overflow Effect

Stack Overflow has run a notably small number of servers for its traffic for years. Their .NET monolith handles billions of page views a month from a handful of physical machines, and Nick Craver has written in detail about why: the monolith allows aggressive caching, in-memory sharing, and optimizations that aren’t available when data has to cross service boundaries. The performance characteristics of a well-tuned monolith differ from those of a well-tuned distributed system, and the monolith often wins on raw throughput within a single host.

That context helps frame the architectural decision as a trade-off rather than a concession. A monolith at 1M LOC requires specific disciplines to remain workable, but it also preserves performance and operational simplicity that the microservices model surrenders. The lessons in the semicolon&sons article are largely about paying the cost of those disciplines deliberately rather than discovering the price tag late.

Reading 113 Lessons

Reading a list that long in sequence, you look for the through-line. The value of the format is its honesty about how granular this work gets. It’s not a think-piece about architecture philosophy; it’s someone saying “here is the specific thing I learned about database migrations at scale” and “here is the specific thing I learned about communicating technical risk to non-technical stakeholders.” The breadth of 113 separate observations is its own argument that there’s no single trick that makes a large codebase manageable.

Most of the lessons, if you read them closely, are about discipline: the discipline to enforce boundaries, to write tests that run fast, to document decisions, to communicate clearly as the organizational distance between you and the code increases. None of that is monolith-specific; it’s what good software engineering looks like when you can no longer hide the cost.

Was this interesting?