· 6 min read ·

Getting Useful Flow Data from a NixOS Router Without the Overhead

Source: lobsters

If you run a NixOS router or a Linux server sitting at a network boundary, you probably want flow data. Not necessarily a full NetFlow pipeline feeding into Elasticsearch, but enough to answer reasonable questions: which internal host is saturating the uplink, what destinations are getting the most traffic, whether something is phoning home that shouldn’t be. The problem is that most of the tools built for this job carry noticeable overhead, and on modest router hardware, that overhead matters.

A recent NixOS Discourse post introduces a very lightweight flow data collector aimed at exactly this scenario, with the note that it should work on non-NixOS Linux systems as well. The “very lightweight” framing is the interesting part, because it points directly at a design decision about where flow data actually comes from in the kernel.

What Flow Data Is and Why Collectors Get Expensive

A network flow is a group of packets sharing the same 5-tuple: source IP, destination IP, source port, destination port, and transport protocol. Flow records attach metadata to that tuple: byte count, packet count, start and end timestamps, TCP flags. The IPFIX specification (RFC 7011), which is the IETF standardization of Cisco’s NetFlow v9, defines the template format that most modern collectors speak.

The traditional way to generate flow records from a Linux box is with a tool like softflowd or fprobe. Both use libpcap to capture packets on an interface and aggregate them into flow records, exporting via UDP to a collector. On a lightly loaded network this is fine. On a router pushing 200+ Mbps with many small flows, libpcap forces every packet through a copy into userspace. That costs CPU cycles proportional to packet rate, not bandwidth, which is the worst case for traffic patterns with lots of small TCP packets.

The resource profile is measurable. A softflowd instance on a busy home router can consume 5 to 15 percent of a single core continuously. On a device running at 1 to 1.5 GHz with everything else the router needs to do, that is a real budget.

The Kernel Already Has the Data

Linux’s connection tracking subsystem, nf_conntrack, maintains state for every tracked connection passing through netfilter. Each entry records the 5-tuple, connection state, and byte and packet counters that the kernel updates in the fast path. This data lives in kernel memory and is available via /proc/net/nf_conntrack or, more efficiently, through a Netlink socket using the NETLINK_NETFILTER family.

A conntrack entry looks like this:

ipv4     2 tcp      6 86394 ESTABLISHED 
  src=192.168.1.5  dst=93.184.216.34  sport=54312  dport=443 packets=143 bytes=18432 
  src=93.184.216.34 dst=192.168.1.5   sport=443    dport=54312 packets=89  bytes=102400
  [ASSURED] mark=0 use=1

Every active connection is in that table. The kernel updates the counters on every packet without any userspace involvement. Reading the table periodically and computing deltas gives you flow-level visibility with essentially no overhead beyond the polling itself. There is no packet copying, no ring buffer, no libpcap filter evaluation.

For event-driven collection rather than polling, conntrack -E (or the equivalent Netlink subscription) delivers events as connections are created, updated, and destroyed. This gets you close to real-time flow data with CPU usage that scales with connection churn rather than packet rate.

The ipt-netflow Alternative

For setups that want proper NetFlow/IPFIX export rather than a custom format, ipt-netflow is a kernel module that hooks directly into netfilter and exports flow records from kernel space. You pair it with a netfilter rule:

modprobe ipt_NETFLOW destination=192.168.1.10:2055 protocol=9
iptables -A FORWARD -j NETFLOW
iptables -A INPUT -j NETFLOW
iptables -A OUTPUT -j NETFLOW

The overhead is substantially lower than libpcap-based tools because the export path is in the kernel. CPU cost is typically under two percent even on busy routers, and memory footprint stays below five megabytes. The tradeoff is that it requires building a kernel module, which on NixOS involves boot.extraModulePackages.

Where NixOS Fits

NixOS is a particularly good platform for this kind of infrastructure because the configuration is atomic. A NixOS module for a flow collector can declare the systemd service, kernel module loading, and any required netfilter rules in one place. If the collector needs ipt-netflow, the module expresses that as a kernel module dependency and the system handles it at activation. Nothing is left to manual modprobe invocations or undeclared iptables state.

A module wrapping conntrack-based collection might look something like this:

{ config, lib, pkgs, ... }:
{
  services.flowCollector = {
    enable = true;
    interface = "eth0";
    outputPath = "/var/lib/flow-data";
    intervalSeconds = 30;
  };
}

Under the hood that becomes a systemd service running a small binary or script, with after = [ "network.target" ] and appropriate restart semantics. The reproducibility means you can deploy identical monitoring across multiple routers in a flake without drift.

NixOS also makes it straightforward to pair the collector with a local metrics exporter. A common pattern for home routers is exporting flow summaries to Prometheus via a small exporter, then pulling those metrics into Grafana. Because Nix manages service dependencies declaratively, you can express “the flow collector feeds this exporter which is scraped by this Prometheus instance” as a unit of configuration that either fully works or fails to activate.

nftables Counters for Aggregate Views

For cases where per-flow granularity is not needed and you only want aggregate bandwidth by interface or subnet, nftables native counters are even lighter than conntrack polling:

networking.nftables.ruleset = ''
  table inet accounting {
    counter wan_bytes_in  { }
    counter wan_bytes_out { }
    counter lan_to_wan    { }

    chain forward {
      type filter hook forward priority 0; policy accept;
      iifname "wan0" counter name "wan_bytes_in"
      oifname "wan0" counter name "wan_bytes_out"
      iifname "lan0" oifname "wan0" counter name "lan_to_wan"
    }
  }
'';

You can read these counters with nft list counter inet accounting wan_bytes_in. A small script polling these and writing to a time-series store gives you bandwidth graphs with negligible overhead. The limitation is that you lose the per-flow breakdown, so you cannot see which host on the LAN is responsible for the outbound traffic.

The Broader NixOS Router Ecosystem

The NixOS community has been building out home router configurations more aggressively over the past couple of years. Projects like nixos-router on GitHub demonstrate full gateway setups using NixOS, combining nftables for firewalling, Kea for DHCP, Unbound for DNS, and systemd-networkd for interface management. Flow collection slots naturally into that stack as another systemd service.

For the collector side of the pipeline, where flow records land and get stored, goflow2 handles NetFlow v5/v9 and IPFIX decoding and can forward to Kafka, a file, or directly to a Prometheus exporter. Akvorado goes further, combining a flow collector with ClickHouse-backed storage and a web interface, though it is sized more for small business than home use. nfdump and the associated nfsen remain solid choices for simpler setups.

What Lightweight Actually Buys You

The practical difference between libpcap-based collection and conntrack-based collection on a 1 GHz ARM router running at 50 to 100 Mbps is the difference between noticing the collector running and not noticing it at all. On a device with a single core handling NAT, DNS, DHCP, and Wi-Fi management, that headroom matters. The conntrack approach also degrades gracefully: under high load, polling intervals stretch, but the router does not drop packets because a userspace process is struggling to keep up with a libpcap ring buffer.

The NixOS Discourse project extends the same philosophy to the configuration layer. Getting flow data from your router should be a three-line module declaration, not a documentation expedition through softflowd flags and systemd unit syntax. The fact that it works on other distributions too suggests the core logic is portable, with the NixOS module being the clean integration layer rather than the whole story.

For anyone running a NixOS router who has wanted network visibility without the overhead, this is worth a look.

Was this interesting?