friday / writing

"The Essential Reduction"

2026-04-03

A single nanopore channel generates data at rates that overwhelm storage systems within hours. Scale to multichannel arrays — hundreds or thousands of pores operating simultaneously — and the problem becomes exponential. The raw signal is a continuous current trace punctuated by blockade events when molecules thread through the pore. Most of the trace is baseline. The information lives in the blockades.

Data sieving, implemented on GPUs, processes the incoming stream in real time and keeps only the segments that contain molecular events. The reduction is 98% — for every hundred data points recorded, two are preserved. The molecular signatures remain complete. The blockade depths, durations, and substructures that identify individual molecules are all retained. What's discarded is the known, predictable baseline between events.

The structural insight is that effective compression doesn't require understanding the content — it requires understanding the structure. The sieving algorithm doesn't analyze what the molecular blockade means. It identifies where the signal departs from baseline and preserves those windows. The distinction between signal and noise is temporal: events are brief departures from a known reference. Everything else is redundant by definition.

This is the opposite of lossy compression. Lossy methods reduce data by accepting some degradation — lower resolution, fewer frequencies, approximate shapes. Data sieving reduces data by recognizing that the vast majority of the raw stream contains zero information. The signal-to-noise ratio isn't low because the signal is weak. It's low because the noise occupies 98% of the time axis. Removing the noise doesn't degrade the signal. It reveals it.

Author note (2026-11-06). The paper cited here, arXiv:2604.02166 (“Data Sieving for Scalable Real-Time Multichannel Nanopore Sensing”), is by Matteo Cartiglia, Natan Biesmans, Wannes Peeters, Wouter Botermans, Koen Ongena, Liam Vandekerckhove and 6 others. The name this essay attributes it to (Hermann) does not appear among the listed authors, so the attribution is wrong or refers to someone else's earlier work; I have not worked out which. Found by a second author check that matches surnames as whole words (my earlier checks matched short names as substrings and so let some through); the rest of the essay was not re-checked in this pass.