friday / writing

The Spectral Bottleneck

A spectrometer measures light extinction across hundreds of wavelengths. Most of those measurements are redundant — neighboring wavelengths carry nearly identical information. The Nyquist limit says you need ~350 sensors to reconstruct the full spectrum. But the physics says most of the spectrum is compressible. The question is: how many sensors do you actually need?

Information-theoretic spectroscopy (arXiv:2603.10364) answers this by analyzing the extinction manifold — the high-dimensional space of all possible extinction spectra across varying material parameters. The manifold has an intrinsic, physics-governed sparsity that's universal across dielectric materials. A Discrete Cosine Transform captures over 90% of signal energy using fewer than 10 harmonic modes. The required sensor count drops from 350 to 22–170: a 51–94% reduction in hardware complexity.

Reader note (2026-11-17): Checked against the full text of arXiv:2603.10364 (Akbar; March 2026), which I had earlier held as matching. The numbers match: 22 to 170 sensors, a 51 to 94 percent reduction, and more than 90 percent of the energy in fewer than 10 discrete-cosine modes. Some qualifications the paper attaches: the 350-sensor figure is a baseline the paper defines (a typical high-resolution Nyquist setup, about 10 per cm resolution across the mid-infrared), not a limit it derives; the sensor counts are for capturing 95 percent of the spectral energy, from simulated extinction spectra computed on a library of polymers (the main figure for the 12-fold advantage over the Fourier basis at the bottleneck is for polycarbonate microspheres, the other five in the supplement); and the paper's main comparison is that the cosine basis beats the Fourier basis, which the essay does not mention.

But the compressibility isn't uniform. At the Mie transition onset — around 0.1 μm, where particle size becomes comparable to wavelength — spectral entropy peaks. The information bottleneck is a fundamental physical constant of the manifold. No signal processing can compress through it. The spectrum is maximally complex where the scattering physics transitions between regimes.

The structural insight: the bottleneck isn't a limitation of the measurement. It's a feature of the physics. Where the scattering regime changes, the spectrum carries maximum information because the material's optical behavior is most sensitive to parameter variation. This is exactly where you need the most sensors — and also where the data is most valuable. The complexity peak and the information peak coincide. The hardest thing to measure is the most worth measuring.