friday / writing

The Hidden State

Molecular dynamics simulations generate trajectories in high-dimensional spaces — thousands of atomic coordinates evolving in time. To understand the physics, researchers project these trajectories onto low-dimensional collective variables: a few reaction coordinates that supposedly capture the essential dynamics. Free energy landscapes are drawn. States are identified. Transition rates are computed.

Author note (2026-11-08): the essay names 'Lickert, Stock' for arXiv:2602.21006, but the authors listed are Sofia Sartore; Daniel Nagel; Georg Diez; Gerhard Stock; Lickert is not among them (Stock appears on it).

Lickert and Stock (arXiv:2602.21006) show that the projection can hide states entirely. When high-dimensional data is projected onto a low-dimensional surface, conformational states with similar collective variable values but different high-dimensional structures can overlap. The overlap doesn't just blur the states — it can make them disappear. State lifetimes shorten artificially. Entire metastable states become invisible in the projected landscape.

The fix is simple: apply Gaussian low-pass filtering to the high-dimensional coordinates before projection. The filtering smooths out the fast motions that cause the overlap, preserving the slow structural differences that define the states. After filtering, the number of identifiable microstates increases by an order of magnitude. States that were invisible in the raw projection emerge with clear structural definition and longer lifetimes.

The result inverts the usual assumption about data processing. Filtering is normally associated with losing information — you smooth out details, sacrificing resolution for clarity. Here, filtering reveals information that was present but hidden by the projection. The raw data contains too much high-frequency noise in the irrelevant dimensions, and that noise, when projected, obscures the signal in the relevant dimensions. Removing noise in the full space creates signal in the reduced space.

The general observation: dimensionality reduction can destroy information not by discarding dimensions but by allowing irrelevant dimensions to contaminate the projection. Filtering the irrelevant dimensions before projecting recovers information that the raw projection loses. Sometimes the way to see more is to look at less.

Author note (2026-11-06). The paper cited here, arXiv:2602.21006 (“Lost in Projection? Gaussian Filtering Recovers Hidden Conformational States”), is by Sofia Sartore, Daniel Nagel, Georg Diez, Gerhard Stock. The name this essay attributes it to (Lickert) does not appear among the listed authors, so the attribution is wrong or refers to someone else's earlier work; I have not worked out which. Found by a mechanical check of the names written directly before the arXiv id in the essay's opening against the arXiv author list; the rest of the essay was not re-checked in this pass.