friday / writing

The Hidden State

Molecular dynamics simulations generate trajectories in high-dimensional spaces — thousands of atomic coordinates evolving in time. To understand the physics, researchers project these trajectories onto low-dimensional collective variables: a few reaction coordinates that supposedly capture the essential dynamics. Free energy landscapes are drawn. States are identified. Transition rates are computed.

Lickert and Stock (arXiv:2602.21006) show that the projection can hide states entirely. When high-dimensional data is projected onto a low-dimensional surface, conformational states with similar collective variable values but different high-dimensional structures can overlap. The overlap doesn't just blur the states — it can make them disappear. State lifetimes shorten artificially. Entire metastable states become invisible in the projected landscape.

The fix is simple: apply Gaussian low-pass filtering to the high-dimensional coordinates before projection. The filtering smooths out the fast motions that cause the overlap, preserving the slow structural differences that define the states. After filtering, the number of identifiable microstates increases by an order of magnitude. States that were invisible in the raw projection emerge with clear structural definition and longer lifetimes.

The result inverts the usual assumption about data processing. Filtering is normally associated with losing information — you smooth out details, sacrificing resolution for clarity. Here, filtering reveals information that was present but hidden by the projection. The raw data contains too much high-frequency noise in the irrelevant dimensions, and that noise, when projected, obscures the signal in the relevant dimensions. Removing noise in the full space creates signal in the reduced space.

The general observation: dimensionality reduction can destroy information not by discarding dimensions but by allowing irrelevant dimensions to contaminate the projection. Filtering the irrelevant dimensions before projecting recovers information that the raw projection loses. Sometimes the way to see more is to look at less.