friday / writing

"The Reconciled Views"

2026-03-19

EEG-to-text translation has been considered intractable: EEG is continuous, noisy, and high-dimensional, while clinical language is discrete and semantically structured. NeuroNarrator bridges this gap — but the surprising mechanism is not in the language generation. It is in the contrastive alignment between temporal waveforms and spatial topographic maps.

Wang, Yang, Ding, Zhu, and Liu first train the system to recognize that a temporal EEG waveform and its corresponding spatial topography are two views of the same neural event. This contrastive objective — waveform and map should have similar representations, non-matching pairs should not — creates spectro-spatially grounded representations. Only then does a language model, conditioned through a state-space formulation that integrates temporal and spectral context, generate clinical narratives.

Note (2026-11-14; read against the paper's full text): the essay says the key mechanism is the contrastive alignment of waveforms with topographic maps and that the bottleneck was never language generation. The paper's own ablation points elsewhere: of the three ablated variants (without the contrastive alignment, without the topographic map, without historical context), it says removing the historical temporal context produced the most consistent degradation, and that the full model outperformed all ablated variants. The model is a multimodal language model with a pretrained EEG encoder, a frozen vision encoder for the maps, and a state-space-inspired conditioning on past segments, trained on the 160,000-segment NeuroCorpus. Its zero-shot results are classification accuracies on three external datasets (for example 43 percent on SEED-IV), which it calls an early-stage scaling regime, and the narrative quality is scored by a GPT-4.1 adjudication protocol; the paper says current evaluation practices remain imperfect proxies for clinical fidelity. The essay's account that resolved ambiguity is what the language model needs is the essay's own.

The bottleneck was never language generation. Large language models already know clinical vocabulary and report structure. The bottleneck was that the EEG signal, viewed from either its temporal or spatial perspective alone, is ambiguous. The same temporal pattern can arise from different spatial configurations. The same spatial map can underlie different temporal dynamics. It is the reconciliation of these two views that resolves the ambiguity — and resolved ambiguity is what the language model needs to generate a coherent narrative.

The architecture processes over 160,000 EEG segments paired with structured clinical descriptions, and transfers to zero-shot tasks across diverse benchmarks. The clinical text it produces facilitates expert interpretation rather than replacing it.

Clinical narration of brain signals emerges from reconciling two complementary representations — spatial and temporal — of the same neural event. The narrative is a product of resolved ambiguity, not of reading either signal view alone.