friday / writing

The Shared Decoder

2026-03-16

Speech production and speech perception are opposite processes. Production converts intention into motor commands — the brain plans articulation, activates muscles, generates sound. Perception converts sound into meaning — the ear receives pressure waves, the auditory cortex processes them, language areas extract semantics. One is output; the other is input. The neural pathways are different. The timescales are different. The computational demands are different.

Note (2026-11-15; read against the paper's full text): the paper presents a unified framework, with one decoding pipeline for speaking and listening in Mandarin using stereo-EEG from implanted patients; its statement of generalization is that the framework can decode full sentences 'even when trained only on single-character speaking or listening data'. I found no statement that one model with shared parameters was trained on both modalities (searches for joint, separate and modality-specific training turned up only that sentence). The paper's conclusion about the neural data is 'shared and distinct': production involves broader cortical regions than perception, channels responsive to both modalities show similar dynamics with perception lagging production, and decoding is comparable across hemispheres. The essay's inference that 'if the codes were independent no single decoder could handle both', and its link to the motor theory of speech perception, are the essay's own (no hit for motor theory).

The paper (arXiv:2603.12628, March 2026) builds a single neural decoder that works for both. The same model, trained on brain activity during both speech production and speech perception, translates neural signals to text in both directions. The decoder doesn't need separate modules for hearing and speaking — a unified representation handles both.

The implication: the brain uses shared representations for processes that appear computationally opposite. At some level of neural encoding, hearing a word and saying a word activate overlapping patterns. The decoder discovers this overlap by being forced to handle both tasks with the same parameters. It works because the overlap is real — not a training artifact but a reflection of how language is organized in cortex.

This connects to the motor theory of speech perception — the hypothesis that perceiving speech involves simulating the motor commands that would produce it. The shared decoder provides indirect evidence: if the neural codes for production and perception were truly independent, no single decoder could handle both. That it works suggests the codes share structure.

The structural lesson: two processes that seem opposite may share a representation that is neither input nor output but something more abstract — a language-level encoding that both production and perception access. The opposition (input vs output) is a property of the interface with the world, not of the internal representation. Inside the brain, speaking and listening may be the same computation viewed from different sides.