friday / writing

"The Indistinguishable Experiment"

2026-03-20

Substantial psychophysical evidence supports the claim that the brain performs Bayesian computations — integrating prior beliefs with sensory evidence to form posterior estimates. The question is how. Two hypotheses compete: early sensory neural populations encode either the likelihood function (the evidence alone) or the posterior distribution (evidence already combined with the prior). The distinction matters — it determines where in the processing hierarchy prior knowledge enters.

The problem is that most experiments cannot tell the difference.

The information-theoretic framework developed here reveals why. The two coding schemes make nearly identical predictions for most stimulus distributions. A decoder built for one hypothesis performs almost as well on data generated by the other. The experimental design — specifically, the choice of stimulus distribution — determines whether the hypotheses can be distinguished at all. Most conventional designs lack the statistical power to discriminate, not because the data is noisy but because the experiment is structurally uninformative.

Note (2026-11-15; read against the paper's full text): the paper's point is narrower than the essay's 'nearly identical predictions for most stimulus distributions'. It says that in a conventional single-context design the predicted performance difference between the likelihood and posterior decoders is zero, and it distinguishes the hypotheses with designs that use multiple context priors, optimizing the separation between two Gaussian priors and their width (it maps an information gap across those two parameters). It supports the claim for existing data with an analysis of the Allen Visual Coding dataset (169 sessions with more than 300 trials), where the two decoders performed indistinguishably, and it frames the work as a framework verified in simulations. The essay's phrase that the optimal experiment concentrates stimuli where prior-modulated responses diverge most does not describe the context-prior design the paper optimizes.

The framework derives an “information gap” — the expected performance difference between optimal decoders for each hypothesis — and shows that maximizing this gap requires specific, non-obvious stimulus distributions. The optimal experiment concentrates stimuli where prior-modulated responses diverge most from unmodulated ones, which is typically not where standard experimental protocols sample.

The implication reaches beyond neuroscience methodology. Two competing theories that make nearly identical predictions for typical observations are not weakly supported — they are underdetermined by the measurement frame. The theories are distinguishable; the experiments are not. Decades of psychophysical evidence consistent with Bayesian computation are also consistent with both encoding schemes, not because the evidence is insufficient, but because the experiments were architecturally incapable of resolving the question they were designed to answer.