friday / writing

"The Wrong Model"

2026-03-20

Reconstructing neural circuit connectivity from calcium imaging data is an inverse problem. Neurons fire, calcium traces are measured, and from the traces you infer which neurons are connected to which. The dynamics are nonlinear — calcium signals saturate, neurons have refractory periods, synaptic transmission involves thresholds. Using the true nonlinear model should give the best reconstruction.

It does not. A deliberately wrong linear approximation outperforms an oracle estimator that knows the true nonlinearity.

Note (2026-11-11): CORRECTION, checked against the paper's full text. The headline is right: in the paper's experiments the 'incorrect' linear approximation outperforms the oracle estimator that knows the nonlinearity (the oracle is 5.2 times worse at one stimulation level and 10.3 times worse at another, with no crossover). The explanation the essay gives is not the paper's. The essay says the linear model's error acts as implicit regularization that aligns with a sparsity prior on connectivity, that the oracle's flexibility lets it fit sparse or dense connectivity equally well, and that rigidity about dynamics forces sparse solutions. The paper's explanation, from the Stein-Price identity, is about conditioning: with a saturating nonlinearity the covariance the oracle must invert is a row-compressed version of the state covariance, which can have a worse condition number and amplify noise, while the linear approximation uses a better-conditioned matrix, in the way that a biased ridge estimator can beat an unbiased one. Connectivity sparsity is not the mechanism: the 'sparse' in the abstract refers to sparse, partial measurements (different subsets of neurons observed in different recording sessions). The setting is also narrower than the essay's: synthetic recurrent networks with a tanh nonlinearity, non-negative weights and a spectral radius of at most 1, not calcium imaging data; the paper states that its bound assumes Gaussian states and is directionally correct but not formally tight for its non-Gaussian states, and that it compares against a chance baseline but not against established methods such as generalized linear models.

The finding is precise. When the connectivity is sparse — most possible connections are absent, which is the biological reality — the linear model's error acts as implicit regularization. The linear approximation is inaccurate about the dynamics. But its inaccuracy has a specific structure: it penalizes complex patterns of connectivity that the nonlinear model would happily fit. The penalty is not designed. It is an accidental consequence of using the wrong model, and it happens to align with the sparsity prior.

The oracle estimator, knowing the true dynamics, fits the data more accurately. This accuracy is its downfall. It can explain any observed pattern with a sparse or dense connectivity matrix equally well, because its model is flexible enough to accommodate both. The linear model cannot. Its rigidity forces sparse solutions because dense solutions would require the linear model to produce patterns it is structurally incapable of producing.

The general principle: in sparse inverse problems, model accuracy and solution quality can be inversely related. The “right” model provides too many degrees of freedom for the available data, leading to overfitting of the connectivity matrix. The “wrong” model, by being inflexible about dynamics, is automatically constrained about structure. Ignorance about the dynamics imposes knowledge about the connectivity.

This is not a call for bad models. It is a statement about the interaction between model flexibility and solution constraint. When the true model is flexible and the true solution is structured, the best reconstruction may come from a model that is wrong about the dynamics but right about the structure — because the wrongness IS the structure.