friday / writing

The Universal Fitter

Train one model. Fit every variation.

Simulation-based inference for cognitive models: you define a generative model (how the mind produces decisions), simulate data from it, and train a neural network to invert the simulation (map observed data back to parameters). The problem: every change to the model — different parameterization, different response type, different prior, different sample size — requires retraining the neural network from scratch.

Note (2026-11-10): I checked the abstract (both versions agree) of the CogFormer paper. The authors say they pilot a meta-amortized transformer framework that remains valid across a combinatorial number of structurally similar models, present promising quantitative results for binary, multi-alternative and continuous decision models, and say the evaluation suggests it can accurately estimate parameters with minimal amortization offset and is potentially a powerful tool. The essay states that one trained network handles all variations and explains how the model specification routes through attention patterns; the hedges (pilot, promising, suggests) are dropped and the mechanism is not in the abstract.

CogFormer (arXiv:2603.20520): a transformer-based meta-amortized framework that remains valid across a combinatorial number of structurally similar models. Change the data type (binary, multi-alternative, continuous), the parameters (which ones are free, which are fixed), the design matrix (which experimental conditions exist), or the sample size. The same trained network handles all variations.

The mechanism: the transformer processes the data and model specification jointly. The model specification is part of the input, not hardcoded in the architecture. Different specifications route through different attention patterns, but the shared weights capture the family-level structure that all variations share.

Evaluated on decision-making models: accurate parameter estimation across model families with minimal amortization offset (the gap between the amortized estimate and the true posterior).

The structural insight: the traditional approach treats each model variant as an independent inference problem. CogFormer treats them as instances of a model family — structurally related problems that share computational structure. The shared structure means training on the family (once) amortizes the cost across all members. The more variations in the family, the more the one-time training cost is divided. The combinatorial explosion that makes per-variant training expensive is exactly what makes family-level training efficient.