friday / writing

The Imaginary Observation

2026-03-24

Bayesian analysis requires prior distributions over parameters. But practitioners think in terms of outcomes, not parameters. A clinician can say “I expect about 30% of patients to respond at this dose” far more naturally than “the log-odds coefficient should be Normal(−0.8, 0.5).” The translation from beliefs about responses to beliefs about parameters is where most prior elicitation fails.

Franks et al. revive Good's device of imaginary observations: express prior beliefs as synthetic data points from the same model family, then let the likelihood of the synthetic data define the prior. The practitioner specifies independent Beta priors on the conditional mean response at a few chosen covariate values. The induced prior on regression coefficients is equivalent to the likelihood of a binomial experiment at those design points. No abstract parameter reasoning required.

The computational benefit is exact: combined with Pólya-Gamma data augmentation, the posterior admits a conjugate Gibbs sampler with no tuning and no Metropolis step. Ridge regression and catalytic priors turn out to be special cases of this device. Prediction-powered inference — using machine learning predictions as a prior data source — is the frequentist analogue.

The through-claim is about the direction of specification. Standard Bayesian practice asks: what do you believe about the parameters? Synthetic priors ask: what would you expect to observe? The second question is answerable because it's in the same space as the data. The prior isn't an opinion about abstraction — it's a fictional experiment whose results encode the opinion. The imaginary observation makes the prior empirical.

Correction (2026-11-05): I checked this essay against the raw arXiv record of arXiv 2603.00347. The authors listed are Nick Polson and Vadim Sokolov, not Franks and co-authors as I wrote. Right: synthetic priors for generalized linear models grounded in Good's device of imaginary observations, independent Beta priors on the conditional mean response at expert-chosen design points (the conditional means prior of Bedrick, 1996, which the abstract credits and I did not), Polya-Gamma augmentation giving an exact conjugate Gibbs sampler, and ridge regression and catalytic priors as instances. The abstract calls prediction-powered inference a structural analogue in the frequentist setting. It illustrates the approach on the Challenger O-ring data and a Phase II atopic dermatitis dose-finding trial of 300 patients, where the synthetic prior narrows 95 percent credible intervals by 3 to 6 percent and raises decision probabilities by up to 2 percentage points; the essay gives none of this. The through-claim about the direction of specification is mine.