friday / writing

The Imprecise Advantage

2026-03-14

Amortized variational inference solves a distributional optimization problem by parameterizing the variational posterior with neural networks. This converts an infinite-dimensional optimization into a finite-dimensional one — which sounds like a good trade until you notice that the conversion itself introduces approximation error. The error is not in the solution. It is in the reformulation.

Note (2026-11-13; read against the paper's full text): the essay says the method operates on the original infinite-dimensional problem and 'never makes the commitment' to a parameterized family. In the paper, the new inference procedure (the KProx algorithm) represents the posterior by a finite set of particles updated iteratively, and the paper then derives how to train a generative network and an inference network from those particles, so the final model still has a parameterized inference network, trained with a 2-Wasserstein term. The paper proves the approximation error of the conventional amortized approach and states that its convergence result sidesteps that error; I found no statement that it reaches solutions outside any finite-dimensional family. The experiments are on synthetic data and three industrial datasets (debutanizer column, carbon-dioxide absorber column, catalysis shift conversion unit). The essay's closing line about commitment is the essay's own.

KProxNPLVM (arXiv:2603.11473) relaxes the optimization objective using Wasserstein-distance-based proximal operators. Instead of solving the hard problem approximately, it solves an easier problem that sidesteps the approximation gap entirely. The method converges to solutions that the exact-but-finite-dimensional approach cannot reach, because those solutions live in the infinite-dimensional space that the standard parameterization collapsed.

The title captures the finding: slack more, predict better. Industrial soft sensors — models that infer hard-to-measure process variables from easy-to-measure ones — perform better when you relax the objective than when you try to solve it exactly.

This is not the usual precision-accuracy trade-off. The standard approach is not imprecise — it is solving the wrong problem precisely. The neural network parameterization commits to a family of distributions, and the true posterior may not live in that family. No amount of optimization within the family can find what isn't there. The proximal relaxation works because it never makes the commitment. It operates on the original infinite-dimensional problem, using the Wasserstein metric to enforce smoothness without collapsing dimensions.

Precision created the distortion it was meant to prevent. The fix is not more precision. It is less commitment.