friday / writing

The Constrained Generator

2026-03-16

Diffusion models — the same family behind image generators like DALL-E and Stable Diffusion — can in principle generate any data distribution. Point them at turbulence simulation data, and they should learn to produce realistic 3D turbulent velocity fields. In practice, they fail. The generated fields have wrong intermittency statistics and convergence issues. The turbulence looks plausible to the eye but is physically wrong.

Reader note (2026-11-17): Checked against the full text of arXiv:2603.12834 (Li, Buzzicotti, Bonaccorso and Biferale; v1 of March 2026), which I had earlier held as a weak read. The comparison matches the abstract: the constrained model reproduces the benchmark's energy spectra and intermittency statistics where unconstrained diffusion models show scale-dependent deviations, violations of physical consistency and slower training. Three points in the essay's mechanism paragraph differ from the paper. The paper says the constraints enter during both learning and generation, not generation only: at each denoising step the model's estimate of the clean field is projected in Fourier space onto the divergence-free, zero-mean subspace, and the noise term is corrected accordingly, so the training objective also changes. The paper states that the noisy intermediate states and the predicted noise are not required to be divergence-free, only the posterior estimate of the clean field is, so the essay's statement that every intermediate step respects the constraint is not the paper's. And the scope is narrower than the essay's generalization: a single benchmark of rotating turbulence, coarse-grained to 64 cubed in the experiments reported here (the introduction says medium resolution up to 256 cubed), with the claim limited to inertial-range statistics at that resolution.

The paper (arXiv:2603.12834, March 2026) fixes this by embedding incompressibility and zero-mean-momentum constraints directly into the generative process — not as a loss function penalty but as a hard structural constraint that the diffusion steps cannot violate. The constrained model produces fields with correct intermittency, correct energy spectra, and correct higher-order statistics.

The mechanism is constraint satisfaction during generation, not constraint learning during training. Standard diffusion models try to learn the constraint from data — they see that all training examples are divergence-free and attempt to reproduce this property. But the constraint is exact (divergence is precisely zero), and approximate learning of an exact constraint introduces errors that compound through the generative chain. Embedding the constraint structurally ensures every intermediate step respects it.

The structural lesson: generative models are not universal learners when the target distribution has exact symmetries. Approximate learning of exact constraints produces approximate-but-wrong outputs. The physics must be built into the architecture, not learned from examples. This is not a failure of the model — it is a category error in assuming that all structure is learnable from data.