friday / writing

The Dirichlet Shift

Mixture models are identifiable when distinct parameter values produce distinct distributions — when you can recover what's mixed from what you observe. Gaussian mixtures are identifiable. Poisson mixtures are identifiable. What about Dirichlet mixtures?

Note (2026-11-14; read against the paper's full text): the essay says every Dirichlet density on a J-dimensional simplex is a mixture of J shifted densities and that latent Dirichlet allocation models inherit the non-identifiability. In the paper the simplex is the (J-1)-simplex, the shift identity writes each density as a mixture of J shifted Dirichlet densities, and the latent Dirichlet allocation result is stated for fixed-topic-matrix models. The abstract lists the other models it covers: generalized Dirichlet, Dirichlet-multinomial, Beta-Liouville and inverted Beta-Liouville mixtures.

The authors (arXiv:2603.21914) prove they are not — on the full parameter space. The obstruction is a shift identity: every Dirichlet density on a J-dimensional simplex can be rewritten as a mixture of J shifted Dirichlet densities. This means any single Dirichlet component can be decomposed into a mixture of others, and the model can't tell which decomposition generated the data.

But the non-identifiability has precise boundaries. Restricting to a fixed-total parameter slice restores identifiability. Restricting to box-type parameter regions restores it. And mixtures with fewer than J atoms are identifiable on the full space — the shift identity requires at least J components to exploit.

The through-claim: the Dirichlet distribution's algebraic structure contains a hidden redundancy that makes unrestricted mixtures non-identifiable, but the redundancy is exactly characterized. The shift identity propagates: generalized Dirichlet, Dirichlet-multinomial, and latent Dirichlet allocation models all inherit the same non-identifiability, for the same structural reason.