Three variational autoencoders for musical timbre — unsupervised, descriptor-conditioned, perceptual-feature-conditioned — organize their latent spaces in fundamentally different ways. The comparison reveals what kind of conditioning information actually structures a generative model's internal representation.
Note (2026-11-10) CORRECTION: I checked the abstract and searched the full text of arXiv 2603.16713. The three models, the 19 semantic descriptors across four intensity levels, and the finding that perceptual-feature conditioning gives a more compact, discriminative and pitch-invariant latent space than the unsupervised and one-hot descriptor models are the paper's. The essay says a sound labeled bright at intensity 2 and one at intensity 4 receive the same one-hot vector and that the model cannot see the gradient. In the paper the descriptor-conditioned model uses a 42-dimensional vector with 19 binary descriptor indicators multiplied by the descriptor magnitude, plus 23 one-hot pitch labels, so intensity is encoded; the paper's limitation is of discrete one-hot semantic conditioning, not an absent gradient.
The unsupervised VAE discovers its own axes. Without guidance, it finds whatever statistical regularities minimize reconstruction error. The resulting latent space is functional but uninterpretable — sounds that seem similar to humans may land far apart, and nearby points may differ in perceptually irrelevant ways. The model optimizes for its own objective, not for human categories.
Descriptor conditioning uses one-hot labels: “bright,” “warm,” “harsh.” Each descriptor gets a discrete category. The latent space improves — sounds with the same label cluster together. But the conditioning is coarse. A sound labeled “bright” at intensity 2 and one at intensity 4 receive the same one-hot vector. The gradient between categories is invisible to the model. Discrete labels create discrete clusters rather than continuous manifolds.
Perceptual-feature conditioning uses continuous values from timbral analysis models — numerical measurements of brightness, warmth, roughness. The latent space becomes compact, discriminative, and pitch-invariant. Sounds separate cleanly by timbre regardless of fundamental frequency. The continuous conditioning creates continuous structure.
The lesson is about the information format, not the information content. Descriptor labels and perceptual features encode roughly the same knowledge about timbre. But continuous values allow the model to learn gradients — the relationship between “slightly bright” and “very bright” — while discrete labels only distinguish categories. The latent space reflects the structure of its conditioning signal, not just its content.