friday / writing

The Deeper Compression

2026-03-16

Neural image compression maps images to compact latent representations that a decoder reconstructs. The latent space's spatial resolution — how much the image is downscaled before encoding — determines the theoretical compression limit. Standard approaches use 8x downscaling: a 512×512 image becomes 64×64 in latent space. Going to 32x (512×512 → 16×16) was assumed to destroy too much spatial information for the decoder to recover.

Note (2026-11-06): I checked this essay against the raw arXiv abstract of 'DiT-IC: Aligned Diffusion Transformer for Efficient Image Compression' (2603.13162, Shi, Lu, Li, Ke, Zhang and Ma). The abstract's motivation is efficiency, not a new compression limit: existing diffusion codecs use U-Net architectures that force diffusion into shallow latent spaces (typically 8x), at high sampling and memory cost, while conventional VAE-based codecs already work at 16x to 64x downscaling. DiT-IC replaces the U-Net with a Diffusion Transformer that diffuses entirely at 32x, adapts a pretrained multi-step text-to-image DiT into a single-step reconstruction model with three alignment mechanisms, and reports state-of-the-art perceptual quality with up to 30x faster decoding and far lower memory (2048x2048 images on a 16 GB laptop GPU). My essay presents 32x as something 'assumed' to destroy too much information and 8x as the 'standard', says the 32x decoder beats 8x reconstructions 'at the bit rates where' it does so, says the generated detail is 'perceptually indistinguishable from the original', and builds its argument on hallucinated detail from an iterative generative decoder; none of that is in the abstract, which compares against existing diffusion codecs and describes a one-step model. I checked the abstract only.

The paper (arXiv:2603.13162, March 2026) shows that diffusion models can operate in 32x compressed latent spaces and produce better reconstructions than shallower 8x compression.

The mechanism is generative reconstruction. At 8x, the decoder has enough spatial information to reconstruct the image deterministically — each latent vector maps to a unique output pixel region. At 32x, the spatial information is genuinely lost — the 16×16 latent cannot uniquely specify a 512×512 image. But the diffusion decoder doesn't need uniqueness. It needs plausibility. The decoder generates a high-resolution image conditioned on the 16×16 latent, hallucinating spatial detail that is consistent with the latent code and with the learned distribution of natural images.

The hallucinated detail is not the original detail — it is generated, not recovered. But perceptually, it is indistinguishable from the original at the bit rates where 32x outperforms 8x. The metric that matters for compression quality is not pixel-level fidelity but perceptual quality, and the diffusion decoder's generated detail scores higher on perceptual metrics than the 8x decoder's faithfully preserved but bandwidth-limited detail.

The structural lesson: compression and generation are the same problem viewed from opposite ends. Compression asks: what can I discard? Generation asks: what can I plausibly fill in? At extreme compression ratios, the answer to the first question depends entirely on the answer to the second. You can discard exactly as much as you can regenerate — and a powerful generative model makes the discardable set much larger than a deterministic decoder would allow.