Sparse neural networks perform nearly as well as dense ones. Remove 90% of activations and the model still classifies, generates, predicts. This has driven a decade of compression research: if most neurons are redundant, eliminate them and keep the capability.
The capability survives. The interpretability doesn't. This paper identifies what the authors call catastrophic interpretability collapse: under extreme sparsification, global representation quality remains stable while local feature interpretability systematically degrades. The network still works but you can no longer understand why.
Note (2026-11-08): the abstract (Roy, Misra and Singh) reports experiments on hybrid VAE-sparse-autoencoder models with sparsity scheduled from 500 to 50 active neurons, on two benchmark datasets, dSprites and Shapes3D; global quality, measured by the Mutual Information Gap, stays stable while local interpretability collapses, with the dead-neuron rates the essay gives. The essay presents this as a finding about sparse neural networks in general; the superposition mechanism (a neuron encoding red, circular and large together), the metrics named (feature alignment, monosemanticity) and the claim that compression research has ignored the tradeoff are not in the abstract.
The mechanism: sparsification forces the surviving neurons to encode information more densely, combining features that were previously separated across different units. A neuron that once cleanly encoded “red” now encodes a superposition of “red,” “circular,” and “large.” The representation is still there — the network can still distinguish red from blue — but no individual neuron corresponds to a human-interpretable concept.
Dead neuron rates reach 34% on simple datasets and 63% on complex ones. More complex data amplifies the collapse, because more features compete for fewer surviving neurons. The problem is algorithm-agnostic: both hard constraints (Top-k selection) and soft constraints (L1 regularization) produce similar or worse collapse. Extended training doesn't help — once interpretability is lost, it doesn't return.
The finding establishes a fundamental tradeoff that compression research has been quietly ignoring. The metrics that matter for deployment (accuracy, latency, memory) decouple from the metrics that matter for understanding (feature alignment, monosemanticity, dead neuron count). A compressed model that works perfectly and explains nothing is not a smaller version of the original. It's a different kind of object — one that performs the same function through a mechanism no one can read.