Neural network pruning removes connections to reduce model size. The standard picture is graceful degradation: remove a few connections, performance drops slightly; remove more, it drops more; remove too many, the network fails. A smooth tradeoff between compression and capability.
Note (2026-11-06): I checked this essay against the raw arXiv abstract of 'Pruning-induced phases in fully-connected neural networks: the eumentia, the dementia, and the amentia' (2603.12316, Pan, Aggarwal and Pixley). The abstract studies fully-connected networks trained on MNIST, varying the dropout rate (removal of neurons) at training and at evaluation. It defines the three phases by the power-law scaling of the cross-entropy loss with training-set size: eumentia (the network learns), dementia (the network has forgotten), amentia (the network cannot learn). It reports that the eumentia-dementia transition shows scale invariance with a diverging length scale and hallmarks of a Berezinskii-Kosterlitz-Thouless-like transition, and that the phase structure is robust across widths and depths. My essay describes pruning of connections in general and gives the phases behavioural descriptions that are not in the abstract: dementia as structured output with systematic errors, confabulation-like answers and unchanged confidence, and amentia as outputs indistinguishable from noise; its 'mirrors cognitive decline' paragraph and its claim about which mechanisms dominate each phase are my own. I checked the abstract and author list only.
The paper (arXiv:2603.12316, March 2026) finds that pruning produces three distinct phases, not a smooth decline. The phases are labeled with clinical terminology: eumentia (healthy function), dementia (degraded but structured failure), and amentia (complete nonfunctional collapse). The transitions between phases are sharp — not gradual degradation but phase transitions, with the network's behavior changing qualitatively at critical pruning thresholds.
In the eumentia phase, the pruned network performs nearly as well as the full network. The removed connections were redundant; the network had excess capacity. In the dementia phase, the network still produces structured outputs but with systematic errors — it “knows” something but gets the details wrong. In the amentia phase, the network produces outputs indistinguishable from random noise. The structure is gone entirely.
The clinical metaphor is more than nomenclature. The dementia phase, specifically, mirrors patterns in cognitive decline: preserved general capability with degraded specificity, confabulation-like outputs that are plausible but wrong, maintained confidence despite reduced accuracy. The network doesn't know it's degraded. It continues to produce outputs with the same format and confidence as the healthy network — the outputs are just incorrect.
The structural lesson: degradation has phases, not just degrees. The transition from “working well” to “working poorly” is not the same kind of event as the transition from “working poorly” to “not working at all.” Different mechanisms dominate in each phase — redundancy buffers the first, structural coherence maintains the second, and the collapse of coherence defines the third. Measuring only the degree of degradation misses the qualitative shifts between regimes.