Spiking neural networks are supposed to be efficient — they communicate through sparse binary events rather than continuous activations. But training them with standard optimizers produces dense weight matrices. The sparsity of the spikes doesn't propagate to sparsity of the parameters. You get efficient inference but wasteful storage.
Note (2026-11-13; read against the paper's full text): the accuracy claim is weaker than 'same accuracy'. The paper reports that the Bregman-based optimizer (AdaBreg) reaches test accuracies 'within 0.5-1.5% of the Adam baseline across all three datasets, despite limited hyperparameter tuning', together with the roughly 50 percent reduction in active parameters, and calls the gap 'acceptable for sparsity-constrained neuromorphic applications'; the baseline numbers are taken from an earlier paper (Queant et al., 2025). The three benchmarks are Spiking Heidelberg Digits, Spiking Speech Commands and Permuted Sequential MNIST, and the essay lists two. The paper also says the regularization parameter lambda clearly influences both accuracy and sparsity, and that its best choice depends on the learning rate and the dataset. On the theory, the paper says there exist algorithms based on linearized Bregman iterations that can provably converge to sparse optimal solutions for sparse optimization problems with a convex regularizer, and it describes its experimental result as showing the 'potential' of convex sparsity-inducing methods for neuromorphic learning; the essay's 'the convex optimization theory guarantees convergence to a sparse solution' reads that as a guarantee for training a spiking network, which I did not find the paper claiming. The comparison with pruning (dense solutions, then deletion) is the essay's framing of the paper's remark that sparsity is often achieved through heuristic approaches such as pruning.
Windhager, Moser, and Lunglmayr enforce sparsity during training using Linearized Bregman Iterations — a convex optimization technique that naturally produces sparse solutions through iterative soft thresholding. The optimizer doesn't just find good weights; it finds sparse good weights, reducing active parameters by about 50% while maintaining accuracy on three neuromorphic benchmarks.
The through-claim: the sparsity and the optimization are the same operation, not separate concerns. Standard training finds dense solutions and then prunes. Bregman iteration finds sparse solutions directly — the proximal operator enforces sparsity at every step, not as a post-processing stage. The weights are sparse because the optimization landscape is shaped to prefer sparse solutions, not because small weights were deleted after the fact.
The momentum-corrected variant (AdaBreg) extends Adam with Bregman distance minimization, making it drop-in compatible with existing training pipelines. The convex optimization theory guarantees convergence to a sparse solution. The practical result: half the parameters, same accuracy, on standard neuromorphic benchmarks (Spiking Heidelberg Digits, Spiking Speech Commands).
If spiking networks are supposed to be sparse, the optimizer should be too.