Data with power-law eigenvalue decay — covariance eigenvalues falling as λ_k ~ k^{-α} — is ubiquitous: natural images, text embeddings, genomic data. Pass this data through a random feature layer (random linear projection followed by a nonlinear activation), and the output has its own spectral structure. Does the power law survive?
Yes. The output covariance inherits the exponent α exactly, modified only by a logarithmic correction that depends on the degree p of the activation function's dominant monomial term. The power law is preserved through the nonlinear transformation; only the sub-leading behavior changes.
Note (2026-11-08): the abstract treats a specific case: Gaussian data with an alpha-power-law covariance spectrum, a Gaussian sketch matrix and an entrywise monomial f(y) = y^p. It proves that for indices j up to c1 d log^-(p+1)(d) the j-th eigenvalue is of order (log^(p-1)(j+1)/j)^alpha, and for larger j up to d it is of order j^-alpha up to a polylog factor, so the exponent alpha is inherited. The essay states the result for a generic activation function and says the exponent is inherited with one logarithmic correction throughout; it also says deep networks stay in the power-law regime regardless of depth, and describes the proof through dyadic decomposition and Wick chaos expansions. Those statements are not in the abstract, which concerns a single layer with monomial activation.
The proof uses dyadic decomposition — splitting the spectrum into octaves — combined with Wick chaos expansions that decompose the nonlinearity into orthogonal polynomial components. Each monomial degree p contributes its own spectral decay, and the dominant contribution preserves the input exponent. Random matrix concentration handles the finite-width effects.
The matching upper and lower bounds are the key technical achievement. An upper bound alone would say “the output can't have a slower decay than the input.” The matching lower bound says “the output can't have a faster decay either.” The spectral structure is genuinely inherited, not just bounded.
This matters for understanding depth. If each layer preserves the spectral exponent, then deep networks process power-law data in the power-law regime regardless of depth. The spectral structure is a conserved quantity of the random feature transformation — the nonlinearity changes the representation but preserves this statistical fingerprint.