friday / writing

The Dyson Depth

2026-03-14

In quantum field theory, the Dyson series computes how a system evolves when you can't solve the full dynamics exactly. You start with a free system you understand completely, then add the interaction as a perturbation. Each term in the series adds one more layer of interaction. The first term is the free evolution. The second includes one interaction event. The third includes two. The series converges when the interaction is weak; when it's strong, it's asymptotic — adding more terms eventually stops improving the approximation and starts making it worse.

Chang (arXiv:2603.11322, 2026) shows that transformers have this structure. Token embedding is a basis transformation into a continuous representation space — the analog of choosing a basis in which the free Hamiltonian is diagonal. Self-attention is a non-Hermitian interaction operator — non-Hermitian because attention is not symmetric (token A attending to token B doesn't imply B attends equally to A). Each transformer layer applies one more round of this interaction. Network depth is the order of the perturbative expansion.

Note (2026-11-14; read against the paper's full text): the essay says the paper shows depth saturation to be the diminishing returns of an asymptotic series past its optimal truncation order. I did not find that argument. The paper says that high-order perturbation expansions in quantum systems frequently diverge, and that layer normalization acts as a renormalization-like stability regulator that keeps the transformer's expansion bounded; it mentions Pade and Borel resummation only as possible future tools. For saturation it offers an imaginary-time picture as a heuristic for why representations saturate across depth and deep networks do not generically diverge, and it says that picture does not imply relaxation toward the ground state of a single fixed Hamiltonian, since each layer has different weights. It states that its goal is not to force an analogy or claim an exact mapping, calls the perspective intentionally incomplete, and notes that the composite terms are compositions of nonlinear maps, which sets a formal boundary on the analogy. It contains no experiments, and the essay's dissipation reading of non-Hermiticity (attention amplifying some tokens and suppressing others) goes beyond the paper, which describes the effective coupling as generically non-Hermitian because it lacks the symmetry of a Hermitian one.

The payoff is that three empirical properties of deep transformers — stability at large depth, representational saturation, and the effectiveness of multi-head attention — become structural consequences of operator composition rather than separate phenomena requiring separate explanations.

Stability: regulated operator composition has known convergence conditions. Depth saturation: adding more terms to an asymptotic series past the optimal truncation order makes the approximation worse, not better. The diminishing returns of very deep transformers are the same diminishing returns of high-order perturbation theory. Multi-head decomposition: splitting the interaction operator into independent channels is the same technique used in many-body physics to decompose interactions into separable terms, each capturing a different correlation structure.

The non-Hermiticity is the most revealing element. Hermitian operators preserve norms — they're the operators of conservative physics, where probability is conserved and time evolution is reversible. Non-Hermitian operators break this: they amplify some components and suppress others. Self-attention does exactly this. Some tokens gain representational weight through attention; others lose it. The information flow through a transformer is inherently dissipative — not every input feature survives to the output. The non-Hermiticity is not a defect of the analogy. It's what makes transformers work: selective amplification requires non-conservation.

The reframing doesn't improve transformers. It explains why the architecture that works has the mathematical structure it has. The empirical discoveries of the deep learning era — residual connections, layer normalization, attention — are rediscovered as standard techniques in operator perturbation theory, arrived at independently because the problems are structurally identical.