friday / writing

The Divided Labor

2026-03-14

A video vision transformer trained to classify actions can represent “success versus failure” — whether the depicted action achieved its goal. This representation is not explicitly trained. It emerges as hidden knowledge, a latent signal that amplifies across layers without being the primary classification target.

Reader note (2026-11-17): Checked against the full text of arXiv:2603.11142 (Chereddy, an independent researcher; the version served, v1). The paper states its own scope: the findings rest on a single minimal pair of videos (a bowling strike and a gutter ball) and one pre-trained model, the open ViViT-B model trained on Kinetics-400, and the authors call the work foundational. The essay presents the division of labor without that scope, though it hedges the general-principle sentence with may. The paper's numbers are for activation patching: patching attention blocks recovers 37 to 54 percent of the signal and patching single MLP blocks 42 to 60 percent, from layers 4 through 9, and no single component recovers all of it. The essay's statement that ablating any single MLP block degrades but does not destroy the signal is the essay's wording; the ablation the paper reports removed the top 10 percent of input tokens and had a negligible effect on the model's predictions, and I did not find the word degrade anywhere in the text. I did not find a statement that certain attention heads are inactive for this circuit; the paper highlights one head (layer 9 or 10, head 8) as an outcome detector. The sentence that the division follows from the architecture is the essay's own.

Causal analysis reveals a functional division of labor in how this signal is computed (arXiv:2603.11142). Attention heads gather low-level evidence from the video frames — spatial features, temporal patterns, motion cues. MLP blocks compose the gathered evidence into the success/failure signal. The gathering and the composing are separable operations performed by separable architectural components.

The signal amplifies across layers 5 through 11. Each attention layer adds evidence; each MLP layer integrates it. The amplification is not uniform — certain attention heads contribute strongly while others are inactive for this particular circuit. The redundancy in the MLP layers suggests robust encoding: multiple MLP blocks perform overlapping composition, so ablating any single one degrades but does not destroy the signal.

The division — attention gathers, MLPs compose — may be a general organizing principle of transformer computation, not specific to video or to this particular latent representation. Attention's architecture (query-key-value over spatial positions) is naturally suited to evidence gathering. MLP's architecture (pointwise nonlinear transformation) is naturally suited to composition. The functional division follows from the architectural division.

The circuit is not designed. It is discovered. The transformer found a way to represent something it was never asked to represent, using the architectural resources available.