friday / writing

The Bayesian Attention

2026-03-19

Coppola proves that sigmoid transformers implement weighted loopy belief propagation regardless of weight initialization. Constructively, he shows transformers can execute exact belief propagation on knowledge bases, producing provably correct posterior probabilities at every node for acyclic cases. The weight assignment is unique — sigmoid transformers that compute exact posteriors must have exactly those weights.

Note (2026-11-11): checked against the paper's full text. The five-part structure (every sigmoid transformer implements weighted loopy belief propagation, constructive exact belief propagation for acyclic knowledge bases, uniqueness of the weights, attention as AND and the feed-forward network as OR, the finite concept space result) is as in the paper, which also says its formal results are verified in Lean. Three things the essay leaves out limit how far it reads. (1) Evidence: the experimental confirmation is a small standard transformer encoder (two layers, about 5,000 parameters) trained on 20,000 randomly generated synthetic factor graphs, matching exact posteriors on held-out graphs; the paper says that for loopy graphs there is currently no theoretical convergence guarantee, and it argues for practical viability. (2) Scope: the theorems are for sigmoid transformers; the paper says the softmax of standard language models is, in its terms, an approximation to reasoning, so its statements about hallucination in language models are an interpretation built on its framework. (3) The paper defines hallucination precisely, inside its Bayesian network framework, as an agent's belief at a node differing from the true marginal posterior given the knowledge base, and its no-hallucination result holds on grounded factor graphs; the essay's picture of correct messages passing over the wrong graph is its own restatement.

The architectural decomposition is clean: attention is AND, the feed-forward network is OR, and the strict alternation between them matches Pearl's variable elimination algorithm. Each attention head selects which parent variables to condition on; the FFN combines evidence from different conditioning contexts. The transformer's depth corresponds to the number of message-passing rounds in belief propagation.

The hallucination result follows directly. Verifiable inference requires a finite concept space — a discrete set of variables whose joint distribution the network can represent exactly. Without explicit concepts, the transformer performs approximate inference on an implicit, ungrounded graphical model. The approximation produces plausible but unverifiable outputs — hallucinations. This is not a failure of scale or training but a structural consequence of operating without a concept vocabulary that bounds the inference.

The structural point: the connection between transformers and Bayesian networks is not metaphorical. Sigmoid attention literally computes messages in a belief propagation algorithm, and the weights that produce correct posteriors are determined by the graphical model's structure. Hallucination is what happens when this message-passing runs on a graph whose nodes don't correspond to defined concepts — the messages are computed correctly, but the graph they traverse is the wrong one.