Chain-of-thought reasoning and data compression seem like different problems. Reasoning adds tokens — extending the output to include intermediate steps. Compression removes tokens — finding shorter representations that preserve essential information. One expands; the other contracts.
Note (2026-11-14; read against the paper's full text): the essay says the paper proves chain-of-thought is lossy compression, that budget forcing is equivalent to adjusting the compression rate, and that accuracy follows a rate-distortion curve with a calculable minimum token count. The paper posits that an ideal reasoning chain is the minimal sufficient statistic of the prompt for the answer, and proves that standard length-based penalties correspond to the rate term of its conditional information bottleneck with non-informative priors. It argues that uniform length penalties and hard limits conflate essential reasoning with redundancy, and proposes a language-model prior that gives a per-token cost instead. Rate-distortion appears only as the trade-off parameter in its objective; I found no minimum token count, and its evidence is experiments on two model families across five benchmarks.
The paper (arXiv:2603.08462, March 2026) proves they are the same problem. Chain-of-thought generation is lossy compression through the conditional information bottleneck. The reasoning chain is a compressed representation of the input that preserves exactly the information needed to predict the output. The “thinking” tokens are a code — not in the cryptographic sense but in the information-theoretic sense. They are the minimal sufficient statistic for the answer, expressed in natural language.
The unification has a concrete consequence: budget forcing — limiting the number of reasoning tokens a model can use — is equivalent to adjusting the compression rate. Fewer tokens means a coarser code, which preserves less information, which produces less accurate answers. The relationship between token budget and accuracy follows the rate-distortion curve from information theory: there is a theoretical minimum number of tokens needed to achieve a given accuracy, and this minimum is calculable.
The equivalence also explains why chain-of-thought sometimes hurts. If the reasoning chain includes information that is not needed for the answer — decorative tokens, redundant restatements, organizational scaffolding — the compression is inefficient. The chain is longer than the minimal code, wasting capacity on irrelevant information. An ideal reasoner would produce exactly the tokens needed to compress the input to the answer and no more.
The structural lesson: thinking is compression. The purpose of intermediate reasoning is not to “show work” or to make the process legible — it is to construct a compressed representation that maps input to output. The tokens that seem like reasoning are a communication channel between the problem and the solution, and the efficiency of that channel is governed by the same laws as any other communication channel.