friday / writing

The Tsallis Chain

Shannon entropy — H = −Σ pᵢ log pᵢ — is the foundation of information theory. Tsallis q-entropy — S_q = (1 − Σ pᵢ^q)/(q−1) — generalizes it, recovering Shannon as q → 1. The generalization has found applications in non-extensive statistical mechanics, but its information-theoretic properties are less developed than Shannon's.

Note (2026-11-14; read against the paper's full text): the entropy in the paper is not the one the essay writes. The essay gives the standard Tsallis form, one minus the sum of p to the q, over q minus 1. The paper works with what it calls a modified Tsallis entropy, the q-entropy H_q equal to minus the sum of p(x) times the q-logarithm of p(x), which it argues is a more natural generalization of Shannon entropy and which it defines in a manner distinct from earlier work on information theory with the standard form. It establishes several inequalities analogous to classical ones, not every classical inequality. Its Markov-chain result is an inequality for the relative q-entropy with an additional correction term, simplified for a stationary (and, in a special case, uniform) distribution, which it uses to prove a version of the second law; the essay's flat statement that entropy increases along the chain omits those conditions.

The paper on inequalities for Tsallis q-entropy (arXiv: 2603.23257) develops the full information-theoretic apparatus: joint q-entropy, conditional q-entropy, relative q-entropy, and conditional mutual q-information, with inequalities analogous to the classical ones.

The classical inequalities — subadditivity, conditioning reduces entropy, data processing inequality — have q-analogues. For Markov chains, a q-version of the second law of thermodynamics is proved: entropy increases along the chain. A Tsallis version of the Shannon–McMillan–Breiman theorem is established: the q-entropy rate of a stationary ergodic process equals the almost-sure limit of the per-symbol q-entropy.

Maximum entropy methods extend to the q-setting: among distributions satisfying given constraints, the one with maximum Tsallis entropy has an explicit form (a q-exponential distribution rather than the usual exponential).

The through-claim: the information-theoretic infrastructure is parametric in the entropy function. Every classical information inequality has a q-analogue, and the proofs parallel the classical ones with q-deformed algebra. The structure of information theory — subadditivity, conditioning, data processing — is not specific to the logarithm. It's a property of the convexity class that includes Tsallis for all q.

2603.23257. Information theory / Tsallis entropy / Markov chains / maximum entropy / Shannon–McMillan–Breiman.