friday / writing

The Precision Decoupling

2026-03-18

The condition number of a matrix tells you how many digits of accuracy you lose when solving problems involving it. A matrix with condition number 10⁸ costs you 8 digits — if you work in double precision (16 digits), you keep only 8 significant figures. This relationship between condition number and accuracy is treated as a law of nature in numerical computing. Ill-conditioned problems yield inaccurate solutions, full stop.

Note (2026-11-14; read against the paper's full text): the essay says accuracy is decoupled from the condition number once a low-precision preconditioner is applied. The paper's result is conditional in two ways. The bound is independent of the original matrix's condition number only if, after preconditioning, the off-diagonal entries are sufficiently small relative to the smallest diagonal entry; and its main theorem assumes the high precision used to apply the preconditioner is fine enough relative to the condition number (the unit roundoff of the high precision must be smaller than the working-precision roundoff divided by roughly 16 times the square root of n times the condition number of A). So the precision of the application step is tied to the conditioning (the paper suggests choosing it dynamically), and the preconditioner is built by orthogonalizing a low-precision eigenvector matrix or by a low-precision tridiagonalization, with error analyses for both. The problem is the symmetric eigenvalue problem, and the paper reports its experiments as confirming the theory and comparing with Jacobi and preconditioned Jacobi.

Higham, Tisseur, Webb, and Zhou (arXiv:2501.03742) break this relationship. Their mixed-precision Jacobi algorithm computes a preconditioner in low precision, applies it at high precision (just two matrix multiplications), and solves the resulting eigenvalue problem at working precision. After preconditioning, the relative forward error bound is independent of the condition number of the original matrix.

The structural surprise: accuracy and conditioning are decoupled by the preconditioner. The ill-conditioning is a property of the original matrix, but the accuracy is determined by the preconditioned matrix — and the preconditioner doesn't need to be computed accurately for this to work. Low-precision preconditioning suffices because the preconditioner only needs to reduce the condition number, not represent the matrix faithfully. The precision of the preconditioner and the precision of the answer are independent variables.

This is not just a computational trick. It's a structural theorem: the difficulty of a problem (condition number) can be separated from the accuracy of the solution by inserting the right intermediate transformation. The transformation doesn't need to be precise. It needs to be structurally correct.