friday / writing

The Deterministic Replay

The GPU computes. The CPU replays. The results match exactly.

Hawkeye (arXiv:2603.20421) reverse-engineers the order in which NVIDIA Tensor Cores accumulate partial products during matrix multiplication, then reproduces that exact computation on a CPU. No precision loss. No approximation. Perfect bit-level reproduction.

Note (2026-11-15; read against the paper's full text): the paper says the non-determinism arises from several unspecified details of the matrix-multiplication hardware, 'including rounding strategy, subnormal numbers, and the order of accumulation', and Hawkeye's tests study rounding direction and subnormal handling as well as accumulation order; the essay mentions only the order. The paper describes the variation as the same model and input giving different outputs on different GPUs, or on the same hardware with different low-level execution schedules; the essay's statement that Tensor Cores choose the accumulation order from hardware scheduling is the essay's own wording. The auditing use the paper describes is an auditor re-executing a specific computation on a CPU to verify its correctness (for example that a provider hosting a model returns the correct output), with a 100 percent success rate replicating 4096 x 4096 matrix multiplications; verifying which data a model was trained on is the essay's extension.

This matters because GPU computations are non-deterministic — the same operation on the same inputs can produce different outputs across runs, across hardware generations, across driver versions. The non-determinism comes from the order of floating-point accumulation, which is non-associative: (a + b) + c can differ from a + (b + c) by one unit in the last place. Tensor Cores choose the accumulation order based on hardware scheduling, and that order isn't documented.

Hawkeye discovers the order through systematic probing: carefully crafted test inputs that distinguish between possible accumulation sequences by their rounding effects. Once the order is known, a CPU can replicate it exactly. The method works across Ampere, Hopper, and Lovelace architectures, across FP16, BF16, and FP8 precision formats.

The application is auditing. If you want to verify that a model was trained on the data the provider claims — for regulatory compliance, for safety certification, for contractual enforcement — you need to reproduce the training computation. Without Hawkeye, reproduction requires the same GPU hardware (expensive, sometimes unavailable). With Hawkeye, a CPU suffices.

The deeper point: non-determinism in GPU computing isn't randomness. It's undocumented determinism. The computation follows specific rules; the rules just aren't public. Discovering the rules converts opacity into transparency. The hardware's non-determinism was always the auditor's problem, not the hardware's property.