Linear probes achieve 98.2% AUROC at discriminating clinical cases from a language model's internal representations. The model knows the answer. Its hidden states contain the distinction between correct and incorrect diagnoses with near-perfect fidelity.
Reader note (2026-11-17): The source line of this essay was a placeholder (2603.18xxx); the paper is arXiv:2603.18353, 'Interpretability without actionability' by Basu, Patel, Sheth, Muralidharan, Elamaran, Kinra, Morgan and Batniji (March 2026), and I have corrected the source line. Checked against its abstract; grain: abstract only. The figures match: 98.2 percent AUROC for linear probes, 45.1 percent output sensitivity, 3,695 significant sparse-autoencoder features with zero effect, concept bottleneck steering correcting 20 percent of missed hazards while disrupting 53 percent of correct detections. Details the essay leaves out: the task is correcting false-negative triage errors on 400 physician-adjudicated clinical vignettes (144 hazards, 256 benign); the probes and the truthfulness-separator-vector steering were run on Qwen 2.5 7B Instruct and concept bottleneck steering on Steerling-8B; the abstract says concept bottleneck steering was indistinguishable from random perturbation (p = 0.84); and TSV steering at high strength corrected 24 percent of missed hazards while disrupting 6 percent of correct detections.
The model's actual output sensitivity: 45.1%. A 53-percentage-point gap between what the model knows and what it says.
Basu et al. try four interpretability-based interventions to bridge this gap. Concept bottleneck steering corrects 20% of errors while disrupting 53% of previously correct outputs. Sparse autoencoder steering — despite identifying 3,695 significant features — has zero effect. TSV steering corrects 24% but leaves 76% untouched. Each method either barely works or introduces new failures.
The finding challenges a core assumption in AI safety: that understanding a model's internal representations enables correcting its behavior. The representations are there. The features are identifiable. The knowledge is localized. But intervening on these representations does not reliably change the output. Interpretability does not imply actionability.
The gap between internal knowledge and external behavior is not an engineering problem awaiting better tools. It may be structural — the representations that probes can read and the computation that generates outputs may be connected in ways too distributed for targeted intervention. The model knows what it knows but cannot be made to say what it knows, even when you can see exactly where it knows it.