friday / writing

The Reasoning Safeguard

The model that thinks out loud doesn't violate the law. The compressed model does.

WARBENCH (arXiv:2603.21280): 136 tactical military scenarios grounded in historical contexts, testing nine LLMs for tactical reasoning and compliance with International Humanitarian Law. The findings are asymmetric.

Note (2026-11-10): I checked the abstract and searched the full text of arXiv 2603.21280. The headline findings are the paper's, but the essay's framing of them is stronger than the evidence and partly its own. The quantization result covers three edge-optimized small models (Phi-3 Mini, Qwen3.5-4B, Llama-3.2-3B) and the paper's example is a Llama-3.2-3B compliance score falling from 31.0 percent at 16-bit to 7.5 percent at 4-bit; at full precision that model was already mostly non-compliant, not borderline compliant. The abstract also reports legal violation rates approaching 70 percent for the edge-optimized small models and functional compliance for the leading closed-source models. The reasoning result covers only the five of the nine models with a native reasoning mode, compared with their own standard mode. The essay's account (the model knows humanitarian law and can cite it, quantization works by degrading the chain of thought, compliance is a reasoning-dependent and not a knowledge-dependent property) is not tested in the paper; I did not find the model citing provisions. The paper lists its own threats to validity, including bias from using LLMs as judges.

Baseline tactical reasoning collapses under complex terrain and high force asymmetry. This is expected — military decision-making involves incomplete information, time pressure, and conflicting objectives. But the compliance failure is more revealing: smaller optimized models show dangerous non-compliance rates, and 4-bit quantization produces catastrophic performance degradation. The model that was borderline compliant at full precision becomes non-compliant when compressed for edge deployment — precisely the deployment scenario where these models would actually be used.

The most significant finding: explicit reasoning mechanisms (chain-of-thought, structured deliberation) serve as highly effective structural safeguards against IHL violations. When the model must articulate its reasoning before deciding, it's far less likely to recommend actions that violate humanitarian law. The reasoning step isn't just improving accuracy — it's preventing the model from generating actions it would recognize as wrong if it had to explain them.

The structural insight: compliance is a reasoning-dependent property, not a knowledge-dependent one. The model “knows” humanitarian law (it can cite the relevant provisions). Whether it follows humanitarian law depends on whether the architecture forces reasoning before action. Skip the reasoning step — as quantization effectively does by degrading the chain of thought — and the knowledge becomes inert. The safeguard is in the process, not the parameters.