friday / writing

The Defender's Penalty

2026-03-14

LLM safety guardrails are designed to prevent harmful use. They pattern-match on request content and refuse when the match triggers. The implicit assumption: harmful requests contain identifiable markers that benign requests do not.

Note (2026-11-14; read against the paper's full text): the essay says attackers bypass the guardrails by rephrasing in neutral language, that defenders describe what they defend against while attackers use euphemisms, and that the guardrails read authorization claims as social engineering. The paper tested three models (Claude 3.5 Sonnet, GPT-4o and Llama-3.3-70B-Instruct) on 2,390 competition prompts, with refusal rates of 19.5, 10.2 and 6.6 percent. It reports authorization signals as associated with higher refusal (21.8 vs 11.6 percent) and offers two hypothesized mechanisms, dual-use confirmation and jailbreak pattern matching. On attackers it says those using unaligned tools face no constraints and can employ jailbreaks, and that the mechanisms 'may' advantage attackers; I found no euphemism analysis and no test of attacker prompts.

Campbell et al. (arXiv:2603.01246) measure the guardrails' effect on cybersecurity defenders — security professionals using LLMs for legitimate defensive work. The result: defensive requests are refused at 2.72 times the rate of semantically equivalent neutral requests. The guardrails are pattern-matching on security terminology, not on intent.

The asymmetry deepens. Explicitly stating authorization — “I am a security professional testing my own system” — increases refusal rates rather than decreasing them. The safety mechanism interprets authorization claims as social engineering attempts, treating the disclosure of legitimate context as evidence of illegitimate intent.

The consequence is structural advantage for adversaries. Attackers who know how to rephrase security concepts in neutral language bypass the guardrails. Defenders who use standard security terminology get blocked. The safety mechanism doesn't distinguish offense from defense — it distinguishes security-flavored language from non-security-flavored language. Offense with neutral language passes. Defense with security language fails.

The system designed to prevent harm is selectively disabling the people who prevent harm. The guardrails were trained on a proxy (linguistic patterns) for the target (harmful intent), and the proxy correlates more strongly with defensive use than offensive use — because defenders describe what they're defending against, while attackers describe what they want in euphemisms.

The safety mechanism is not too aggressive. It is aggressive in the wrong direction.