LLM-based security code review is becoming standard — from interactive assistants to autonomous CI/CD agents. The implicit assumption is that the model evaluates code on its merits, not on how the code is presented. This paper tests that assumption.
The answer is unsettling. Framing a code change as “bug-free” in the review prompt reduces vulnerability detection by 16-93% across four state-of-the-art models. The bias is asymmetric: false negatives increase sharply while false positives barely change. The model is easily told that safe code is safe but hard to convince that dangerous code is safe — except that it IS easily convinced, through framing rather than through logic.
The exploitability study is worse. Adversarial pull requests that reintroduce known vulnerabilities succeed in 35% of cases against GitHub Copilot with one-shot framing, and in 88% of cases against Claude Code when adversaries can iteratively refine their framing. The fix — metadata redaction and explicit debiasing instructions — restores detection in 94% of autonomous cases. The bias isn't in the model's capability; it's in its attention to social context. Stripping the social signal recovers the technical judgment. But deployed systems include the social context by default, which means confirmation bias is the default operating mode of LLM code review.
Note (2026-11-06): the abstract of arXiv:2603.18740 (now v5; first and current abstracts share about 26 percent of their wording) has changed substantially since its first version. I wrote this essay in early 2026 and may have worked from the earlier version, so it can be accurate about the version I read and still differ from the paper as it now stands. I have not compared this essay with either version in this pass.