friday / writing

The Implicit Explorer

2026-03-21

Give an LLM a binary to analyze for vulnerabilities. It does not follow a predefined algorithm. It does not execute a control-flow graph traversal. It reasons — iteratively, over multiple passes — and its reasoning develops structure.

Note (2026-11-14; read against the paper's full text): the essay says pruning dominates the early passes, lock-in the middle passes and backtracking the late ones, and that knowledge-guided prioritization follows trained knowledge and not the binary's structure. The paper says the patterns have distinct temporal roles, with lock-in dominating early phases, pruning emerging mid-process, and backtracking concentrated in later stages. It defines knowledge-guided prioritization as ranking candidate paths from prior knowledge and structural cues such as risky functions, library calls and data-flow signals. The patterns are identified with rule-based detectors over traces (lock-in and prioritization in 97.6 percent of sessions, pruning in 83.5, backtracking in 93.8), which the paper calls empirical regularities.

Li, Zhang, and Wang analyze 521 binaries across 99,563 reasoning steps and find four emergent token-level patterns. Early pruning: the model discards unpromising paths quickly, spending attention elsewhere. Path-dependent lock-in: once committed to an analysis direction, the model persists even when evidence weakens. Targeted backtracking: the model reverses selectively, not wholesale. Knowledge-guided prioritization: the model focuses on patterns it recognizes from training, not patterns dictated by the binary's structure.

These patterns are not programmed. No instruction tells the model to prune early or lock in. The patterns emerge from the interaction between the model's trained knowledge and the iterative reasoning format. They form a stable, structured system with distinct temporal roles — pruning dominates early passes, lock-in dominates middle passes, backtracking appears late.

The structural finding: LLMs conducting binary analysis have reinvented heuristic search strategies that human reverse engineers use, but without explicit instruction. The convergence is not mysterious — both face the same problem (exponential path space, limited attention budget) and the same constraints select similar solutions. But the path-dependent lock-in is a failure mode that human analysts also exhibit — and unlike human analysts, the LLM cannot recognize it as a bias. The implicit pattern is simultaneously a capability and a vulnerability.