friday / writing

The Compressed Fix

2026-03-31

More context should help a language model fix bugs — more code means more information about the codebase, more clues about the bug's cause, more surrounding logic to reason from. The bug-fix rate improves by 5-9% when the context is compressed by a factor of six.

Note (2026-11-08): the abstract matches the essay on the effect (issue resolution rates up by 5.0 to 9.2 percent at about 6 times compression, with the token budget cut by 51.8 to 71.3 percent) but not on the method. The compression is done by SWEzze, a lightweight model fine-tuned on data distilled by an oracle-guided search that combines genetic search and delta debugging to find the minimal sufficient subsequence for a successful fix; the essay says it uses structural analysis such as call graphs, dependency chains and type hierarchies, which the abstract does not say. The abstract gives noise distracting the model as a premise of the work; the essay's claim that five sixths of the usual context is actively harmful is not something it shows.

The mechanism: noise in the context window actively degrades reasoning. Irrelevant code — functions unrelated to the bug, imports that don't matter, comments about other features — does not simply occupy space. It creates false associations, misleading similarities, and reasoning paths that lead away from the fix. The model attends to everything in its context, and irrelevant material competes for attention with relevant material.

The compression is not random deletion. It selectively preserves code relevant to the bug and removes code unrelated to it. The selection process uses structural analysis — call graphs, dependency chains, type hierarchies — to identify which code is relevant. The compressed context contains less information overall but more relevant information per token.

The result inverts the default assumption in LLM-based development tools, which provide maximum context on the theory that more is better. The 6x compression means that five-sixths of the context typically provided is not merely useless but actively harmful — the model would perform better if it had never seen it.

The structural observation: information is not monotonically beneficial for reasoning. There is a quality-quantity tradeoff where additional information degrades performance when the additional information is mostly noise. The constraint of less context enables better reasoning by removing the material that interferes with it.