Training language models on domain-specific data improves performance in that domain — that's expected. Train on medical text, get better at medical questions. Train on legal documents, get better at legal reasoning. The transfer is within-domain.
Note (2026-11-15; read against the paper's full text): This paper (OpenSWE, titled daVinci-Env in the latest version) is mainly a software-engineering environment and training dataset (45,320 executable environments, about 1.47 million dollars of cost); the transfer result is a secondary finding. The paper's no-degradation claim is stated as not degrading factual recall, and its general-capability table (MMLU, MMLU-Pro, TriviaQA) shows MMLU flat for the 32B model, small gains on TriviaQA and larger gains on MMLU-Pro; it does not test common-sense reasoning, which the essay lists. The paper's own explanation is that the training practises procedural problem-solving also used in math, with the domain gap limiting the size of the gain; the essay's list of skills (state tracking, logical dependencies and so on) is the essay's elaboration.
The paper (arXiv:2603.13023, March 2026) trains models on software engineering tasks — resolving GitHub issues, writing patches, debugging code — and finds that the models gain up to 12 points on mathematical reasoning benchmarks and 5 points on science benchmarks. Training on code improves math. The transfer is cross-domain, and the direction is unexpected.
The mechanism is structural reasoning. Software engineering tasks require decomposing problems into steps, tracking state through transformations, identifying logical dependencies, and debugging inconsistencies. These are the same skills that mathematical reasoning demands — not the notation or the theorems, but the underlying logical infrastructure. Training on code exercises this infrastructure, and the exercise transfers.
The transfer doesn't sacrifice existing knowledge. The models improve on math and science without degrading on language understanding, common-sense reasoning, or other general capabilities. The code training adds a capability rather than trading one for another. This distinguishes it from most fine-tuning, where gains in one area come at the cost of losses in others.
The structural lesson: code is not a domain — it is a medium for structured reasoning. Training on code doesn't teach the model about software. It teaches the model about logical structure, sequential dependency, state tracking, and error identification. These are domain-general skills that happen to be exercised intensely by coding tasks. The code is the gym; the reasoning is the muscle. And the muscle works in any domain.