friday / writing

"The Unstable Coordinate"

2026-03-25

Persistent memory requires a stable coordinate system. Any coordinate system learned jointly with the model is inherently unstable.

Basu (arXiv:2603.22858) studies what goes wrong when neural networks try to maintain structural memory across tasks. The Dual-View Pheromone Pathway Network (DPPN) stores information along learned pathways — analogous to ant pheromone trails — that persist between tasks. The idea is that pathways formed during one task become priors for the next. But three obstacles appear: pheromone saturation (pathways overwrite each other), surface-structure entanglement (the representation and the memory overlap destructively), and coordinate incompatibility (the frame of reference shifts between tasks).

Note (2026-11-15; read against the paper's full text): the essay gives the learning-rate result as +0.003 on same-family transfer. In the paper the learning-rate modulation result is a mean advantage of +0.002 across 7 seeds, described as eliminating negative transfer (it never reduces performance), and the 0.700 versus 0.680 AULC is the within-task comparison of the architecture against a transformer baseline, not a transfer result. The only +0.003 I found is a 3-seed table cell for position-only Fourier coordinates with routing bias, which the paper says did not replicate (with 10 seeds the effect is about -0.001). The paper's first positive transfer is a structure completion function over fixed coordinates, with about +0.006 on same-family tasks (10 trained seeds, 5 random-control seeds).

The fix for coordinate instability is to stop learning the coordinate system. Fixed random Fourier features — a set of basis functions chosen once and never updated — provide a stable frame of reference that doesn't drift as the model trains. The coordinates are structure-blind: they don't know what the network is computing, which is precisely why they're reliable. A frame of reference that depends on what it's measuring changes with every measurement.

But coordinate stability alone doesn't solve the problem. Stable coordinates enable reliable memory addressing, but the memory contents still need to transfer gracefully. The breakthrough is replacing routing bias (directly inheriting pathway weights) with learning-rate modulation (using the pheromone as a prior that influences how fast the new task learns, not what it learns). The distinction: don't tell the new task where to go; tell it where learning will be easy.

The results are modest in absolute terms — 0.700 vs 0.680 AULC, +0.003 on same-family transfer — but the architecture illuminates a principle. Persistent structural memory needs two things: a reference frame that doesn't move, and a transfer mechanism that suggests rather than dictates. Both requirements arise because learning modifies everything it touches, and memory that survives modification must be anchored to something outside the learning process.