Learn the new physics without forgetting the old.
Scientific surrogate models — trained to approximate expensive simulations — face a continual learning problem. The simulation conditions change (different geometries, different boundary conditions, different regimes). The original training data may be unavailable. The model must adapt to new distributions without losing its ability on previous ones (arXiv:2603.20410).
Note (2026-11-14; read against the paper's full text): the essay says zero forgetting is by construction and that the architecture methods eliminate the conflict that regularization methods suffer. The paper's zero forgetting depends on an out-of-distribution detector that identifies the task at inference, since each task gets its own extra layer and the frozen backbone plus the right layer must be selected; each new task adds between 1.5 and 4.4 percent of the total parameters, so the model grows with every task, and the paper says finite model capacity implicitly limits the number of tasks that can be accommodated. In its results the replay methods reach strong accuracy and the regularization methods are lighter but weaker (EWC ends at 0.771 average accuracy with mean forgetting 0.114; LwF falls to 0.565), and PiggyBack and LoRA are grouped with SLE-FNO as achieving the best retention. The comparison is on one dataset: 230 simulations of aneurysmal blood flow in four tasks. The architecture-versus-loss framing is the essay's own.
SLE-FNO: single-layer extensions for Fourier Neural Operators. When new data arrives, add a single new layer to the existing network rather than retraining the whole model. The existing layers are frozen — they can't change, so they can't forget. The new layer learns only the delta between the old model's predictions and the new data's targets. Zero forgetting by construction.
Tested against seven continual learning methods (EWC, LwF, replay, OGD, GEM, PiggyBack, LoRA) on pulsatile aneurysmal blood flow — mapping transient concentration fields to time-averaged wall shear stress across four sequential configurations. SLE-FNO achieves the best balance: strong accuracy on new data, zero forgetting of old data, minimal additional parameters.
The structural insight: the catastrophic forgetting problem is an architectural problem, not an optimization problem. Methods that prevent forgetting through regularization (EWC, LwF) make the optimization harder — they ask gradient descent to navigate between conflicting objectives. Methods that prevent forgetting through architecture (SLE-FNO, PiggyBack) eliminate the conflict — old knowledge is structurally protected. The optimization only needs to learn the new part. Separating old from new at the architecture level is simpler and more reliable than separating them at the loss level.