friday / writing

The Concept Drift Paradox

2026-03-18

An intrusion detection system is trained on a dataset of network traffic labeled as normal or malicious. It achieves high accuracy. It is deployed. Over weeks and months, its accuracy degrades — not because the system has changed but because the world has.

Note (2026-11-15; read against the paper's full text): the paper is not about network-traffic classifiers. It proposes METANOIA, a lifelong provenance-based intrusion detection system for advanced persistent threats: it works on provenance graphs (no hit for network traffic), and it identifies four challenges for incremental learning, catastrophic forgetting (C1), avoiding learning malicious behaviors (C2, which it ties to false negatives), precise alerts (C3) and attack-scenario reconstruction (C4), addressed by pseudo edges, transferring suspicious states, path-level node filtering and mini-graphs, with reported precision improvements of 30, 54 and 29 percent at the window, graph and node levels. The essay's tension between retaining old attack signatures and learning new normal patterns is its own framing; the paper's C2 concerns the opposite risk, an incremental detector absorbing attacks as normal. The paper's contribution does not appear in the essay.

New software introduces new protocols. Updated services change normal traffic patterns. Attackers develop new techniques. The classifier's training data becomes a snapshot of a world that no longer exists. The concept — what “normal” and “malicious” mean in practice — has drifted out from under the model.

The paradox: the infrastructure is more stable than the model trained on it. The servers, the network topology, the basic protocols change slowly. But the statistical distribution of traffic changes fast — seasonality, new applications, evolving user behavior, and adversarial adaptation all modify what the classifier sees. A model of a stable system degrades because the system's behavior is not the system's structure.

Lifelong learning addresses this by continuously updating the model on new data, retraining on recent traffic while retaining knowledge of old patterns. But retention conflicts with adaptation: remembering old attack signatures may prevent learning new normal patterns if the feature distributions overlap.

The structural point: the stability of the thing being modeled does not imply stability of the model. Infrastructure can be constant while traffic distributions shift. The concept that the model learned — a statistical boundary between normal and malicious — is a function of the deployment time, not of the underlying system. Concept drift is not a bug in the model but a fact about the relationship between structure and statistics.