Sparse mixture-of-experts transformers route each token to a subset of available expert networks. The routing mechanism is usually understood as load balancing — distributing computation to prevent bottlenecks. The question is whether routing carries semantic information beyond balancing.
Note (2026-11-14; read against the paper's full text): the figures are the paper's, but its evidence is one model (OLMoE-1B-7B) and 80 prompts, 20 in each of four categories (code, math, story and factual question answering), with a signature of expert usage frequencies across layers (1024 dimensions) and 5-fold cross-validation (92.5 percent, plus or minus 6.1). The essay says the routing decision is made on the input before any computation and that routing partitions the input space before processing begins. The paper's layer-wise analysis finds that task structure becomes increasingly apparent in deeper layers. The paper's conclusion is that routing is not merely a balancing mechanism but a measurable task-sensitive component, and that the separation exceeds its permutation and load-balancing baselines; the closing lines of the essay about a learned task taxonomy are the essay's.
Routing signatures — vector representations of expert activation patterns — are task-specific (arXiv:2603.11114). Prompts from the same task category produce routing signatures with high similarity (0.84 ± 0.09). Prompts from different categories produce lower similarity (0.62 ± 0.17). A logistic regression classifier achieves 92.5% accuracy on four-way task classification using routing signatures alone — no access to the token embeddings, no access to the output, just the pattern of which experts were activated.
The routing decision is made before the expert computation occurs. The model decides which experts to use based on the input, and this decision already encodes what kind of task the input represents. The task classification is a byproduct of the routing — it exists in the routing mechanism itself, not in the expert outputs.
This means routing is not merely load balancing. It is a form of conditional computation that partitions the input space into task-relevant regions before processing begins. Each task type has a characteristic fingerprint in expert activation space. The routing network has learned a task taxonomy as a side effect of learning to route efficiently.
The pattern of which experts activate is as informative as what the experts compute. The routing is the first stage of understanding, not the plumbing that precedes it.