When a foundation model trained on single-cell gene expression data (scGPT, Geneformer) processes a cell's transcriptome, it produces internal representations — high-dimensional vectors for each gene. These representations live in a geometric space. The space has structure: distances, neighborhoods, topology. The question is whether this structure is biological or computational. Does the model's internal geometry reflect the actual organization of gene regulation, or is it an artifact of the training process — attention patterns, tokenization, and gradient descent producing structure that looks meaningful but encodes nothing about biology?
Note (2026-11-12): checked against the paper's full text (Kendiukhov). The three headline findings and the numbers match: persistent homology significant against feature-shuffle nulls in 11 or 12 of 12 transformer layers, canonical correlation of 0.80 and 72 percent gene retrieval between scGPT and Geneformer with no method among 19 tested recovering gene-level correspondences, and robust signal concentrated in immune tissue under the strictest audit while lung becomes fragile. The essay's first finding is stronger than the paper's. Under degree-preserving rewiring of the nearest-neighbor graph the topological signal vanished completely (0 of 24 layer tests significant, across six progressively refined controls), which the paper lists among its key negative findings and reads as meaning that the topology is not a deeper geometric invariant but comes from the specific pattern of which genes are each other's nearest neighbors. The essay mentions only the shuffled controls, and its sentence that the topology 'reflects something about the relationships between genes' is not what that null supports.
Kendiukhov (arXiv 2602.22289, February 2026) tests this with 141 hypotheses, screened through 52 iterations of an AI-driven experimental loop, using persistent homology, manifold geometry, cross-model alignment, community structure, and directed topology — all with explicit null controls.
Three findings survive the controls. First: the models learn genuine geometric structure. Gene embedding neighborhoods have non-trivial persistent homology — topological features (loops, voids) that persist across scales and are absent in shuffled controls. The topology is not random. It reflects something about the relationships between genes that the model has extracted from expression patterns.
Second: the structure is shared across independently trained models. scGPT and Geneformer, trained on different data with different architectures, produce representations with high canonical correlation — the global geometry agrees. But the agreement breaks down at the gene level. Individual genes don't map consistently between the two models. The models converge on the same large-scale landscape but populate it with different features. The map agrees; the labels don't.
Third: the signal concentrates in immune tissue. Under the most stringent null controls — where the bar for declaring a result biologically meaningful is highest — immune cells and their gene programs show robust topological and geometric signal. Other tissue types show weaker or absent signal. The immune system, with its dramatic cell-type transitions, sharp expression programs, and well-characterized regulatory networks, provides the clearest training signal. The model learns best where the biology is most structured.
The result is neither a vindication nor a dismissal of biological foundation models. The structure they learn is partially real — it captures topological features of gene regulation that are reproduced across independent models. But it's concentrated in the biological domains where the training data provides the strongest signal, and it fragments at the level of individual genes. The models learn the topology of the territory but draw the map in different coordinates.