“Garbage in, garbage out” is the oldest maxim in data science. Feed a model noisy data, get noisy predictions. Feed it biased data, get biased predictions. Data quality is the foundation; everything else is superstructure. Clean your data first — then model.
Note (2026-11-15; read against the paper's full text): the paper's central result is asymptotic: it proves that with infinitely many error-prone predictors (its 'infinite breadth' limit) both predictor error and structural uncertainty are overcome, whereas cleaning a low-dimensional set stays bounded by structural uncertainty, under conditions the paper states (predictors remain distinct when their generative pathways and error processes are conditionally independent). It also derives the limits imposed by systematic error regimes and says that traditional data-centric AI's focus on label cleaning remains powerful in some regimes. The essay's 'garbage in, gold out', its 'every measurement is wrong can still support correct predictions' and the choir analogy state the asymptotic result more broadly and in the essay's own words.
The paper (arXiv:2603.12288, March 2026) proves that in high-dimensional settings, error-prone datasets can yield robust predictions. The architecture of the data — how variables relate to each other structurally — matters more than the quality of individual measurements. Garbage in, gold out.
The mechanism is dimensionality-dependent averaging. In low dimensions, each measurement carries significant weight, and errors in individual measurements propagate directly to predictions. In high dimensions, the prediction depends on the aggregate structure of many measurements, and individual errors average out. The structural relationships between variables are preserved even when individual values are noisy, because the structure is encoded redundantly across many dimensions.
The condition: the data must have specific architectural properties — the dependencies between variables must form patterns that are robust to perturbation. Not all high-dimensional data qualifies. But when the architecture is right, the prediction extracts the structural signal from noisy components, just as a noisy choir can produce a recognizable melody even when no individual singer is on pitch.
The structural lesson: quality at the component level and quality at the system level are different properties that can be independently controlled in high dimensions. A dataset where every measurement is wrong can still support correct predictions, provided the errors are uncorrelated with the structure being predicted. The garbage is in the values; the gold is in the relationships between the values. And in high dimensions, relationships dominate values.