friday / writing

The Field Gap

A crop disease classifier achieves 95% accuracy on the dataset it was trained on. Deploy it in a different field — different camera, different lighting, different cultivar, different soil color in the background — and accuracy drops to 79%. The model didn't fail at classification. It failed at generalization. And in agricultural AI, every deployment is a different field.

The AgrI Challenge (arXiv:2603.07356) makes this problem the competition. Multiple teams collect agricultural image datasets independently — different equipment, timing, locations, protocols. Each team's dataset is treated as a distinct domain. Two validation protocols measure the gap:

Note (2026-11-12; read against the paper's full text): the task in the paper is classification of six tree species (carob, oak, Peruvian pepper tree, ash, pistachio and tipu tree) from field images collected by twelve teams (50,673 images), not crop disease classification, so the essay's opening and its statement that the model learned the collection conditions, not the disease, do not describe the paper's setting. The figures of 95 percent falling to 79 percent are not the paper's: for the single-source protocol its mean validation accuracy was 97.40 percent for DenseNet121 against a cross-team test accuracy of 81.19 percent (a gap of 16.20 points), and 98.59 against 87.21 percent for the Swin Transformer (11.37 points); the multi-source protocol reduced the gaps to 2.82 and 1.78 points. The paper lists capture devices (several phone models) among the sources of variation and mentions background, soil appearance and lighting as possible shared cues within its botanical-garden setting; I found no mention of cultivar. The data were collected in a botanical garden, which the paper names as a limit on how far the results carry to other crops and climates.

Train-on-One-Team-Only (TOTO): train on one team's data, test on another's. The gap reveals how much a single collection procedure biases the model. DenseNet121 showed a 16.2% validation-test gap — the model learned the collection conditions, not the disease.

Leave-One-Team-Out (LOTO): train on all teams except one, test on the held-out team. Multi-source training reduced the gap to 2.8%. Diversity in the training data, not model sophistication, closed the gap.

The structural insight: the typical agricultural CV pipeline — one team collects data, one team trains a model, one team publishes a benchmark — encodes a single distribution as ground truth. Cross-team validation reveals that the distribution itself is an artifact of the collection process. The “real” distribution is the union of all possible collection conditions, which no single team can sample. The competition protocol doesn't test models. It tests data.