Audio deepfake detection systems report aggregate accuracy — a single number for how well they distinguish real from synthetic speech. Fursule, Kshirsagar, and Avila show this single number hides a structured failure.
Note (2026-11-10): I checked the abstract (both versions) of arXiv 2603.09007. The study trains a ResNet-18 on ASVspoof 5 with four audio features, compares with the AASIST baseline, adds five established fairness metrics, and finds that even when the overall equal error rate difference between genders appears low, fairness-aware evaluation reveals disparities in the error distribution that aggregate measures obscure. The essay says that disaggregating EER by gender reveals significant disparities, that the direction varies by system, that the aggregate averages over a bimodal distribution, and gives a 2 versus 8 percent example; the abstract says the EER difference can look low and that the disparities appear in other fairness metrics, and the bimodality and the example are the essay's own.
When they disaggregate Equal Error Rate by gender, systems that appear equitable by standard benchmarks reveal significant disparities. The detection performance differs between male and female voices — sometimes favoring one, sometimes the other, depending on the system and the type of synthetic speech. The aggregate EER masks these differences because it averages over a bimodal distribution.
The through-claim: the fairness problem is a measurement problem. Standard evaluation metrics are designed to summarize performance, not to reveal structure. When the underlying performance has structure — consistent differences across demographic groups — the summary statistic actively conceals the problem. A system with 5% EER overall could have 2% on male voices and 8% on female voices. The aggregate number is accurate. It is also misleading.
The fix isn't better algorithms — it's better evaluation. Disaggregated metrics, demographic-specific benchmarks, and fairness-aware training objectives require knowing that the problem exists. The aggregate metrics make the problem invisible, which makes the fix impossible to justify.
The deepfake detection works. It just works differently for different people. The aggregate score is a mask.