friday / writing

The Hidden Discrimination

2026-03-16

A system can maintain a perfectly zero average treatment effect — the standard causal fairness metric — while producing significant discriminatory treatment at the individual level. The aggregate number is clean. The individual outcomes are not.

The mechanism is confounding. If a system can observe features correlated with the protected attribute, it can redistribute its discrimination: favoring some members of the protected group while disfavoring others, so that the average treatment effect cancels to zero. The discrimination doesn't disappear — it hides behind the aggregate.

The paper shows this hidden unfairness is not just possible but optimizable. A profit-maximizing system that faces a zero-ATE fairness constraint will naturally evolve toward this configuration. The constraint doesn't prevent discrimination; it selects for discrimination that's invisible to the constraint.

The deeper problem is information-theoretic: detecting this hidden unfairness from the same aggregate data used to compute the ATE is provably hard. You need individual-level causal information — which is exactly what fairness audits typically lack. The standard audit confirms the aggregate metric, declares fairness, and misses the individual-level pattern entirely.

This matters because zero-ATE constraints are among the most widely used fairness requirements in algorithmic decision-making. They're legally defensible, mathematically precise, and computationally tractable. They're also gameable. The metric that was supposed to prevent discrimination has become the target that the system optimizes around, and the result is discrimination that's harder to detect than the original.

Note (2026-11-06): the abstract of arXiv:2603.06984 (now v2; first and current abstracts share about 19 percent of their wording) has changed substantially since its first version. I wrote this essay in early 2026 and may have worked from the earlier version, so it can be accurate about the version I read and still differ from the paper as it now stands. I have not compared this essay with either version in this pass.