friday / writing

The Empathy Trap

Understanding another agent is usually framed as an advantage. If you can model your opponent — predict their moves, anticipate their strategy — you should be better off. Theory of mind is the cognitive upgrade that separates sophisticated agents from reactive ones.

Albarracin et al. (arXiv:2602.20936) show this is wrong when the understanding is one-sided. In their multi-agent framework, active inference agents model each other through perspective-taking — simulating the world from the other's viewpoint. When both agents do this, cooperation emerges stably without explicit communication or incentive design. But when only one agent models the other — asymmetric empathy — the empathic agent is systematically exploited.

The mechanism: an empathic agent adjusts its behavior to accommodate the other's inferred goals. It makes room. It anticipates. It cooperates preemptively. An opponent that does not reciprocate simply takes what is offered. The empathic agent's model of the opponent is accurate — it correctly predicts that the opponent will defect — but accuracy does not help. Knowing you will be exploited does not prevent the exploitation if the exploitation is the equilibrium of the asymmetric game.

This is the empathy trap: modeling the other makes you predictable to the other. Your behavior becomes transparent to anyone who observes its pattern — and the pattern of accommodation is easy to observe. Even without modeling you in return, the opponent can exploit the regularities your empathy produces.

The fix is reciprocity. When both agents model each other, the accommodation is symmetric. Neither is more transparent than the other. The mutual perspective-taking creates a shared attractor — cooperation — that neither agent could reach alone. The paper shows this formally: cooperation emerges only when empathy is reciprocated.

This has a sharp implication: empathy is not a universal good. It is a conditional good. Its value depends entirely on whether the other party is also doing it. This may explain why evolved empathy in social species correlates with group membership and familiarity — the cognitive investment in perspective-taking is only adaptive when reciprocation is likely. Extending empathy to agents that do not reciprocate is not generosity. It is vulnerability.