The synthetic control method constructs a counterfactual by weighting untreated units so their pre-treatment trajectories match the treated unit. Over 5,000 applications in two decades. The method is widely used and, the authors argue, widely misunderstood.
Three accepted beliefs are tested and found unsupported. First: “synthetic control is robust to various implementations.” It is not. Different implementation choices (weighting scheme, matching variables, donor pool) produce substantially different estimates. The method is sensitive to decisions the analyst makes, which means different researchers analyzing the same intervention can reach different conclusions. Second: “covariates are unnecessary.” They are not unnecessary. Matching only on the outcome trajectory without conditioning on covariates loses information that matters for extrapolation. The pre-treatment match can be misleading without covariate balance. Third: “pre-treatment prediction error should guide model selection.” It should not, or at least not alone. Low pre-treatment error does not guarantee good post-treatment prediction — the model can overfit the pre-treatment match in ways that degrade the counterfactual.
Note (2026-11-11): checked against the paper's full text. The three misconceptions, the 5,000 uses in twenty years and the recommendation of more cautious practice match the paper. Three details differ. (1) The essay lists the implementation choices that change the estimates as the weighting scheme, the matching variables and the donor pool. The paper names the choice of algorithm for generating the weights and the choice of reference category for categorical or compositional covariates; I did not find donor-pool choice among its tested implementation choices (the donor pool in its simulations is the 49 other states). (2) The evidence is simulation, calibrated to data on Alaska (the Permanent Fund Dividend), and the abstract says the simulations suggest the beliefs are not supported; the essay states the findings as flatly established ('It is not', 'It should not'). (3) The paper's covariate argument, shown with a toy example, is that with few pre-treatment periods covariates can reduce overfitting by adding information; the essay's account, that matching only the outcome path loses information needed for extrapolation, is its own wording. On the third misconception the paper says that over-reliance on pre-treatment imbalance may lead to overfitting and worse estimates, since the implementations with the lowest imbalance are not necessarily those with the lowest bias.
The structural insight is about the gap between usage volume and understanding. The method's simplicity and flexibility made it easy to adopt. The same flexibility creates degrees of freedom that the typical application doesn't acknowledge. The three “truths” are not innocent simplifications — they are specific ways that flexibility converts to fragility when the analyst doesn't realize choices are being made.
(arXiv:2603.19211)