How Much Should We Trust Differences-In-Differences Estimates?

Marianne Bertrand, Esther Duflo and Sendhil Mullainathan

The Quarterly Journal of Economics, 2004, vol. 119, issue 1, 249-275

Abstract: Most papers that employ Differences-in-Differences estimation (DD) use many years of data and focus on serially correlated outcomes but ignore that the resulting standard errors are inconsistent. To illustrate the severity of this issue, we randomly generate placebo laws in state-level data on female wages from the Current Population Survey. For each law, we use OLS to compute the DD estimate of its "effect" as well as the standard error of this estimate. These conventional DD standard errors severely understate the standard deviation of the estimators: we find an "effect" significant at the 5 percent level for up to 45 percent of the placebo interventions. We use Monte Carlo simulations to investigate how well existing methods help solve this problem. Econometric corrections that place a specific parametric form on the time-series process do not perform well. Bootstrap (taking into account the autocorrelation of the data) works well when the number of states is large enough. Two corrections based on asymptotic approximation of the variance-covariance matrix work well for moderate numbers of states and one correction that collapses the time series information into a "pre"- and "post"-period and explicitly takes into account the effective sample size works well even for small numbers of states.

Date: 2004
References: Add references at CitEc
Citations: View citations in EconPapers (5794)

Downloads: (external link)
http://hdl.handle.net/10.1162/003355304772839588 (application/pdf)
Access to full text is restricted to subscribers.

Related works:
Working Paper: How Much Should We Trust Differences-in-Differences Estimates? (2002)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:oup:qjecon:v:119:y:2004:i:1:p:249-275.

Ordering information: This journal article can be ordered from
https://academic.oup.com/journals

Access Statistics for this article

The Quarterly Journal of Economics is currently edited by Robert J. Barro, Lawrence F. Katz, Nathan Nunn, Andrei Shleifer and Stefanie Stantcheva

More articles in The Quarterly Journal of Economics from President and Fellows of Harvard College
Bibliographic data for series maintained by Oxford University Press ().