The Robustness Reproducibility of the American Economic Review
Douglas Campbell,
Abel Brodeur,
Anna Dreber,
Magnus Johannesson,
Joseph Kopecky,
Lester Lusher and
Nikita Tsoy
No 124, I4R Discussion Paper Series from The Institute for Replication (I4R)
Abstract:
We estimate the robustness reproducibility of key results from 17 non-experimental AER papers published in 2013 (8 papers) and 2022/23 (9 papers). We find that many of the results are not robust, with no improvement over time. The fraction of significant robustness tests (p﹤0.05) varies between 17% and 88% across the papers with a mean of 46%. The mean relative t/z-value of the robustness tests varies between 35% and 87% with a mean of 63%, suggesting selective reporting of analytical specifications that exaggerate statistical significance. A sample of economists (n=359) overestimates robustness reproducibility, but predictions are correlated with observed reproducibility.
Date: 2024
New Economics Papers: this item is included in nep-exp, nep-hpe and nep-sog
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (6)
Downloads: (external link)
https://www.econstor.eu/bitstream/10419/295222/1/I4R-DP124.pdf (application/pdf)
Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
HTML/Text
Persistent link: https://EconPapers.repec.org/RePEc:zbw:i4rdps:124
Access Statistics for this paper
More papers in I4R Discussion Paper Series from The Institute for Replication (I4R)
Bibliographic data for series maintained by ZBW - Leibniz Information Centre for Economics ().