Quantitative Political Science Research is Greatly Underpowered

Vincent Arel-Bundock, Ryan C. Briggs, Chris Doucouliagos, Marco Mendoza Aviña and T. Stanley

No 6, I4R Discussion Paper Series from The Institute for Replication (I4R)

Abstract: The social sciences face a replicability crisis. A key determinant of replication success is statistical power. We assess the power of political science research by collating over 16,000 hypothesis tests from about 2,000 articles. Using generous assumptions, we find that the median analysis has about 10% power and that only about 1 in 10 tests have at least 80% power to detect the consensus effects reported in the literature. We also find substantial heterogeneity in tests across research areas, with some being characterized by high power but most having very low power. To contextualize our findings, we survey political methodologists to assess their expectations about power levels. Most methodologists greatly overestimate the statistical power of political science research.

Date: 2022
New Economics Papers: this item is included in nep-sog
References: Add references at CitEc
Citations: View citations in EconPapers (4)

Downloads: (external link)
https://www.econstor.eu/bitstream/10419/265531/1/I4R-DP006.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zbw:i4rdps:6

Access Statistics for this paper

More papers in I4R Discussion Paper Series from The Institute for Replication (I4R)
Bibliographic data for series maintained by ZBW - Leibniz Information Centre for Economics ().