The power of experiments: How big is your n?

Igor Asanov (), Christoph Buehren () and Panagiota Zacharodimou ()
Additional contact information
Igor Asanov: University of Kassel
Christoph Buehren: Clausthal University of Technology
Panagiota Zacharodimou: European Parliament

MAGKS Papers on Economics from Philipps-Universität Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung)

Abstract: The replicability and credibility crisis in psychology and economics sparked the debate on underpowered experiments, publication biases, and p-hacking. Analyzing the number of independent observations of experiments published in Experimental Economics, Games and Economic Behavior, and the Journal of Economic Behavior and Organization, we observe that we did not learn much from this debate. The median experiment in our sample has too few independent observations and, thus, is underpowered. Moreover, we find indications for biases in reporting highly significant results. We investigate for which papers and experiments it is more likely to find reporting biases, and we suggest remedies that could help to overcome the replicability crisis.

Keywords: Statistical power; statistical significance; meta-study; balanced randomization; caliper test (search for similar items in EconPapers)
JEL-codes: C10 C12 C18 (search for similar items in EconPapers)
Pages: 32 pages
Date: 2020
New Economics Papers: this item is included in nep-cbe, nep-exp and nep-hpe
References: View references in EconPapers View complete reference list from CitEc
Citations:

Forthcoming in

Downloads: (external link)
https://www.uni-marburg.de/en/fb02/research-groups ... s/32-2020_asanov.pdf First 202032 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:mar:magkse:202032

Access Statistics for this paper

More papers in MAGKS Papers on Economics from Philipps-Universität Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung) Contact information at EDIRC.
Bibliographic data for series maintained by Bernd Hayo ().