The influence of hidden researcher decisions in applied microeconomics

Nick Huntington‐Klein, Andreu Arenas, Emily Beam, Marco Bertoni, Jeffrey Bloem, Pralhad Burli, Naibin Chen, Paul Grieco, Godwin Ekpe, Todd Pugatch, Martin Saavedra () and Yaniv Stopnitzky
Authors registered in the RePEc Author Service: Nick Huntington-Klein

Economic Inquiry, 2021, vol. 59, issue 3, 944-960

Abstract: Researchers make hundreds of decisions about data collection, preparation, and analysis in their research. We use a many‐analysts approach to measure the extent and impact of these decisions. Two published causal empirical results are replicated by seven replicators each. We find large differences in data preparation and analysis decisions, many of which would not likely be reported in a publication. No two replicators reported the same sample size. Statistical significance varied across replications, and for one of the studies the effect's sign varied as well. The standard deviation of estimates across replications was 3–4 times the mean reported standard error.

Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (42)

Downloads: (external link)
https://doi.org/10.1111/ecin.12992

Related works:
Working Paper: The Influence of Hidden Researcher Decisions in Applied Microeconomics (2020)
Working Paper: The Influence of Hidden Researcher Decisions in Applied Microeconomics (2020)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:ecinqu:v:59:y:2021:i:3:p:944-960

Ordering information: This journal article can be ordered from
https://ordering.onl ... s.aspx?ref=1465-7295

Access Statistics for this article

Economic Inquiry is currently edited by Tim Salmon

More articles in Economic Inquiry from Western Economic Association International Contact information at EDIRC.
Bibliographic data for series maintained by Wiley Content Delivery ().