The Influence of Hidden Researcher Decisions in Applied Microeconomics
Nick Huntington-Klein (),
Andreu Arenas (),
Marco Bertoni (),
Jeffrey R. Bloem,
Martin Saavedra () and
No 537, GLO Discussion Paper Series from Global Labor Organization (GLO)
Researchers make hundreds of decisions about data collection, preparation, and analysis in their research. We use a many-analysts approach to measure the extent and impact of these decisions. Two published causal empirical results are replicated by seven replicators each. We find large differences in data preparation and analysis decisions, many of which would not likely be reported in a publication. No two replicators reported the same sample size. Statistical significance varied across replications, and for one of the studies the effect’s sign varied as well. The standard deviation of estimates across replications was 3-4 times the typical reported standard error.
Keywords: Replication; Metascience; Research (search for similar items in EconPapers)
JEL-codes: C81 C10 B41 (search for similar items in EconPapers)
New Economics Papers: this item is included in nep-ecm
References: View references in EconPapers View complete reference list from CitEc
Citations: Track citations by RSS feed
Downloads: (external link)
Working Paper: The Influence of Hidden Researcher Decisions in Applied Microeconomics (2020)
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
Persistent link: https://EconPapers.repec.org/RePEc:zbw:glodps:537
Access Statistics for this paper
More papers in GLO Discussion Paper Series from Global Labor Organization (GLO) Contact information at EDIRC.
Bibliographic data for series maintained by ZBW - Leibniz Information Centre for Economics ().