Meta-analysis and publication bias: How well does the FAT-PET-PEESE procedure work? A replication study of Alinaghi & Reed (Research Synthesis Methods, 2018)

Sanghyun Hong

International Journal for Re-Views in Empirical Economics (IREE), 2019, vol. 3, 1-22

Abstract: A meta-analysis is a tool for aggregating estimates of a similar "effect" across many studies. Publication bias is the phenomenon where literature is sample selected in favor of studies having statistically significant results and/or having estimates that satisfy pre-conceived expectations. A popular procedure used for conducting meta-analyses in the presence of publication bias is the FAT-PET-PEESE (FPP) procedure. In a recent paper published in Research Synthesis Methods, Alinaghi and Reed (2018), utilizing Monte Carlo simulations, report that the FPP procedure does not work well when used in 'realistic' data environments where true effects differ both across and within studies. AR's findings are important because the FPP approach is dominant in the economics meta-analysis literature. I replicate their results and discover two mistakes, which I subsequently correct. The first mistake is found in a descriptive statistics table, misrepresenting the overview of simulated dataset. The second is associated with the fixed effect estimation, generating erroneous estimated effects and Type I error. Further, I extend their analysis by making their simulation environment even more realistic. Despite producing somewhat different results, my replications generally confirm AR's conclusions about the unreliability of the FPP procedure in realistic data environments.

Keywords: Meta-Analysis; Publication Bias; Funnel Asymmetry Test (FAT); Precision Effect Estimate with Standard Error (PEESE); Monte Carlo Simulations; Replication Study (search for similar items in EconPapers)
JEL-codes: B41 C15 C18 (search for similar items in EconPapers)
Date: 2019
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.econstor.eu/bitstream/10419/206820/1/IREE-2019-04.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zbw:ireejl:206820

DOI: 10.18718/81781.13

Access Statistics for this article

International Journal for Re-Views in Empirical Economics (IREE) is currently edited by David A. Jaeger, Maren Duvendack, W. Robert Reed

More articles in International Journal for Re-Views in Empirical Economics (IREE) from ZBW - Leibniz Information Centre for Economics Contact information at EDIRC.
Bibliographic data for series maintained by ZBW - Leibniz Information Centre for Economics ().