Performance of Bias-Detection Methods in Psychological Meta-Analysis: A Large-Scale Simulation Study

Sanghyun Hong (), W. Robert Reed (), R.C.m van Aert and M.A.L.m van Assen
Additional contact information
Sanghyun Hong: University of Canterbury, https://www.canterbury.ac.nz
W. Robert Reed: University of Canterbury, https://www.canterbury.ac.nz

Working Papers in Economics from University of Canterbury, Department of Economics and Finance

Abstract: Meta-analyses are routinely assessed for publication bias, yet the relative performance of bias-detection methods under realistic conditions remains unclear, particularly when bias may arise from both selective publication and questionable research practices (QRPs). We present a large-scale simulation study evaluating 21 commonly used biasdetection methods across 864 conditions varying in true effect size, effect size heterogeneity, number of studies, publication bias, and QRPs. Data were generated using the simulation framework of Carter et al. (2019), which jointly models publication bias and QRPs. Results show that heterogeneity is the primary determinant of method performance because it strongly influences false positive rates of many methods. When heterogeneity was low to moderate, TESS performed best overall. When heterogeneity was high, selection models performed best in small to medium meta-analyses, and caliper tests performed best in large meta-analyses. Precision–based methods, including variants of Egger’s test, performed poorly and often exhibited inflated false-positive rates under heterogeneity. These findings suggest that biasdetection methods should be selected based on heterogeneity and the number of studies rather than applied routinely, with TESS providing a strong general-purpose test across many settings.

JEL-codes: C12 C15 C18 C52 C87 (search for similar items in EconPapers)
Pages: 66 pages
Date: 2026-06-01
References: Add references at CitEc
Citations:

Downloads: (external link)
https://repec.canterbury.ac.nz/cbt/econwp/2603.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cbt:econwp:26/03

Access Statistics for this paper

More papers in Working Papers in Economics from University of Canterbury, Department of Economics and Finance Private Bag 4800, Christchurch, New Zealand. Contact information at EDIRC.
Bibliographic data for series maintained by Albert Yee ().