Meta-analyzing non-preregistered and preregistered studies

Robbie Cornelis Maria van Aert

No 2bj85, MetaArXiv from Center for Open Science

Abstract: Preregistration is gaining ground in psychology, and a consequence of this is that preregistered studies are more often included in meta-analyses. Preregistered studies mitigate the effect of publication bias in a meta-analysis, because preregistered studies can be located in the registries they were registered in even if they do not get published. However, current meta-analysis methods do not take into account that preregistered studies are less susceptible to publication bias. Traditional methods treat all studies as equivalent while meta-analytic conclusions can be improved by taking advantage of preregistered studies. The goal of this paper is to introduce a new method, the Hybrid Extended Meta-Analysis (HYEMA) method, that takes into account whether a study is preregistered or not to correct for publication bias in only the non-preregistered studies. The proposed method is applied to two meta-analyses on prominent effects in the psychological literature: the red-romance hypothesis and money priming. Applying HYEMA to these meta-analyses shows that the average effect size is substantially closer to zero than of the random-effects meta-analysis model. Two simulation studies tailored to the two applications are also presented to illustrate the method's superior performance compared to the random-effects meta-analysis model when publication bias is present. Hence, I recommend to always apply HYEMA as a sensitivity analysis if a mix of both preregistered and non-preregistered studies are present in a meta-analysis. Software is also developed and described in the paper to facilitate application of the method.

Date: 2023-11-27
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://osf.io/download/656076fb932b9f132e760892/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:metaar:2bj85

DOI: 10.31219/osf.io/2bj85

Access Statistics for this paper

More papers in MetaArXiv from Center for Open Science
Bibliographic data for series maintained by OSF ().