Alternatives for Mixed-Effects Meta-Regression Models in the Reliability Generalization Approach

José Antonio López-López, Juan Botella, Julio Sánchez-Meca and Fulgencio Marín-Martínez
Additional contact information
José Antonio López-López: University of Murcia, Spain
Juan Botella: Autonomous University of Madrid, Spain
Julio Sánchez-Meca: University of Murcia, Spain

Journal of Educational and Behavioral Statistics, 2013, vol. 38, issue 5, 443-469

Abstract: Since heterogeneity between reliability coefficients is usually found in reliability generalization studies, moderator analyses constitute a crucial step for that meta-analytic approach. In this study, different procedures for conducting mixed-effects meta-regression analyses were compared. Specifically, four transformation methods for the reliability coefficients, two estimators of the residual between-studies variance, and two methods for testing regression coefficients significance were combined in a Monte Carlo simulation study. The different methods were compared in terms of bias and mean square error (MSE) of the slope estimates, and Type I error and statistical power rates for the slope statistical tests. The results of the simulation study did not vary as a function of the residual variance estimator. All transformation methods provided negatively biased estimates, but both bias and MSE were reasonably small in all cases. In contrast, important differences were found regarding statistical tests, with the method proposed by Knapp and Hartung showing a better adjustment to the nominal significance level and higher power rates than the standard method.

Keywords: reliability; reliability generalization; meta-analysis; coefficient alpha; mixed-effects meta-regression (search for similar items in EconPapers)
Date: 2013
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://journals.sagepub.com/doi/10.3102/1076998612466142 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:jedbes:v:38:y:2013:i:5:p:443-469

DOI: 10.3102/1076998612466142

Access Statistics for this article

More articles in Journal of Educational and Behavioral Statistics
Bibliographic data for series maintained by SAGE Publications ().