On the Hedges Correction for a t-Test

Nathan M. VanHoudnos and Joel B. Greenhouse
Additional contact information
Nathan M. VanHoudnos: Northwestern University
Joel B. Greenhouse: Carnegie Mellon University

Journal of Educational and Behavioral Statistics, 2016, vol. 41, issue 4, 392-419

Abstract: When cluster randomized experiments are analyzed as if units were independent, test statistics for treatment effects can be anticonservative. Hedges proposed a correction for such tests by scaling them to control their Type I error rate. This article generalizes the Hedges correction from a posttest-only experimental design to more common designs used in practice. We show that for many experimental designs, the generalized correction controls its Type I error while the Hedges correction does not. The generalized correction, however, necessarily has low power due to its control of the Type I error. Our results imply that using the Hedges correction as prescribed, for example, by the What Works Clearinghouse can lead to incorrect inferences and has important implications for evidence-based education.

Keywords: model misspecification; cluster randomized trials; multilevel models; meta-analysis (search for similar items in EconPapers)
Date: 2016
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.sagepub.com/doi/10.3102/1076998616644990 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:jedbes:v:41:y:2016:i:4:p:392-419

DOI: 10.3102/1076998616644990

Access Statistics for this article

More articles in Journal of Educational and Behavioral Statistics
Bibliographic data for series maintained by SAGE Publications ().