Do Typical RCTs of Education Interventions Have Sufficient Statistical Power for Linking Impacts on Teacher Practice and Student Achievement Outcomes?

Peter Z. Schochet

Mathematica Policy Research Reports from Mathematica Policy Research

Abstract: For randomized control trials (RCTs) of education interventions, it is often of interest to estimate associations between student and mediating teacher practice outcomes, to examine the extent to which a study’s conceptual model is supported by data, and to identify mediators most associated with student learning. This paper develops statistical power formulas for such exploratory analyses under clustered school-based RCTs using ordinary least squares (OLS) and instrumental variable (IV) estimators, and uses these formulas to conduct a simulated power analysis. The power analysis finds that for currently available mediators, the OLS approach will yield precise estimates of associations between teacher practice measures and student test score gains only if the sample contains about 150 to 200 study schools. The IV approach, which can adjust for potential omitted variable and simultaneity biases, has very little statistical power for mediator analyses. For typical RCT evaluations, these results may have design implications for the scope of the data collection effort for obtaining costly teacher practice mediators.

Keywords: RCTs; Statistical; Power; Teacher; Practice; Student; Achievement (search for similar items in EconPapers)
Pages: 46
Date: 2009-10-30
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://ies.ed.gov/ncee/pdf/20094065.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:mpr:mprres:8bb2ecd6a142422db269c1e0b9dec26f

Access Statistics for this paper

More papers in Mathematica Policy Research Reports from Mathematica Policy Research Mathematica Policy Research P.O. Box 2393 Princeton, NJ 08543-2393 Attn: Communications. Contact information at EDIRC.
Bibliographic data for series maintained by Joanne Pfleiderer () and Cindy George ( this e-mail address is bad, please contact ).