Pooling Interactions Into Error Terms in Multisite Experiments

Wendy Chan and Larry Vernon Hedges
Additional contact information
Wendy Chan: University of Pennsylvania
Larry Vernon Hedges: Northwestern University

Journal of Educational and Behavioral Statistics, 2022, vol. 47, issue 6, 639-665

Abstract: Multisite field experiments using the (generalized) randomized block design that assign treatments to individuals within sites are common in education and the social sciences. Under this design, there are two possible estimands of interest and they differ based on whether sites or blocks have fixed or random effects. When the average treatment effect is assumed to be identical across sites, it is common to omit site by treatment interactions and “pool†them into the error term in classical experimental design. However, prior work has not addressed the consequences of pooling when site by treatment interactions are not zero. This study assesses the impact of pooling on inference in the presence of nonzero site by treatment interactions. We derive the small sample distributions of the test statistics for treatment effects under pooling and illustrate the impacts on rejection rates when interactions are not zero. We use the results to offer recommendations to researchers conducting studies based on the multisite design.

Keywords: multisite experiments; randomized block designs; pooling; ANOVA; interactions (search for similar items in EconPapers)
Date: 2022
References: Add references at CitEc
Citations:

Downloads: (external link)
https://journals.sagepub.com/doi/10.3102/10769986221104800 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:jedbes:v:47:y:2022:i:6:p:639-665

DOI: 10.3102/10769986221104800

Access Statistics for this article

More articles in Journal of Educational and Behavioral Statistics
Bibliographic data for series maintained by SAGE Publications ().