On mitigating the analytical limitations of finely stratified experiments

Colin B. Fogarty

Journal of the Royal Statistical Society Series B, 2018, vol. 80, issue 5, 1035-1056

Abstract: Although attractive from a theoretical perspective, finely stratified experiments such as paired designs suffer from certain analytical limitations that are not present in block‐randomized experiments with multiple treated and control individuals in each block. In short, when using a weighted difference in means to estimate the sample average treatment effect, the traditional variance estimator in a paired experiment is conservative unless the pairwise average treatment effects are constant across pairs; however, in more coarsely stratified experiments, the corresponding variance estimator is unbiased if treatment effects are constant within blocks, even if they vary across blocks. Using insights from classical least squares theory, we present an improved variance estimator that is appropriate in finely stratified experiments. The variance estimator remains conservative in expectation but is asymptotically no more conservative than the classical estimator and can be considerably less conservative. The magnitude of the improvement depends on the extent to which effect heterogeneity can be explained by observed covariates. Aided by this estimator, a new test for the null hypothesis of a constant treatment effect is proposed. These findings extend to some, but not all, superpopulation models, depending on whether the covariates are viewed as fixed across samples.

Date: 2018
References: Add references at CitEc
Citations: View citations in EconPapers (9)

Downloads: (external link)
https://doi.org/10.1111/rssb.12290

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:jorssb:v:80:y:2018:i:5:p:1035-1056

Ordering information: This journal article can be ordered from
http://ordering.onli ... 1111/(ISSN)1467-9868

Access Statistics for this article

Journal of the Royal Statistical Society Series B is currently edited by P. Fryzlewicz and I. Van Keilegom

More articles in Journal of the Royal Statistical Society Series B from Royal Statistical Society Contact information at EDIRC.
Bibliographic data for series maintained by Wiley Content Delivery ().