Inference in Experiments with Matched Pairs

Azeem Shaikh and Joseph P. Romano
Additional contact information
Joseph P. Romano: Institute for Fiscal Studies

No CWP19/19, CeMMAP working papers from Centre for Microdata Methods and Practice, Institute for Fiscal Studies

Abstract: This paper studies inference for the average treatment effect in randomized controlled trials where treatment status is determined according to a "matched pairs" design. By a "matched pairs" design, we mean that units are sampled i.i.d. from the population of interest, paired according to observed, baseline covariates and fi nally, within each pair, one unit is selected at random for treatment. This type of design is used routinely throughout the sciences, but results about its implications for inference about the average treatment effect are not available. The main requirement underlying our analysis is that pairs are formed so that units within pairs are suitably "close" in terms of the baseline covariates, and we develop novel results to ensure that pairs are formed in a way that satis es this condition. Under this assumption, we show that, for the problem of testing the null hypothesis that the average treatment effect equals a pre-speci ed value in such settings, the commonly used two-sample t-test and "matched pairs" t-test are conservative in the sense that these tests have limiting rejection probability under the null hypothesis no greater than and typically strictly less than the nominal level. We show, however, that a simple adjustment to the standard errors of these tests leads to a test that is asymptotically exact in the sense that its limiting rejection probability under the null hypothesis equals the nominal level. We also study the behavior of randomization tests that arise naturally in these types of settings. When implemented appropriately, we show that this approach also leads to a test that is asymptotically exact in the sense described previously, but additionally has fi nite-sample rejection probability no greater than the nominal level for certain distributions satisfying the null hypothesis. A simulation study con rms the practical relevance of our theoretical results.

Keywords: Experiment; matched pairs; matched pairs t-test; permutation test; randomized controlled trial; treatment assignment; two-sample t-test (search for similar items in EconPapers)
Date: 2019-04-25
New Economics Papers: this item is included in nep-ecm and nep-exp
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.ifs.org.uk/uploads/Inference%20in%20Ex ... %20Pairs_CWP1919.pdf (application/pdf)
Our link check indicates that this URL is bad, the error code is: 404 Not Found (https://www.ifs.org.uk/uploads/Inference%20in%20Experiments%20with%20Matched%20Pairs_CWP1919.pdf [302 Found]--> https://ifs.org.uk/uploads/Inference%20in%20Experiments%20with%20Matched%20Pairs_CWP1919.pdf)

Related works:
Journal Article: Inference in Experiments With Matched Pairs (2022)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ifs:cemmap:19/19

Ordering information: This working paper can be ordered from
The Institute for Fiscal Studies 7 Ridgmount Street LONDON WC1E 7AE

Access Statistics for this paper

More papers in CeMMAP working papers from Centre for Microdata Methods and Practice, Institute for Fiscal Studies The Institute for Fiscal Studies 7 Ridgmount Street LONDON WC1E 7AE. Contact information at EDIRC.
Bibliographic data for series maintained by Emma Hyman ().