Design choices in randomization tests that affect power

Abba M. Krieger, David Azriel, Michael Sklar and Adam Kapelner

Communications in Statistics - Theory and Methods, 2024, vol. 53, issue 9, 3276-3291

Abstract: We consider the problem of evaluating designs for a small-sample two-arm randomized experiment using the power of the one-sided randomization test as a criterion. Our evaluation assumes a linear response in one observed covariate, an unobserved component and an additive treatment effect where the model's randomness is due to different treatment allocations. It is well-known that the power depends on the allocations’ imbalance in the observed covariate. We show that power is additionally affected by two other design choices: the number of allocations and the degree of the allocations’ dependence. We prove that the more allocations, the higher the power and the lower the variability in power. Our theoretical findings and simulation studies show that the designs with the highest power provide thousands of highly independent allocations each providing nominal imbalance in the observed covariates. These high-powered designs exhibit less randomness than complete randomization and more randomness than recently proposed designs that employ numerical optimization. This advantage of high power is easily accessible to practicing experimenters via the popular rerandomization design and a greedy pair switching design, where both outperform complete randomization and numerical optimization. The tradeoff we find also provides a means to specify rerandomization’s imbalance threshold parameter.

Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/03610926.2022.2152286 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:lstaxx:v:53:y:2024:i:9:p:3276-3291

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/lsta20

DOI: 10.1080/03610926.2022.2152286

Access Statistics for this article

Communications in Statistics - Theory and Methods is currently edited by Debbie Iscoe

More articles in Communications in Statistics - Theory and Methods from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().