Optimal Experimental Design for Staggered Rollouts

Ruoxuan Xiong (), Susan Athey, Mohsen Bayati () and Guido Imbens ()
Additional contact information
Ruoxuan Xiong: Department of Quantitative Theory and Methods, Emory University, Atlanta, Georgia 30322
Mohsen Bayati: Graduate School of Business, Stanford University, Stanford, California 94305
Guido Imbens: Graduate School of Business, Stanford University, Stanford, California 94305

Management Science, 2024, vol. 70, issue 8, 5317-5336

Abstract: In this paper, we study the design and analysis of experiments conducted on a set of units over multiple time periods in which the starting time of the treatment may vary by unit. The design problem involves selecting an initial treatment time for each unit in order to most precisely estimate both the instantaneous and cumulative effects of the treatment. We first consider nonadaptive experiments, in which all treatment assignment decisions are made prior to the start of the experiment. For this case, we show that the optimization problem is generally NP-hard, and we propose a near-optimal solution. Under this solution, the fraction entering treatment each period is initially low, then high, and finally low again. Next, we study an adaptive experimental design problem, in which both the decision to continue the experiment and treatment assignment decisions are updated after each period’s data are collected. For the adaptive case, we propose a new algorithm, the precision-guided adaptive experiment algorithm, which addresses the challenges at both the design stage and the stage of estimating treatment effects, ensuring valid post-experiment inference, accounting for the adaptive nature of the design. Using realistic settings, we demonstrate that our proposed solutions can reduce the opportunity cost of the experiments by more than 50%, compared with static design benchmarks.

Keywords: instantaneous and cumulative effects; treatment effect estimation and inference; nonadaptive and adaptive experiments; panel data; dynamic programming (search for similar items in EconPapers)
Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/mnsc.2023.4928 (application/pdf)

Related works:
Working Paper: Optimal Experimental Design for Staggered Rollouts (2023)
Working Paper: Optimal Experimental Design for Staggered Rollouts (2019)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ormnsc:v:70:y:2024:i:8:p:5317-5336

Access Statistics for this article

More articles in Management Science from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().