Multi-armed Bandit Experimental Design: Online Decision-Making and Adaptive Inference

David Simchi-Levi () and Chonghuan Wang ()
Additional contact information
David Simchi-Levi: Institute for Data, Systems, and Society, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139; and Department of Civil and Environmental Engineering, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139; and Operations Research Center, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139
Chonghuan Wang: Department of Civil and Environmental Engineering, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139; and Center for Computational Science and Engineering, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139

Management Science, 2025, vol. 71, issue 6, 4828-4846

Abstract: Multi-armed bandit has been well known for its efficiency in online decision-making in terms of minimizing the loss of the participants’ welfare during experiments (i.e., the regret). In clinical trials and many other scenarios, the statistical power of inferring the treatment effects (i.e., the gaps between the mean outcomes of different arms) is also crucial. Nevertheless, minimizing the regret entails harming the statistical power of estimating the treatment effect because the observations from some arms can be limited. In this paper, we investigate the trade-off between efficiency and statistical power by casting the multi-armed bandit experimental design into a minimax multi-objective optimization problem . We introduce the concept of Pareto optimality to mathematically characterize the situation in which neither the statistical power nor the efficiency can be improved without degrading the other. We derive a useful sufficient and necessary condition for the Pareto optimal solutions to the minimax multi-objective optimization problem. Additionally, we design an effective Pareto optimal multi-armed bandit experiment that can be tailored to different levels of the trade-off between the two objectives. Moreover, we extend the design and analysis to the setting where the outcome of each arm consists of an adversarial baseline reward and a stochastic treatment effect, demonstrating the robustness of our design. Finally, motivated by clinical trials, we examine the setting where the employed experiment must split the experimental units into a small number of batches, and we propose a flexible Pareto optimal design.

Keywords: adaptive experimental design; multi-armed bandit; minimax multi-objective optimization; Pareto optimality (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/mnsc.2023.00492 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ormnsc:v:71:y:2025:i:6:p:4828-4846

Access Statistics for this article

More articles in Management Science from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().