Diffusion Approximations for a Class of Sequential Experimentation Problems

Victor F. Araman () and René A. Caldentey ()
Additional contact information
Victor F. Araman: Olayan School of Business, American University of Beirut, Beirut 1107 2020, Lebanon
René A. Caldentey: Booth School of Business, The University of Chicago, Chicago, Illinois 60637

Management Science, 2022, vol. 68, issue 8, 5958-5979

Abstract: A decision maker (DM) must choose an action in order to maximize a reward function that depends on the DM’s action as well as on an unknown parameter Θ. The DM can delay taking the action in order to experiment and gather additional information on Θ. We model the problem using a Bayesian sequential experimentation framework and use dynamic programming and diffusion-asymptotic analysis to solve it. For that, we consider environments in which the average number of experiments that is conducted per unit of time is large and the informativeness of each individual experiment is low. Under such regimes, we derive a diffusion approximation for the sequential experimentation problem, which provides a number of important insights about the nature of the problem and its solution. First, it reveals that the problems of (i) selecting the optimal sequence of experiments to use and (ii) deciding the optimal time when to stop experimenting decouple and can be solved independently. Second, it shows that an optimal experimentation policy is one that chooses the experiment that maximizes the instantaneous volatility of the belief process. Third, the diffusion approximation provides a more mathematically malleable formulation that we can solve in closed form and suggests efficient heuristics for the nonasympototic regime. Our solution method also shows that the complexity of the problem grows only quadratically with the cardinality of the set of actions from which the decision maker can choose. We illustrate our methodology and results using a concrete application in the context of assortment selection and new product introduction. Specifically, we study the problem of a seller who wants to select an optimal assortment of products to launch into the marketplace and is uncertain about consumers’ preferences. Motivated by emerging practices in e-commerce, we assume that the seller is able to use a crowd voting system to learn these preferences before a final assortment decision is made. In this context, we undertake an extensive numerical analysis to assess the value of learning and demonstrate the effectiveness and robustness of the heuristics derived from the diffusion approximation.

Keywords: sequential testing; experimentation; Bayesian demand learning; experiment design; optimal stopping; dynamic programming; crowdvoting (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://dx.doi.org/10.1287/mnsc.2021.4195 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ormnsc:v:68:y:2022:i:8:p:5958-5979

Access Statistics for this article

More articles in Management Science from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().