Actor-Critic–Like Stochastic Adaptive Search for Continuous Simulation Optimization

Qi Zhang () and Jiaqiao Hu ()
Additional contact information
Qi Zhang: Department of Applied Mathematics and Statistics, State University of New York at Stony Brook, Stony Brook, New York 11794
Jiaqiao Hu: Department of Applied Mathematics and Statistics, State University of New York at Stony Brook, Stony Brook, New York 11794

Operations Research, 2022, vol. 70, issue 6, 3519-3537

Abstract: We propose a random search method for solving a class of simulation optimization problems with Lipschitz continuity properties. The algorithm samples candidate solutions from a parameterized probability distribution over the solution space and estimates the performance of the sampled points through an asynchronous learning procedure based on the so-called shrinking ball method. A distinctive feature of the algorithm is that it fully retains the previous simulation information and incorporates an approximation architecture to exploit knowledge of the objective function in searching for improved solutions. Each step of the algorithm involves simultaneous adaptation of a parameterized distribution and an approximator of the objective function, which is akin to the actor-critic structure used in reinforcement learning. We establish a finite-time probability bound on the algorithm’s performance and show its global convergence when only a single simulation observation is collected at each iteration. Empirical results indicate that the algorithm is promising and may outperform some of the existing procedures in terms of efficiency and reliability.

Keywords: Simulation; simulation optimization; random search; actor-critic; stochastic approximation (search for similar items in EconPapers)
Date: 2022
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/opre.2021.2214 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:70:y:2022:i:6:p:3519-3537

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().