Determining Convergence for Expected Improvement-Based Bayesian Optimization

Nicholas R. Grunloh and Herbert K. H. Lee ()
Additional contact information
Nicholas R. Grunloh: Department of Statistics, University of California, Santa Cruz, CA 95064, USA
Herbert K. H. Lee: Department of Statistics, University of California, Santa Cruz, CA 95064, USA

Mathematics, 2025, vol. 13, issue 20, 1-16

Abstract: Bayesian optimization routines may have theoretical convergence results, but determining whether a run has converged in practice can be a subjective task. This paper provides a framework inspired by statistical process control for monitoring an optimization run for convergence. The maximum Expected Improvement (EI) tends to decrease during an optimization run, but decreasing EI is not sufficient for convergence. We consider both a decrease in EI as well as local stability of the variance in order to assess for convergence. The EI process is made more numerically stable through an expected log-normal approximation. An Exponentially Weighted Moving Average control chart is adapted for automated convergence analysis, which allows assessment of stability of both the EI and its variance. The success of the methodology is demonstrated on several examples.

Keywords: derivative-free optimization; computer simulation; emulator; statistical process control; exponentially weighted moving average (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/13/20/3261/pdf (application/pdf)
https://www.mdpi.com/2227-7390/13/20/3261/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:13:y:2025:i:20:p:3261-:d:1769368

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().