Langevin Dynamics Based Algorithm e-TH ε O POULA for Stochastic Optimization Problems with Discontinuous Stochastic Gradient

Dong-Young Lim (), Ariel Neufeld (), Sotirios Sabanis () and Ying Zhang ()
Additional contact information
Dong-Young Lim: Department of Industrial Engineering, Ulsan National Institute of Science and Technology, Ulsan 44919, South Korea
Ariel Neufeld: Division of Mathematical Sciences, Nanyang Technological University, 637371 Singapore
Sotirios Sabanis: School of Mathematics, The University of Edinburgh, Edinburgh EH9 3FD, United Kingdom; and The Alan Turing Institute, London NW1 2DB, United Kingdom; and National Technical University of Athens, 10682 Athens, Greece
Ying Zhang: Financial Technology Thrust, Society Hub, The Hong Kong University of Science and Technology Guangzhou, Guangzhou, China

Mathematics of Operations Research, 2025, vol. 50, issue 3, 2333-2374

Abstract: We introduce a new Langevin dynamics based algorithm, called the extended tamed hybrid ε -order polygonal unadjusted Langevin algorithm (e-TH ε O POULA), to solve optimization problems with discontinuous stochastic gradients, which naturally appear in real-world applications such as quantile estimation, vector quantization, conditional value at risk (CVaR) minimization, and regularized optimization problems involving rectified linear unit (ReLU) neural networks. We demonstrate both theoretically and numerically the applicability of the e-TH ε O POULA algorithm. More precisely, under the conditions that the stochastic gradient is locally Lipschitz in average and satisfies a certain convexity at infinity condition, we establish nonasymptotic error bounds for e-TH ε O POULA in Wasserstein distances and provide a nonasymptotic estimate for the expected excess risk, which can be controlled to be arbitrarily small. Three key applications in finance and insurance are provided, namely, multiperiod portfolio optimization, transfer learning in multiperiod portfolio optimization, and insurance claim prediction, which involve neural networks with (Leaky)-ReLU activation functions. Numerical experiments conducted using real-world data sets illustrate the superior empirical performance of e-TH ε O POULA compared with SGLD (stochastic gradient Langevin dynamics), TUSLA (tamed unadjusted stochastic Langevin algorithm), adaptive moment estimation, and Adaptive Moment Estimation with a Strongly Non-Convex Decaying Learning Rate in terms of model accuracy.

Keywords: Primary: 65K10; 65C05; 68T07; Langevin dynamics based algorithm; discontinuous stochastic gradient; nonconvex stochastic optimization; nonasymptotic convergence bound; artificial neural networks; ReLU activation function; taming technique; superlinearly growing coefficients (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/moor.2022.0307 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ormoor:v:50:y:2025:i:3:p:2333-2374

Access Statistics for this article

More articles in Mathematics of Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().