Minimization of a class of rare event probabilities and buffered probabilities of exceedance

Amarjit Budhiraja (), Shu Lu (), Yang Yu () and Quoc Tran-Dinh ()
Additional contact information
Amarjit Budhiraja: The University of North Carolina at Chapel Hill (UNC)
Shu Lu: The University of North Carolina at Chapel Hill (UNC)
Yang Yu: The University of North Carolina at Chapel Hill (UNC)
Quoc Tran-Dinh: The University of North Carolina at Chapel Hill (UNC)

Annals of Operations Research, 2021, vol. 302, issue 1, No 3, 49-83

Abstract: Abstract We consider the problem of choosing design parameters to minimize the probability of an undesired rare event that is described through the average of n i.i.d. random variables. Since the probability of interest for near optimal design parameters is very small, one needs to develop suitable accelerated Monte-Carlo methods for estimating its value. One of the challenges in the study is that simulating from exponential twists of the laws of the summands may be computationally demanding since these transformed laws may be non-standard and intractable. We consider a setting where the summands are given as a nonlinear functional of random variables, the exponential twists of whose distributions take a simpler form than those for the original summands. We use techniques from Dupuis and Wang (Stochastics 76(6):481–508, 2004, Math Oper Res 32(3):723–757, 2007) to identify the appropriate Issacs equations whose subsolutions are used to construct tractable importance sampling (IS) schemes. We also study the closely related problem of estimating buffered probability of exceedance and provide the first rigorous results that relate the asymptotics of buffered probability and that of the ordinary probability under a large deviation scaling. The analogous minimization problem for buffered probability, under conditions, can be formulated as a convex optimization problem. We show that, under conditions, changes of measures that are asymptotically efficient (under the large deviation scaling) for estimating ordinary probability are also asymptotically efficient for estimating the buffered probability of exceedance. We embed the constructed IS scheme in gradient descent algorithms to solve the optimization problems, and illustrate these schemes through computational experiments.

Keywords: Importance sampling; Stochastic optimization; Large deviations; Buffered probability; 90C15; 65K10; 65C05; 60F10 (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10479-021-03991-8 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:annopr:v:302:y:2021:i:1:d:10.1007_s10479-021-03991-8

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10479

DOI: 10.1007/s10479-021-03991-8

Access Statistics for this article

Annals of Operations Research is currently edited by Endre Boros

More articles in Annals of Operations Research from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().