Achieving Efficiency in Black-Box Simulation of Distribution Tails with Self-Structuring Importance Samplers

Anand Deo () and Karthyek Murthy ()
Additional contact information
Anand Deo: Decision Sciences, Indian Institute of Management, Bangalore 560076, India
Karthyek Murthy: Engineering Systems & Design, Singapore University of Technology & Design, Singapore 487372

Operations Research, 2025, vol. 73, issue 1, 325-343

Abstract: This paper presents a novel importance sampling (IS) scheme for estimating distribution tails of performance measures modeled with a rich set of tools, such as linear programs, integer linear programs, piecewise linear/quadratic objectives, feature maps specified with deep neural networks, etc. The conventional approach of explicitly identifying efficient changes of measure suffers from feasibility and scalability concerns beyond highly stylized models because of their need to be tailored intricately to the objective and the underlying probability distribution. This bottleneck is overcome in the proposed scheme with an elementary transformation that is capable of implicitly inducing an effective IS distribution in a variety of models by replicating the concentration properties observed in less rare samples. This novel approach is guided by developing a large deviations principle that brings out the phenomenon of self-similarity of optimal IS distributions. The proposed sampler is the first to attain asymptotically optimal variance reduction across a spectrum of multivariate distributions despite being oblivious to the specifics of the underlying model. Its applicability is illustrated with contextual shortest-path and portfolio credit risk models informed by neural networks.

Keywords: Simulation; variance reduction; tail risks; rare event simulation; importance sampling; contextual models; portfolio credit risk; value-at-risk; conditional value-at-risk; log efficient (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/opre.2021.0331 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:73:y:2025:i:1:p:325-343

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().