Enhanced Balancing of Bias-Variance Tradeoff in Stochastic Estimation: A Minimax Perspective

Henry Lam (), Xinyu Zhang () and Xuhui Zhang ()
Additional contact information
Henry Lam: Department of Industrial Engineering and Operations Research, Columbia University, New York, New York 10027
Xinyu Zhang: Department of Industrial Engineering and Operations Research, Columbia University, New York, New York 10027
Xuhui Zhang: Department of Management Science and Engineering, Stanford University, Stanford, California 94305

Operations Research, 2023, vol. 71, issue 6, 2352-2373

Abstract: Biased stochastic estimators, such as finite differences for noisy gradient estimation, often contain parameters that need to be properly chosen to balance impacts from the bias and the variance. Although the optimal order of these parameters in terms of the simulation budget can be readily established, the precise best values depend on model characteristics that are typically unknown in advance. We introduce a framework to construct new classes of estimators based on judicious combinations of simulation runs on sequences of tuning parameter values, such that the estimators consistently outperform a given tuning parameter choice in the conventional approach, regardless of the unknown model characteristics. We argue the outperformance via what we call the asymptotic minimax risk ratio, obtained by minimizing the worst-case asymptotic ratio between the mean square errors of our estimators and the conventional one, where the worst case is over any possible values of the model unknowns. In particular, when the minimax ratio is less than 1, the calibrated estimator is guaranteed to perform better asymptotically. We identify this minimax ratio for general classes of weighted estimators and the regimes where this ratio is less than 1. Moreover, we show that the best weighting scheme is characterized by a sum of two components with distinct decay rates. We explain how this arises from bias-variance balancing that combats the adversarial selection of the model constants, which can be analyzed via a tractable reformulation of a nonconvex optimization problem.

Keywords: Simulation; bias-variance tradeoff; minimax analysis; stochastic estimation; finite difference; robust optimization (search for similar items in EconPapers)
Date: 2023
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/opre.2022.2319 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:71:y:2023:i:6:p:2352-2373

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().