Backtestability and the Ridge Backtest

Carlo Acerbi and Balazs Szekely

Chapter 3 in Peter Carr Gedenkschrift:Research Advances in Mathematical Finance, 2023, pp 61-100 from World Scientific Publishing Co. Pte. Ltd.

Abstract: We propose a formal definition of backtestability for a statistical functional of a distribution: a functional is backtestable if there exists a back-test function depending only on the forecast of the functional and the related random variable, which is strictly monotonic in the former and has zero expected value for an exact forecast. We discuss the relationship with elicitability and identifiability which turn out to be necessary conditions for backtestability. The variance and the expected shortfall are not backtestable for this reason. We compare (absolute) model validation in the context of hypothesis tests, via backtest functions, versus (relative) model selection between competing forecasting models, via scoring functions. We define a backtest to be sharp when it is strictly monotonic with respect to the real value of the functional and not only to its forecast. This decides whether the expected value of the backtest determines also the prediction discrepancy and not only its significance. We show that the quantile backtest is not sharp and in fact it provides no information whatsoever on its true value. The expectile is also not sharp; we provide bounds for its true value, which are looser for outer confidence levels. We then introduce the notion of ridge backtests, applicable to particular non-backtestable functionals, such as the variance and the expected shortfall, which coincide with the attained minimum of the scoring function of another elicitable auxiliary functional (the mean and the value at risk, respectively). This permits approximated sharp backtests up to a small and one-sided sensitivity to the prediction of the auxiliary variable. The ridge mechanism explains why the variance has always been de facto backtestable and allows for similar efficient ways to backtest the expected shortfall. We discuss the relevance of this result in the current debate of financial regulation (banking and insurance), where value at risk and expected shortfall are adopted as regulatory risk measures.

Keywords: Mathematical Finance; Quantitative Finance; Option Pricing; Derivatives; No Arbitrage; Asset Price Bubbles; Asset Pricing; Equilibrium; Volatility; Diffusion Processes; Jump Processes; Stochastic Integration; Trading Strategies; Portfolio Theory; Optimization; Securities; Bonds; Commodities; Futures (search for similar items in EconPapers)
JEL-codes: C02 C6 (search for similar items in EconPapers)
Date: 2023
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.worldscientific.com/doi/pdf/10.1142/9789811280306_0003 (application/pdf)
https://www.worldscientific.com/doi/abs/10.1142/9789811280306_0003 (text/html)
Ebook Access is available upon purchase.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wsi:wschap:9789811280306_0003

Ordering information: This item can be ordered from

Access Statistics for this chapter

More chapters in World Scientific Book Chapters from World Scientific Publishing Co. Pte. Ltd.
Bibliographic data for series maintained by Tai Tone Lim ().