Extending the Scope of Inference About Predictive Ability to Machine Learning Methods

Juan Carlos Escanciano and Ricardo Parra

Papers from arXiv.org

Abstract: The use of machine learning methods for predictive purposes has increased dramatically over the past two decades, but uncertainty quantification for predictive comparisons remains elusive. This paper addresses this gap by extending the classic inference theory for predictive ability in time series to modern machine learners, such as the Lasso or Deep Learning. We investigate under which conditions such extensions are possible. For standard out-of-sample asymptotic inference to be valid with machine learning, two key properties must hold: (I) a zero-mean condition for the score of the prediction loss function and (ii) a "fast rate" of convergence for the machine learner. Absent any of these conditions, the estimation risk may be unbounded, and inferences invalid and very sensitive to sample splitting. For accurate inferences, we recommend an 80%-20% training-test splitting rule. We illustrate the wide applicability of our results with three applications: high-dimensional time series regressions with the Lasso, Deep learning for binary outcomes, and a new out-of-sample test for the Martingale Difference Hypothesis (MDH). The theoretical results are supported by extensive Monte Carlo simulations and an empirical application evaluating the MDH of some major exchange rates at daily and higher frequencies.

Date: 2024-02, Revised 2025-05
New Economics Papers: this item is included in nep-big, nep-cmp, nep-ecm and nep-ets
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://arxiv.org/pdf/2402.12838 Latest version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:arx:papers:2402.12838

Access Statistics for this paper

More papers in Papers from arXiv.org
Bibliographic data for series maintained by arXiv administrators ().