Robust monitoring machine: a machine learning solution for out-of-sample R $$^2$$ 2 -hacking in return predictability monitoring

James Yae () and Yang Luo ()
Additional contact information
James Yae: University of Houston
Yang Luo: University of Houston

Financial Innovation, 2023, vol. 9, issue 1, 1-28

Abstract: Abstract The out-of-sample $$R^2$$ R 2 is designed to measure forecasting performance without look-ahead bias. However, researchers can hack this performance metric even without multiple tests by constructing a prediction model using the intuition derived from empirical properties that appear only in the test sample. Using ensemble machine learning techniques, we create a virtual environment that prevents researchers from peeking into the intuition in advance when performing out-of-sample prediction simulations. We apply this approach to robust monitoring, exploiting a dynamic shrinkage effect by switching between a proposed forecast and a benchmark. Considering stock return forecasting as an example, we show that the resulting robust monitoring forecast improves the average performance of the proposed forecast by 15% (in terms of mean-squared-error) and reduces the variance of its relative performance by 46% while avoiding the out-of-sample $$R^2$$ R 2 -hacking problem. Our approach, as a final touch, can further enhance the performance and stability of forecasts from any models and methods.

Keywords: Machine learning; Out-of-sample R $$^2$$ 2 -hacking; Return predictability; Monitoring (search for similar items in EconPapers)
JEL-codes: C52 C53 C55 C58 G17 (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://link.springer.com/10.1186/s40854-023-00497-z Abstract (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:fininn:v:9:y:2023:i:1:d:10.1186_s40854-023-00497-z

Ordering information: This journal article can be ordered from
http://www.springer. ... nomics/journal/40589

DOI: 10.1186/s40854-023-00497-z

Access Statistics for this article

Financial Innovation is currently edited by J. Leon Zhao and Zongyi

More articles in Financial Innovation from Springer, Southwestern University of Finance and Economics
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().