Targeting predictors in random forest regression

Daniel Borup, Bent Jesper Christensen, Nicolaj N. Mühlbach () and Mikkel S. Nielsen ()
Additional contact information
Nicolaj N. Mühlbach: Aarhus University and CREATES, Postal: Department of Economics and Business Economics, Fuglesangs Allé 4, 8210 Aarhus V, Denmark
Mikkel S. Nielsen: Columbia University, Postal: Department of Statistics, Columbia University, Room 1005 SSW, MC 4690, 1255 Amsterdam Avenue, New York, NY 10027, USA

CREATES Research Papers from Department of Economics and Business Economics, Aarhus University

Abstract: Random forest regression (RF) is an extremely popular tool for the analysis of high-dimensional data. Nonetheless, its benefits may be lessened in sparse settings, due to weak predictors, and a pre-estimation dimension reduction (targeting) step is required. We show that proper targeting controls the probability of placing splits along strong predictors, thus providing an important complement to RF’s feature sampling. This is supported by simulations using representative finite samples. Moreover, we quantify the immediate gain from targeting in terms of increased strength of individual trees. Macroeconomic and financial applications show that the bias-variance tradeoff implied by targeting, due to increased correlation among trees in the forest, is balanced at a medium degree of targeting, selecting the best 10–30% of commonly applied predictors. Improvements in predictive accuracy of targeted RF relative to ordinary RF are considerable, up to 12–13%, occurring both in recessions and expansions, particularly at long horizons.

Keywords: Random forests; LASSO; high-dimensional forecasting; weak predictors; targeted predictors (search for similar items in EconPapers)
JEL-codes: C53 C55 E17 G12 (search for similar items in EconPapers)
Pages: 48
Date: 2020-05-14
New Economics Papers: this item is included in nep-big, nep-cmp and nep-mac
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (13)

Downloads: (external link)
https://repec.econ.au.dk/repec/creates/rp/20/rp20_03.pdf (application/pdf)

Related works:
Journal Article: Targeting predictors in random forest regression (2023)
Working Paper: Targeting predictors in random forest regression (2020)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:aah:create:2020-03

Access Statistics for this paper

More papers in CREATES Research Papers from Department of Economics and Business Economics, Aarhus University
Bibliographic data for series maintained by ().