Machine Learning for Regularized Survey Forecast Combination: Partially-Egalitarian Lasso and its Derivatives

Francis Diebold and Minchul Shin

No 24967, NBER Working Papers from National Bureau of Economic Research, Inc

Abstract: Despite the clear success of forecast combination in many economic environments, several important issues remain incompletely resolved. The issues relate to selection of the set of forecasts to combine, and whether some form of additional regularization (e.g., shrinkage) is desirable. Against this background, and also considering the frequently-found good performance of simple-average combinations, we propose a LASSO-based procedure that sets some combining weights to zero and shrinks the survivors toward equality ("partially-egalitarian LASSO"). Ex-post analysis reveals that the optimal solution has a very simple form: The vast majority of forecasters should be discarded, and the remainder should be averaged. We therefore propose and explore direct subset-averaging procedures motivated by the structure of partially-egalitarian LASSO and the lessons learned, which, unlike LASSO, do not require choice of a tuning parameter. Intriguingly, in an application to the European Central Bank Survey of Professional Forecasters, our procedures outperform simple average and median forecasts – indeed they perform approximately as well as the ex-post best forecaster.

JEL-codes: C53 (search for similar items in EconPapers)
Date: 2018-08
New Economics Papers: this item is included in nep-big, nep-ecm, nep-ets and nep-for
Note: AP EFG IFM
References: Add references at CitEc
Citations: View citations in EconPapers (18)

Published as Francis X. Diebold & Minchul Shin, 2018. "Machine learning for regularized survey forecast combination: Partially-egalitarian LASSO and its derivatives," International Journal of Forecasting, .

Downloads: (external link)
http://www.nber.org/papers/w24967.pdf (application/pdf)

Related works:
Journal Article: Machine learning for regularized survey forecast combination: Partially-egalitarian LASSO and its derivatives (2019)
Working Paper: Machine Learning for Regularized Survey Forecast Combination: Partially Egalitarian Lasso and its Derivatives (2018)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nbr:nberwo:24967

Ordering information: This working paper can be ordered from
http://www.nber.org/papers/w24967

Access Statistics for this paper

More papers in NBER Working Papers from National Bureau of Economic Research, Inc National Bureau of Economic Research, 1050 Massachusetts Avenue Cambridge, MA 02138, U.S.A.. Contact information at EDIRC.
Bibliographic data for series maintained by ().