On the Disagreement of Forecasting Model Selection Criteria
Evangelos Spiliotis (),
Fotios Petropoulos and
Vassilios Assimakopoulos
Additional contact information
Evangelos Spiliotis: Forecasting and Strategy Unit, School of Electrical and Computer Engineering, National Technical University of Athens, 15780 Athens, Greece
Fotios Petropoulos: School of Management, University of Bath, Bath BA2 7AY, UK
Vassilios Assimakopoulos: Forecasting and Strategy Unit, School of Electrical and Computer Engineering, National Technical University of Athens, 15780 Athens, Greece
Forecasting, 2023, vol. 5, issue 2, 1-12
Abstract:
Forecasters have been using various criteria to select the most appropriate model from a pool of candidate models. This includes measurements on the in-sample accuracy of the models, information criteria, and cross-validation, among others. Although the latter two options are generally preferred due to their ability to tackle overfitting, in univariate time-series forecasting settings, limited work has been conducted to confirm their superiority. In this study, we compared such popular criteria for the case of the exponential smoothing family of models using a large data set of real series. Our results suggest that there is significant disagreement between the suggestions of the examined criteria and that, depending on the approach used, models of different complexity may be favored, with possible negative effects on the forecasting accuracy. Moreover, we find that simple in-sample error measures can effectively select forecasting models, especially when focused on the most recent observations in the series.
Keywords: model selection; information criteria; time series; exponential smoothing; M4 competition (search for similar items in EconPapers)
JEL-codes: A1 B4 C0 C1 C2 C3 C4 C5 C8 M0 Q2 Q3 Q4 (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:
Downloads: (external link)
https://www.mdpi.com/2571-9394/5/2/27/pdf (application/pdf)
https://www.mdpi.com/2571-9394/5/2/27/ (text/html)
Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
HTML/Text
Persistent link: https://EconPapers.repec.org/RePEc:gam:jforec:v:5:y:2023:i:2:p:27-498:d:1175547
Access Statistics for this article
Forecasting is currently edited by Ms. Joss Chen
More articles in Forecasting from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().