 Are forecasting competitions data representative of the reality?Evangelos Spiliotis, Andreas Kouloumos, Vassilios Assimakopoulos and Spyros Makridakis International Journal of Forecasting, 2020, vol. 36, issue 1, 37-53 Abstract: Forecasters typically evaluate the performances of new forecasting methods by exploiting data from past forecasting competitions. Over the years, numerous studies have based their conclusions on such datasets, with mis-performing methods being unlikely to receive any further attention. However, it has been reported that these datasets might not be indicative, as they display many limitations. Since forecasting research is driven somewhat by data from forecasting competitions, it becomes vital to determine whether they are indeed representative of the reality or whether forecasters tend to over-fit their methods on a random sample of series. This paper uses the data from M4 as proportionate to the real world and compares its properties with those of past datasets commonly used in the literature as benchmarks in order to provide evidence on that question. The results show that many popular benchmarks of the past may indeed deviate from reality, and ways forward are discussed in response. Keywords: Forecasting competitions; Time series visualization; Time series features; M4; Forecasting methods (search for similar items in EconPapers) Downloads: (external link) Related works: Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text Persistent link: https://EconPapers.repec.org/RePEc:eee:intfor:v:36:y:2020:i:1:p:37-53 DOI: 10.1016/j.ijforecast.2018.12.007 Access Statistics for this article International Journal of Forecasting is currently edited by R. J. Hyndman More articles in International Journal of Forecasting from Elsevier |
Is your work missing from RePEc? Here is how to contribute.
Questions or problems? Check the EconPapers FAQ or send mail to .
EconPapers is hosted by the
School of Business at Örebro University.