Reproducibility in forecasting research

John E. Boylan, Paul Goodwin, Maryam Mohammadipour and Aris A. Syntetos

International Journal of Forecasting, 2015, vol. 31, issue 1, 79-90

Abstract: The importance of replication has been recognised across many scientific disciplines. Reproducibility is a necessary condition for replicability, because an inability to reproduce results implies that the methods have not been specified sufficiently, thus precluding replication. This paper describes how two independent teams of researchers attempted to reproduce the empirical findings of an important paper, “Shrinkage estimators of time series seasonal factors and their effect on forecasting accuracy” (Miller & Williams, 2003). The two teams proceeded systematically, reporting results both before and after receiving clarifications from the authors of the original study. The teams were able to approximately reproduce each other’s results, but not those of Miller and Williams. These discrepancies led to differences in the conclusions as to the conditions under which seasonal damping outperforms classical decomposition. The paper specifies the forecasting methods employed using a flowchart. It is argued that this approach to method documentation is complementary to the provision of computer code, as it is accessible to a broader audience of forecasting practitioners and researchers. The significance of this research lies not only in its lessons for seasonal forecasting but also, more generally, in its approach to the reproduction of forecasting research.

Keywords: Forecasting practice; Replication; Seasonal forecasting; Empirical research (search for similar items in EconPapers)
Date: 2015
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (16)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0169207014001009
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:intfor:v:31:y:2015:i:1:p:79-90

DOI: 10.1016/j.ijforecast.2014.05.008

Access Statistics for this article

International Journal of Forecasting is currently edited by R. J. Hyndman

More articles in International Journal of Forecasting from Elsevier
Bibliographic data for series maintained by Catherine Liu ().