Evaluation Procedures for Forecasting with Spatiotemporal Data

Mariana Oliveira, Luís Torgo and Vítor Santos Costa
Additional contact information
Mariana Oliveira: Department of Computer Science, Faculty of Sciences, University of Porto, Rua Campo Alegre 1055, 4169-007 Porto, Portugal
Luís Torgo: Faculty of Computer Science, Dalhousie University, 6050 University Av., Halifax, NS B3H 1W5, Canada
Vítor Santos Costa: Department of Computer Science, Faculty of Sciences, University of Porto, Rua Campo Alegre 1055, 4169-007 Porto, Portugal

Mathematics, 2021, vol. 9, issue 6, 1-27

Abstract: The increasing use of sensor networks has led to an ever larger number of available spatiotemporal datasets. Forecasting applications using this type of data are frequently motivated by important domains such as environmental monitoring. Being able to properly assess the performance of different forecasting approaches is fundamental to achieve progress. However, traditional performance estimation procedures, such as cross-validation, face challenges due to the implicit dependence between observations in spatiotemporal datasets. In this paper, we empirically compare several variants of cross-validation (CV) and out-of-sample (OOS) performance estimation procedures, using both artificially generated and real-world spatiotemporal datasets. Our results show both CV and OOS reporting useful estimates, but they suggest that blocking data in space and/or in time may be useful in mitigating CV’s bias to underestimate error. Overall, our study shows the importance of considering data dependencies when estimating the performance of spatiotemporal forecasting models.

Keywords: evaluation methods; performance estimation; cross-validation; spatiotemporal data; geo-referenced time series; reproducible research (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
https://www.mdpi.com/2227-7390/9/6/691/pdf (application/pdf)
https://www.mdpi.com/2227-7390/9/6/691/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:9:y:2021:i:6:p:691-:d:522632

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().