Time Series Forecasting with Multi-Headed Attention-Based Deep Learning for Residential Energy Consumption

Seok-Jun Bu and Sung-Bae Cho
Additional contact information
Seok-Jun Bu: Department of Computer Science, Graduate School of Artificial Intelligence, Yonsei University, Seoul 03722, Korea
Sung-Bae Cho: Department of Computer Science, Graduate School of Artificial Intelligence, Yonsei University, Seoul 03722, Korea

Energies, 2020, vol. 13, issue 18, 1-16

Abstract: Predicting residential energy consumption is tantamount to forecasting a multivariate time series. A specific window for several sensor signals can induce various features extracted to forecast the energy consumption by using a prediction model. However, it is still a challenging task because of irregular patterns inside including hidden correlations between power attributes. In order to extract the complicated irregular energy patterns and selectively learn the spatiotemporal features to reduce the translational variance between energy attributes, we propose a deep learning model based on the multi-headed attention with the convolutional recurrent neural network. It exploits the attention scores calculated with softmax and dot product operation in the network to model the transient and impulsive nature of energy demand. Experiments with the dataset of University of California, Irvine (UCI) household electric power consumption consisting of a total 2,075,259 time-series show that the proposed model reduces the prediction error by 31.01% compared to the state-of-the-art deep learning model. Especially, the multi-headed attention improves the prediction performance even more by up to 27.91% than the single-attention.

Keywords: convolutional recurrent neural network; multi-headed attention; time-series forecasting; energy consumption prediction (search for similar items in EconPapers)
JEL-codes: Q Q0 Q4 Q40 Q41 Q42 Q43 Q47 Q48 Q49 (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (9)

Downloads: (external link)
https://www.mdpi.com/1996-1073/13/18/4722/pdf (application/pdf)
https://www.mdpi.com/1996-1073/13/18/4722/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jeners:v:13:y:2020:i:18:p:4722-:d:411730

Access Statistics for this article

Energies is currently edited by Ms. Agatha Cao

More articles in Energies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().