W-Transformers: A Wavelet-based Transformer Framework for Univariate Time Series Forecasting

Lena Sasal, Tanujit Chakraborty and Abdenour Hadid

Papers from arXiv.org

Abstract: Deep learning utilizing transformers has recently achieved a lot of success in many vital areas such as natural language processing, computer vision, anomaly detection, and recommendation systems, among many others. Among several merits of transformers, the ability to capture long-range temporal dependencies and interactions is desirable for time series forecasting, leading to its progress in various time series applications. In this paper, we build a transformer model for non-stationary time series. The problem is challenging yet crucially important. We present a novel framework for univariate time series representation learning based on the wavelet-based transformer encoder architecture and call it W-Transformer. The proposed W-Transformers utilize a maximal overlap discrete wavelet transformation (MODWT) to the time series data and build local transformers on the decomposed datasets to vividly capture the nonstationarity and long-range nonlinear dependencies in the time series. Evaluating our framework on several publicly available benchmark time series datasets from various domains and with diverse characteristics, we demonstrate that it performs, on average, significantly better than the baseline forecasters for short-term and long-term forecasting, even for datasets that consist of only a few hundred training samples.

Date: 2022-09
New Economics Papers: this item is included in nep-big, nep-cmp, nep-ecm, nep-ets and nep-for
References: View references in EconPapers View complete reference list from CitEc
Citations:

Published in 2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA)

Downloads: (external link)
http://arxiv.org/pdf/2209.03945 Latest version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:arx:papers:2209.03945

Access Statistics for this paper

More papers in Papers from arXiv.org
Bibliographic data for series maintained by arXiv administrators ().