STELLM: Spatio-temporal enhanced pre-trained large language model for wind speed forecasting

Tangjie Wu and Qiang Ling

Applied Energy, 2024, vol. 375, issue C, No S030626192401417X

Abstract: As a renewable and clean energy source, wind energy has caught worldwide attention. To ensure the reliability and stability of wind energy production, wind speed forecasting is of great importance. Although existing forecasting models have shown promising results, they remain relatively small in scale and are incapable of learning comprehensive sequential representations from large-scale data to reason intricate spatio-temporal patterns obscured in entangled wind series. Recently, pre-trained large models have demonstrated impressive performance in natural language processing (NLP) and computer vision (CV). However, large models are rarely applied to wind speed forecasting due to the lack of large-scale wind data for training. To resolve this issue, we propose Spatio-Temporal Enhanced Pre-Trained Large Language Model, namely STELLM, innovatively leveraging the powerful reasoning ability of Large Language Models (LLMs) for accurate wind speed forecasting. Specifically, we first perform the series decomposition to separate wind series data into seasonal and trend components, capturing the distinct properties from disentangled patterns. Then, to empower LLMs with the spatio-temporal modeling capability, we introduce the spatial and temporal prompts, which fuse the geographic distribution information and short-term temporal patterns of wind turbines to enrich the input wind series. Moreover, we leverage an autoregressive fine-tuning strategy to align LLMs with wind series data and learn patch-level representations. Afterwards these representations are fed into a localized spatial module to capture the spatial dependencies between wind turbines. Extensive experiments on four public wind datasets demonstrate the performance superiority of STELLM.

Keywords: Wind speed forecasting; Seasonal-trend decomposition; Large language models; Spatio-temporal modeling (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S030626192401417X
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:375:y:2024:i:c:s030626192401417x

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2024.124034

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().