EconPapers    
Economics at your fingertips  
 

MMGPT4LF: Leveraging an optimized pre-trained GPT-2 model with multi-modal cross-attention for load forecasting

Mingyang Gao, Suyang Zhou, Wei Gu, Zhi Wu, Haiquan Liu, Aihua Zhou and Xinliang Wang

Applied Energy, 2025, vol. 392, issue C, No S0306261925006956

Abstract: Accurate load forecasting is crucial for maintaining power system balance. Traditionally, forecasting relies on time series data such as historical loads and corresponding meteorological information. However, non-time-series data like news reports and holiday schedules can also significantly influence outcomes. Existing research primarily focuses on time series data and lacks effective handling of multi-modal inputs. Recent advances in Large Language Models (LLMs) demonstrate inherent advantages in capturing long-term dependencies and complex textual patterns, indicating their potential for load forecasting. Nevertheless, the application of LLMs in this field remains limited. Thus, to fill this gap, we propose MMGPT4LF, a model that combines the pre-trained GPT-2 model with multi-modal data inputs for load forecasting. Specifically, the model designs an additional time series input head to more effectively capture temporal dependencies, particularly the periodicity and long-term trends present in power load data. Furthermore, the model incorporates a Multi-Modal Cross-Attention (MMCA) mechanism, enabling efficient alignment and fusion of high-dimensional feature representations from both time series and textual inputs. Through this framework, MMGPT4LF not only enhances the effectiveness of multi-modal data fusion but also accurately handles the interactions between different modalities, thereby significantly improving load forecasting accuracy and the model’s generalization ability. Extensive experiments on two open-source load forecasting datasets, compared with nine advanced time series forecasting models, validate the effectiveness and accuracy of MMGPT4LF in load forecasting tasks.

Keywords: Load forecasting; Pre-trained large language model; Multi-modal; Deep learning (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261925006956
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:392:y:2025:i:c:s0306261925006956

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2025.125965

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().

 
Page updated 2025-05-20
Handle: RePEc:eee:appene:v:392:y:2025:i:c:s0306261925006956