LLM-Guided Multimodal Information Fusion With Hierarchical Spatio-Temporal Graph Network for Sentiment Analysis

Yujie Jin, Yong Wang, Yuzhe Wang, Qiyang Chen, Bin Hu, Yanling Han, Chaoyin Ma and Witold Pedrycz
Additional contact information
Yujie Jin: Changsha Normal University, China
Yong Wang: Changsha Normal University, China
Yuzhe Wang: Rocket Force University of Engineering, China
Qiyang Chen: Montclair State University, USA
Bin Hu: Changsha Normal University, China
Yanling Han: Rocket Force University of Engineering, China
Chaoyin Ma: Rocket Force University of Engineering, China
Witold Pedrycz: University of Alberta, Canada

International Journal of Information Systems in the Service Sector (IJISSS), 2025, vol. 16, issue 1, 1-15

Abstract: Multimodal sentiment analysis aims to attain a precise comprehension of emotions by integrating complementary textual, visual, and audio information. However, issues such as sentiment discrepancies between modalities, ineffective integration of multi-modal information, and the intricacy of order dependency significantly constrain the models' efficacy. The authors propose an LLM-guided Hierarchical Spatio-Temporal Graph Network (L-HSTGN). By multimodal large model feature enhancement, bidirectional spatio-temporal joint modeling, and dynamic gate fusion mechanism, they effectively address the aforementioned problems. Firstly, they produce cross-modal emotion pseudo-labels based on the multimodal large model, and the single-modal representation was optimized by combining adversarial regularization. Secondly, they develop a bidirectional spatio-temporal convolution module to concurrently extract local-global temporal characteristics and dynamic spatial correlations.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://services.igi-global.com/resolvedoi/resolve ... 0.4018/IJISSS.388002 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:igg:jisss0:v:16:y:2025:i:1:p:1-15

Access Statistics for this article

International Journal of Information Systems in the Service Sector (IJISSS) is currently edited by John Wang

More articles in International Journal of Information Systems in the Service Sector (IJISSS) from IGI Global Scientific Publishing
Bibliographic data for series maintained by Journal Editor ().