A multimodal sentiment classifier for financial decision making

Andrew Todd, James Bowden, Mark Cummins and Yang Su

International Review of Financial Analysis, 2025, vol. 105, issue C

Abstract: This study pioneers a multimodal approach to financial sentiment analysis through the integration of audio and textual data to enhance predictive accuracy. Motivated by the underutilisation of paralinguistic features and deep learning techniques in financial sentiment analysis, we introduce a novel deep learning-enabled multimodal classifier trained on corporate earnings calls using a subset of S&P 100 constituents. Our methodology incorporates FinBERT, a financial variant of Bidirectional Encoder Representation Transformations (BERT), alongside paralinguistic features and a deep learning classifier. Comparative analysis against established sentiment analysis methods, including dictionary approaches and machine learning models, suggests that our multimodal classifier achieves improved out-of-sample accuracy. Specifically, the inclusion of paralinguistic characteristics improves sentiment detection accuracy. Our research provides nuanced insights into sentiment analysis detection of different speakers (managers and analysts) during both the management discussion and Q&A sections of corporate earnings calls. Combined, our results suggest that multimodal sentiment analysis classification possesses the ability to deepen our understanding of the interplay between sentiment and market characteristics.

Keywords: Sentiment analysis; Multimodal analysis; Transformer model; Deep learning; Earnings call (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S1057521925004090
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:finana:v:105:y:2025:i:c:s1057521925004090

DOI: 10.1016/j.irfa.2025.104322

Access Statistics for this article

International Review of Financial Analysis is currently edited by B.M. Lucey

More articles in International Review of Financial Analysis from Elsevier
Bibliographic data for series maintained by Catherine Liu ().