Measuring sentiment news with transformer-based language models

Stefano Borgioli, Giampiero M. Gallo, Chiara Ongari, Maria Saveria Mavillonio and Caterina Giannetti

No 3283, Working Paper Series from European Central Bank

Abstract: Measuring sentiment from financial news is a central task in economics and finance, yet most existing indicators rely on dictionary-based approaches that infer sentiment from word counts and only partially capture context, negation, and semantic structure. This paper proposes a framework for constructing daily news mood indices using transformer-based language models and evaluates whether they better represent sentiment than dictionary-based alternatives. Using 143,755 financial news articles from Factiva, we classify sentiment at the sentence level with FinBERT and aggregate these predictions into article-level and daily sentiment measures through alternative normalization schemes. We compare the resulting indices with benchmark measures based on Shapiro et al., 2022 and Barbaglia et al., 2025. A central contribution is the validation of alternative sentiment measures against human judgments. We conducted an incentivized annotation exercise in which 444 participants evaluated a validation subsample of 588 financial news articles. Consensus ratings from independent human evaluations serve as an external benchmark for assessing the quality of automated sentiment measures. Across correlation, regression, and classification exercises, transformer-based measures show stronger agreement with human judgments than vocabulary-based alternatives and perform substantially better in distinguishing positive, neutral, and negative articles. Overall, the results suggest that incorporating contextual information through transformer-based language models produces sentiment measures that more closely reflect human assessments of financial news. JEL Classification: C55, C81, E32, E37, G14

Keywords: daily mood indices; financial news sentiment; FinBERT; human validation; text-as-data; transformer-based language models (search for similar items in EconPapers)
Date: 2026-09
Note: 339024
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.ecb.europa.eu//pub/pdf/scpwps/ecb.wp3283~df0633dfd3.en.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ecb:ecbwps:20263283

Access Statistics for this paper

More papers in Working Paper Series from European Central Bank 60640 Frankfurt am Main, Germany. Contact information at EDIRC.
Bibliographic data for series maintained by Official Publications ().