EcoFinBench – a natural language processing benchmark for economics and finance

Max Ahrens, Dragos Gorduza and Micheal McMahon
Additional contact information
Max Ahrens: Maihem.ai
Dragos Gorduza: Bank of England
Micheal McMahon: University of Oxford

No 1163, Bank of England Staff Working Paper series from Bank of England

Abstract: We introduce EcoFinBench, a natural language processing (NLP) benchmark suite for the domains of economics and finance. We comprehensively test a large array of NLP models across multiple domain-specific data sets for sentence classification. Specifically, we evaluate dictionary models, word count models, topic models, and modern transformer models. Furthermore, we introduce two new data sets to the research community. The Bluebook data set for text-only sentiment analysis in monetary policy, and the Greenbook data set for multimodal (text and numeric) sentiment analysis. We focus on data sets that require the models to work with relatively few data points and long average text lengths – typical characteristics of data sets in the economic and financial domain. We find that, dictionary models – still widely used as a default text analysis tool in economics and finance – underperform substantially across all evaluated data sets. From our findings, we conclude that given the underperformance of existing solutions in the multimodal domain, future modelling work is needed. With our benchmark suite we aim to lay the foundation for a systematic assessment on the most commonly used NLP models in economics and finance. To our knowledge, we are the first to provide such holistic benchmarking assessment for economics and finance.

Keywords: Machine learning,natural language processing; artificial intelligence; benchmark. (search for similar items in EconPapers)
JEL-codes: C45 C55 C88 G17 Y10 (search for similar items in EconPapers)
Pages: 20
Date: 2025-12-19
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.bankofengland.co.uk/-/media/boe/files/ ... conomics-finance.pdf

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:boe:boeewp:023285

Access Statistics for this paper

More papers in Bank of England Staff Working Paper series from Bank of England Bank of England, Threadneedle Street, London, EC2R 8AH. Contact information at EDIRC.
Bibliographic data for series maintained by Research ().