When Does Aggregating Multiple Skills with Multi-Task Learning Work? A Case Study in Financial NLP

Jingwei Ni, Zhijing Jin, Qian Wang, Mrinmaya Sachan and Markus Leippold
Additional contact information
Jingwei Ni: ETH Zurich
Zhijing Jin: ETH Zurich
Qian Wang: University of Zurich
Mrinmaya Sachan: ETH Zürich
Markus Leippold: University of Zurich; Swiss Finance Institute

No 23-112, Swiss Finance Institute Research Paper Series from Swiss Finance Institute

Abstract: Multi-task learning (MTL) aims at achieving a better model by leveraging data and knowledge from multiple tasks. However, MTL does not always work – sometimes negative transfer occurs between tasks, especially when aggregating loosely related skills, leaving it an open question when MTL works. Previous studies show that MTL performance can be improved by algorithmic tricks. However, what tasks and skills should be included is less well explored. In this work, we conduct a case study in Financial NLP where multiple datasets exist for skills relevant to the domain, such as numeric reasoning and sentiment analysis. Due to the task difficulty and data scarcity in the Financial NLP domain, we explore when aggregating such diverse skills from multiple datasets with MTL can work. Our findings suggest that the key to MTL success lies in skill diversity, relatedness between tasks, and choice of aggregation size and shared capacity. Specifically, MTL works well when tasks are diverse but related, and when the size of the task aggregation and the shared capacity of the model are balanced to avoid overwhelming certain tasks.

Keywords: Multi-Task Learning; Sentiment Analysis; Financial Datasets; FinBERT (search for similar items in EconPapers)
Pages: 24 pages
Date: 2023-11
New Economics Papers: this item is included in nep-big
References: Add references at CitEc
Citations:

Downloads: (external link)
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4456653 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:chf:rpseri:rp23112

Access Statistics for this paper

More papers in Swiss Finance Institute Research Paper Series from Swiss Finance Institute Contact information at EDIRC.
Bibliographic data for series maintained by Ridima Mittal (rps@sfi.ch).