Mathematics of Embeddings: Spillover of Polarities over Financial Texts

Mengda Li and Charles-Albert Lehalle

Chapter 3 in Reviews in Modern Quantitative Finance, 2024, pp 151-188 from World Scientific Publishing Co. Pte. Ltd.

Abstract: In this chapter, we perform a mathematical analysis of the word2vec model. This sheds light on how the decision to use such a model makes implicit assumptions on the structure of the language. Beside, under Markovian assumptions that we discuss, we provide a very clear theoretical understanding of the formation of embeddings and, in particular, the way it captures what we call frequentist synonyms. These assumptions allow to conduct an explicit analysis of the loss function commonly used by these NLP techniques that asymptotically reaches a cross-entropy between the language model and the underlying true generative model.Moreover, we produce synthetic corpora with different levels of structures and show empirically how the word2vec algorithm succeed, or not, to learn them. It leads us to empirically assess the capability of such models to capture structures on a corpus of around 42 millions of financial news covering 12 years. And, we rely on the Loughran–McDonald Sentiment Polarity Word Lists and we show that embeddings are exposed to mixing terms with opposite polarity because of the way they treat antonyms as frequentist synonyms. Besides, we study the non-stationarity of such a financial corpus that has surprisingly not be documented in the literature.

Keywords: Quantitative Finance; Financial Engineering; Mathematical Finance; Computational Finance; Computational Methods; Computational Problems; Pricing of Securities; Trading; Market Microstructures; Risk Theory; Queuing Theory; Asset Management Technique; Liability Management Technique; Risk Measures; Solvency; Financial Instability; Fintech; Cryptocurrencies; Financial Machine Learning; Artificial Intelligence; Fintech; Quantum Computing; Distributed Ledgers; Econophysics (search for similar items in EconPapers)
JEL-codes: C C02 C6 C61 (search for similar items in EconPapers)
Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.worldscientific.com/doi/pdf/10.1142/9789811281747_0003 (application/pdf)
https://www.worldscientific.com/doi/abs/10.1142/9789811281747_0003 (text/html)
Ebook Access is available upon purchase.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wsi:wschap:9789811281747_0003

Ordering information: This item can be ordered from

Access Statistics for this chapter

More chapters in World Scientific Book Chapters from World Scientific Publishing Co. Pte. Ltd.
Bibliographic data for series maintained by Tai Tone Lim ().