The Promise and Peril of Generative AI: Evidence from GPT-4 as Sell-Side Analysts

Edward Li, Zhiyuan Tu and Dexin Zhou

Papers from arXiv.org

Abstract: We investigate how advanced large language models (LLMs), specifically GPT-4, process corporate disclosures to forecast earnings. Using earnings press releases issued around GPT-4's knowledge cutoff date, we address two questions: (1) Do GPT-generated earnings forecasts outperform analysts in accuracy? (2) How is GPT's performance related to its processing of textual and quantitative information? Our findings suggest that GPT forecasts are significantly less accurate than those of analysts. This underperformance can be traced to GPT's distinct textual and quantitative approaches: its textual processing follows a consistent, generalized pattern across firms, highlighting its strengths in language tasks. In contrast, its quantitative processing capabilities vary significantly across firms, revealing limitations tied to the uneven availability of domain-specific training data. Additionally, there is some evidence that GPT's forecast accuracy diminishes beyond its knowledge cutoff, underscoring the need to evaluate LLMs under hindsight-free conditions. Overall, this study provides a novel exploration of the "black box" of GPT-4's information processing, offering insights into LLMs' potential and challenges in financial applications.

Date: 2024-12
New Economics Papers: this item is included in nep-ain, nep-big and nep-cmp
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://arxiv.org/pdf/2412.01069 Latest version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:arx:papers:2412.01069

Access Statistics for this paper

More papers in Papers from arXiv.org
Bibliographic data for series maintained by arXiv administrators ().