Let U​​s Ask a Gen AI Chatbot: Machine Versus Bank Survey Results

Veronica Bayangos, Neil Fidelle Lomibao and Ralph Joshua Macarasig
Additional contact information
Veronica Bayangos: Bangko Sentral ng Pilipinas
Neil Fidelle Lomibao: Bangko Sentral ng Pilipinas
Ralph Joshua Macarasig: Bangko Sentral ng Pilipinas

No 202501, BSP Discussion Paper Series from Bangko Sentral ng Pilipinas

Abstract: This study evaluates the performance of two generative pre-trained transformer (GPT) models, ChatGPT 4o and Gemini Pro 1.5, in responding to a banking sector outlook survey designed to support supervisory policy assessment. The study is structured around three main exercises. The first classifies questions related to the analysis of financial risks, challenges, and outlooks. The second employs prompt engineering to mitigate biases and address potential hallucinations in the models. The third constructs Cosine Similarity Indices (CSIs) to compare the responses of the GPT models with those of banks. The findings suggest that both models can serve as valuable tools for assessing a bank’s financial performance, risks, and macroeconomic outlook. These results underscore their potential to complement a supervisor’s analytical tasks. However, the observed differences in responses may point to the complex and potentially non-linear relationships that underlie bank performance, thereby reinforcing the continued importance of human judgment in bank supervision.

JEL-codes: C55 E42 E5 E61 G28 (search for similar items in EconPapers)
Pages: 37 pages
Date: 2025-02
References: Add references at CitEc
Citations:

Published as BSP Discussion paper No. 2025-01, February 2025

Downloads: (external link)
https://www.bsp.gov.ph/Sites/researchsite/Publicat ... -Papers/DP202501.pdf

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bhd:dpaper:202501

Access Statistics for this paper

More papers in BSP Discussion Paper Series from Bangko Sentral ng Pilipinas Contact information at EDIRC.
Bibliographic data for series maintained by MB Pagalunan ().