Quality and Accountability of Large Language Models (LLMs) in Healthcare in Low- and Middle-Income Countries (LMIC): A Simulated Patient Study using ChatGPT

Yafei Si, Yuyi Yang, Xi Wang, Ruopeng An, Jiaqi Zu, Xi Chen, Xiaojing Fan and Sen Gong

No 1472, GLO Discussion Paper Series from Global Labor Organization (GLO)

Abstract: Using simulated patients to mimic nine established non-communicable and infectious diseases over 27 trials, we assess ChatGPT's effectiveness and reliability in diagnosing and treating common diseases in low- and middle-income countries. We find ChatGPT's performance varied within a single disease, despite a high level of accuracy in both correct diagnosis (74.1%) and medication prescription (84.5%). Additionally, ChatGPT recommended a concerning level of unnecessary or harmful medications (85.2%) even with correct diagnoses. Finally, ChatGPT performed better in managing non-communicable diseases compared to infectious ones. These results highlight the need for cautious AI integration in healthcare systems to ensure quality and safety.

Keywords: ChatGPT; Large Language Models; Generative AI; Simulated Patient; Healthcare; Quality; Safety; Low- and Middle-Income Countries (search for similar items in EconPapers)
JEL-codes: C0 C90 I10 I11 (search for similar items in EconPapers)
Date: 2024
New Economics Papers: this item is included in nep-ain, nep-cmp and nep-hea
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.econstor.eu/bitstream/10419/300730/1/GLO-DP-1472.pdf (application/pdf)

Related works:
Working Paper: Quality and Accountability of Large Language Models (LLMs) in Healthcare in Low- And Middle-Income Countries (LMIC): A Simulated Patient Study Using ChatGPT (2024)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zbw:glodps:1472

Access Statistics for this paper

More papers in GLO Discussion Paper Series from Global Labor Organization (GLO) Contact information at EDIRC.
Bibliographic data for series maintained by ZBW - Leibniz Information Centre for Economics ().