Does LLM Assistance Improve Healthcare Delivery ? An Evaluation Using On-Site Physicians and Laboratory Tests

Jason Abaluck, Robert Pless, Nirmal Ravi, Anja Sautmann and Aaron Schwartz

No 11298, Policy Research Working Paper Series from The World Bank

Abstract: This study tests the effects of large language model (LLM) decision support on patient care at two outpatient clinics in Nigeria. Health workers were given the option to make revisions to their initial care plan based on LLM feedback. The unassisted and assisted plans are evaluated using (1) comparisons with independent care plans created by on-site physicians, (2) laboratory tests for malaria, anemia, and urinary tract infections, and (3) a blinded randomized assessment by the on-site physician who saw the same patient. In response to LLM feedback, health workers changed their prescribing for more than half of the patients and reported high satisfaction with the recommendations. In a selected sample, retrospective review by academic physicians also suggested improvements in care related to long-term risk management. However, the three metrics show mixed effects of LLM-assistance, with on average no significant improvement in diagnostic alignment with physicians, detection rates for the tested conditions, or physician subjective assessments. Health workers follow LLM recommendations that agree with the physician's decisions only slightly more often than those that do not. These results suggest that, despite some benefits, LLM-based frontline health worker support is not yet a public health priority in low- and middle-income countries.

Date: 2026-01-15
References: Add references at CitEc
Citations:

Downloads: (external link)
https://documents.worldbank.org/curated/en/0998145 ... dbc-c23487ab9955.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wbk:wbrwps:11298

Access Statistics for this paper

More papers in Policy Research Working Paper Series from The World Bank 1818 H Street, N.W., Washington, DC 20433. Contact information at EDIRC.
Bibliographic data for series maintained by Roula I. Yazigi ().