Humans vs. large language models: Judgmental forecasting in an era of advanced AI

Mahdi Abolghasemi, Odkhishig Ganbold and Kristian Rotaru

International Journal of Forecasting, 2025, vol. 41, issue 2, 631-648

Abstract: This study investigates the forecasting accuracy of human experts versus large language models (LLMs) in the retail sector, particularly during standard and promotional sales periods. Utilizing a controlled experimental setup with 123 human forecasters and five LLMs—namely, ChatGPT-4, ChatGPT3.5, Bard, Bing, and Llama2—we evaluated forecasting precision through the absolute percentage error. Our analysis centered on the effect of the following factors on forecasters’ performance: the supporting statistical model (baseline and advanced), whether the product was on promotion, and the nature of external impact. The findings indicate that LLMs do not consistently outperform humans in forecasting accuracy and that advanced statistical forecasting models do not uniformly enhance the performance of either human forecasters or LLMs. Both human and LLM forecasters exhibited increased forecasting errors, particularly during promotional periods. Our findings call for careful consideration when integrating LLMs into practical forecasting processes.

Keywords: Judgmental forecasting; Large language models; Promotion; Sales forecasting; Forecast accuracy; Controlled experiment (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0169207024000700
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:intfor:v:41:y:2025:i:2:p:631-648

DOI: 10.1016/j.ijforecast.2024.07.003

Access Statistics for this article

International Journal of Forecasting is currently edited by R. J. Hyndman

More articles in International Journal of Forecasting from Elsevier
Bibliographic data for series maintained by Catherine Liu ().