An Empirical Audit of LLM Product Recommendations

Federico Boffa, Chiara Farronato (), Alessandro Fedele (), Eugenio Levi () and Andrea Mantovani
Additional contact information
Chiara Farronato: Harvard Business School, Boston, MA, USA
Alessandro Fedele: Faculty of Economics and Management, Free University of Bozen-Bolzano, Bolzano, Italy
Eugenio Levi: Link Campus University, Rome, Italy

No 26-09, Working Papers from NET Institute

Abstract: Large language models (LLMs) are increasingly used by consumers to search for information, compare alternatives, and obtain purchasing advice. As conversational AI systems progressively complement traditional search engines and recommender systems, they may influence consumer choice by determining which products receive visibility and attention. Despite their growing importance, little is known about how LLMs construct product recommendations, whether they concentrate attention on a narrow set of products, or whether vertically integrated providers favor affiliated products and services. This paper conducts an audit study of leading LLMs, including ChatGPT, Gemini, Claude, DeepSeek, Grok, and Mistral. Using standardized prompts and repeated queries, we collect recommendations across a broad range of products and service categories that differ in market characteristics and the presence of vertically integrated products. We characterize recommendations along several dimensions, including breadth, first-position concentration, prominence, best-overall selections, and personalization. We find substantial heterogeneity across models in the breadth and structure of their recommendations. At the same time, first-position recommendations are highly concentrated in a substantial subset of markets. In 20 of the 100 product and service categories, a single product is listed first in more than three quarters of responses. We also find that some LLMs are likely to recommend products affiliated with their providers.

Keywords: large language models; empirical audit; product recommendations; AI; algorithmic bias (search for similar items in EconPapers)
JEL-codes: D83 L15 L41 L86 (search for similar items in EconPapers)
Pages: 38 pages
Date: 2026-08
New Economics Papers: this item is included in nep-dcm
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.netinst.org/Levi_26-09.pdf (application/pdf)
no

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:net:wpaper:2609

Access Statistics for this paper

More papers in Working Papers from NET Institute
Bibliographic data for series maintained by Nicholas Economides ().