Frontiers: Determining the Validity of Large Language Models for Automated Perceptual Analysis

Peiyao Li (), Noah Castelo (), Zsolt Katona () and Miklos Sarvary ()
Additional contact information
Peiyao Li: Haas School of Business, University of California, Berkeley, California 94720
Noah Castelo: Alberta School of Business, University of Alberta, Edmonton, Alberta T6G 2R6, Canada
Zsolt Katona: Haas School of Business, University of California, Berkeley, California 94720
Miklos Sarvary: Columbia Business School, Columbia University, New York, New York 10027

Marketing Science, 2024, vol. 43, issue 2, 254-266

Abstract: This paper explores the potential of large language models (LLMs) to substitute for human participants in market research. Such LLMs can be used to generate text given a prompt. We argue that perceptual analysis is a particularly promising use case for such automated market research for certain product categories. The proposed new method generates outputs that closely match those generated from human surveys: agreement rates between human- and LLM- generated data sets reach over 75%. Moreover, this applies for perceptual analysis based on both brand similarity measures and product attribute ratings. The paper demonstrates that, for some categories, this new method of fully or partially automated market research will increase the efficiency of market research by meaningfully speeding up the process and potentially reducing the cost. Further results also suggest that with an ever larger training corpus applied to large language models, LLM-based market research will be applicable to answer more nuanced questions based on demographic variables or contextual variation that would be prohibitively expensive or infeasible with human respondents.

Keywords: artificial Intelligence; perceptual maps; large language model; natural language processing; market research (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (10)

Downloads: (external link)
http://dx.doi.org/10.1287/mksc.2023.0454 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ormksc:v:43:y:2024:i:2:p:254-266

Access Statistics for this article

More articles in Marketing Science from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().