We Need to Talk: Audio Surveys and Information Extraction

Vincenzo Galasso, Tommaso Nannicini and Debora Nozza

No 11530, CESifo Working Paper Series from CESifo

Abstract: Understanding individuals’ beliefs, preferences, and motivations is essential in social sciences. Recent technological advancements—notably, large language models (LLMs) for analyzing open-ended responses and the diffusion of voice messaging—have the potential to significantly enhance our ability to elicit these dimensions. This study investigates the differences between oral and written responses to open-ended survey questions. Through a series of randomized controlled trials across three surveys (focused on AI, public policy, and international relations), we assigned respondents to answer either by audio or text. Respondents who provided audio answers gave longer, though lexically simpler, responses compared to those who typed. By leveraging LLMs, we evaluated answer informativeness and found that oral responses differ in both quantity and quality, offering more information and containing more personal experiences than written responses. These findings suggest that oral responses to open-ended questions can capture richer, more personal insights, presenting a valuable method for understanding individual reasoning.

Keywords: survey design; open-ended questions; Large Language Models; beliefs (search for similar items in EconPapers)
JEL-codes: C83 D83 (search for similar items in EconPapers)
Date: 2024
New Economics Papers: this item is included in nep-exp
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.cesifo.org/DocDL/cesifo1_wp11530.pdf (application/pdf)

Related works:
Working Paper: We Need to Talk: Audio Surveys and Information Extraction (2024)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ces:ceswps:_11530

Access Statistics for this paper

More papers in CESifo Working Paper Series from CESifo Contact information at EDIRC.
Bibliographic data for series maintained by Klaus Wohlrabe ().