Fine-Tuning Large Language Models to Simulate German Voting Behaviour (Working Paper)

Tobias Holtdirk, Dennis Assenmacher, Arnim Bleier and Claudia Wagner

No udz28, OSF Preprints from Center for Open Science

Abstract: Surveys are a cornerstone of empirical social science research, providing invaluable insights into the opinions, beliefs, behaviours, and characteristics of people. However, issues such as refusal to participate, skipping questions, sampling bias, and attrition significantly impact the quality and reliability of survey data. Recently, researchers have started investigating the potential of Large Language Models (LLMs) to role-play a pre-defined set of "characters" and simulate their survey responses with little or no additional training data and costs. While previous research on forecasting, imputing, and simulating survey answers with LLMs has focused on zero-shot and few-shot approaches, this study investigates the viability of fine-tuning LLMs to simulate responses of survey participants. We fine-tune Large Language Models (LLMs) on subsets of the data from the German Longitudinal Election Study (GLES) and evaluate their predictive performance on the "vote choice" for a random set of held-out participants. We compare the LLMs' performance against various baseline methods. Our findings show that small, fine-tuned open-source LLMs can outperform zero-shot predictions of larger LLMs. They are able to match the performance of established tabular data classifiers, are more sample efficient, and outperform them in cases with systematic non-responses. This study contributes to the growing body of research on LLMs for simulating survey data by demonstrating the effectiveness of fine-tuning approaches.

Date: 2024-10-07
New Economics Papers: this item is included in nep-ain, nep-big and nep-cmp
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://osf.io/download/6702c880f240037f8fa2009a/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:osfxxx:udz28

DOI: 10.31219/osf.io/udz28

Access Statistics for this paper

More papers in OSF Preprints from Center for Open Science
Bibliographic data for series maintained by OSF ().