Re-fielding Replicability of Silicon Sample-based Marketing Research

Steffen Jahn, Tobias Klinke, Daniel Guhl and Maja Murr
Additional contact information
Steffen Jahn: Martin Luther University Halle-Wittenberg
Tobias Klinke: experial Research Lab
Daniel Guhl: HU Berlin
Maja Murr: HU Berlin

No 583, Rationality and Competition Discussion Paper Series from CRC TRR 190 Rationality and Competition

Abstract: Large language models are increasingly used to generate silicon samples, synthetic respondents that stand in for human participants in behavioral research. As silicon samples enter the methodological mainstream, the question of their re-fielding replicability becomes critical: when researchers regenerate a silicon sample under the same procedure at a later point in time, how stable are the resulting data? Existing validation studies have compared LLM-generated responses to human responses at a single point in time, but the temporal consistency of silicon samples themselves (whether independent reapplications of the same generation procedure yield comparable samples) has not been examined systematically, especially under sparse-conditioning, re-fielded sample-generation settings. We address this gap by generating silicon samples across three weekly waves, permitting a re-fielding consistency assessment. We benchmark this assessment with a human reference sample recruited via Prolific. Moreover, we systematically compare three structurally distinct generation paradigms that try to increase response variability: single-prompt batch generation, iterative generation with cumulative memory, and two-stage seed-and-expand generation. We find that re-fielding consistency is reasonably high in aggregate and comparable to the human benchmark for the strongest paradigm (two-stage seed-and-expand) but markedly lower for single-prompt batch generation. In contrast, alignment with the human reference is generally low, so that silicon samples reproduce their own outputs more faithfully than they reproduce human data. This means that silicon samples can be internally reproducible without being externally valid.

Keywords: silicon samples; synthetic data; replicability; consistency; sustainability; plant-based meat (search for similar items in EconPapers)
JEL-codes: C45 C52 C83 D12 M31 Q56 (search for similar items in EconPapers)
Date: 2026-08-05
References: Add references at CitEc
Citations:

Downloads: (external link)
https://rationality-and-competition.de/wp-content/ ... ussion_paper/583.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:rco:dpaper:583

Access Statistics for this paper

More papers in Rationality and Competition Discussion Paper Series from CRC TRR 190 Rationality and Competition
Bibliographic data for series maintained by Viviana Lalli ().