The External Validity of College Student Subject Pools in Experimental Research: A Cross-Sample Comparison of Treatment Effect Heterogeneity
Danielle L. Lupton
Political Analysis, 2019, vol. 27, issue 1, 90-97
Are student subject experiment pools comparable across institutions? Despite repeated concerns over the â€œcollege sophomore problem,â€ many experiment-based studies still rely on student subject pools due to their convenience and accessibility. In this paper, I investigate whether student subject pools are comparable across universities by examining how respondents across three student subject pools at distinct educational institutions perform on the same survey experiment about crisis bargaining between states. I argue that, due to selection biases inherent in university matriculation and the self-selection of students into experimental protocols, respondents across these subject pools will exhibit key demographic differences. I also examine whether respondents across these subject pools think similarly about international politics and respond comparably to experimental treatments. I find that, while there are significant demographic differences across subject pools, subjects across institutions respond similarly to experimental treatmentsâ€”with the key exception of information regarding the regime type of a state. Furthermore, there is little evidence that these demographic differences impact conditional average treatment effects across subgroups. These findings carry critical implications for the use of student samples across political science and within international relations more specifically, particularly regarding the current replication crisis in the discipline.
References: Add references at CitEc
Citations: Track citations by RSS feed
Downloads: (external link)
https://www.cambridge.org/core/product/identifier/ ... type/journal_article link to article abstract page (text/html)
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
Persistent link: https://EconPapers.repec.org/RePEc:cup:polals:v:27:y:2019:i:01:p:90-97_00
Access Statistics for this article
More articles in Political Analysis from Cambridge University Press Cambridge University Press, UPH, Shaftesbury Road, Cambridge CB2 8BS UK.
Bibliographic data for series maintained by Keith Waters ().