Measuring subgroup preferences in conjoint experiments

Thomas J. Leeper, Sara Hobolt and James Tilley

LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library

Abstract: Conjoint analysis is a common tool for studying political preferences. The method disentangles patterns in respondents' favorability toward complex, multidimensional objects, such as candidates or policies. Most conjoints rely upon a fully randomized design to generate average marginal component effects (AMCEs). They measure the degree to which a given value of a conjoint profile feature increases, or decreases, respondents' support for the overall profile relative to a baseline, averaging across all respondents and other features. While the AMCE has a clear causal interpretation (about the effect of features), most published conjoint analyses also use AMCEs to describe levels of favorability. This often means comparing AMCEs among respondent subgroups. We show that using conditional AMCEs to describe the degree of subgroup agreement can be misleading as regression interactions are sensitive to the reference category used in the analysis. This leads to inferences about subgroup differences in preferences that have arbitrary sign, size, and significance. We demonstrate the problem using examples drawn from published articles and provide suggestions for improved reporting and interpretation using marginal means and an omnibus F-test. Given the accelerating use of these designs in political science, we offer advice for best practice in analysis and presentation of results.

Keywords: conjoint analysis; survey experiments; factorial experiments; survey design (search for similar items in EconPapers)
JEL-codes: C1 (search for similar items in EconPapers)
Pages: 15 pages
Date: 2020-04-01
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (31)

Published in Political Analysis, 1, April, 2020, 28(2), pp. 207 - 221. ISSN: 1047-1987

Downloads: (external link)
http://eprints.lse.ac.uk/100944/ Open access version. (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ehl:lserod:100944

Access Statistics for this paper

More papers in LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library LSE Library Portugal Street London, WC2A 2HD, U.K.. Contact information at EDIRC.
Bibliographic data for series maintained by LSERO Manager ().