Often Trusted but Never (Properly) Tested: Evaluating Qualitative Comparative Analysis

Michael Baumgartner and Alrik Thiem

Sociological Methods & Research, 2020, vol. 49, issue 2, 279-311

Abstract: To date, hundreds of researchers have employed the method of Qualitative Comparative Analysis (QCA) for the purpose of causal inference. In a recent series of simulation studies, however, several authors have questioned the correctness of QCA in this connection. Some prominent representatives of the method have replied in turn that simulations with artificial data are unsuited for assessing QCA. We take issue with either position in this impasse. On the one hand, we argue that data-driven evaluations of the correctness of a procedure of causal inference require artificial data. On the other hand, we prove all previous attempts in this direction to have been defective. For the first time in the literature on configurational comparative methods, we lay out a set of formal criteria for an adequate evaluation of QCA before implementing a battery of inverse-search trials to test how this method performs in different recovery contexts according to these criteria. While our results indicate that QCA is correct when generating the parsimonious solution type, they also demonstrate that the method is incorrect when generating the conservative and intermediate solution type. In consequence, researchers using QCA for causal inference, particularly in human-sensitive areas such as public health and medicine, should immediately discontinue employing the method’s conservative and intermediate search strategies.

Keywords: Qualitative Comparative Analysis; method evaluation; set theoretic methods; Boolean algebra; causal inference; configurational comparative methods (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (19)

Downloads: (external link)
https://journals.sagepub.com/doi/10.1177/0049124117701487 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:somere:v:49:y:2020:i:2:p:279-311

DOI: 10.1177/0049124117701487

Access Statistics for this article

More articles in Sociological Methods & Research
Bibliographic data for series maintained by SAGE Publications ().