Counting Defiers: A Design-Based Model of an Experiment Can Reveal Evidence Beyond the Average Effect

Neil Christy and Amanda Kowalski

Papers from arXiv.org

Abstract: We estimate the numbers of always takers, compliers, defiers, and never takers in the sample of people in an experiment, rather than a hypothetical population from which it was drawn, using structure from the randomization design. Our data include only a binary intervention and outcome. We develop a visualization to show that samples with defiers can sometimes generate the data in more ways than samples without defiers, yielding a higher design-based likelihood. We propose a maximum likelihood decision rule that can harness this evidence, which is not captured by standard hypothesis tests, and we provide optimality conditions. We illustrate the output of our decision rule for all possible data in samples of 50 and 200 with half in intervention, demonstrating a pattern in when the MLE includes defiers despite a positive average effect. We provide insights into effect heterogeneity in two published experiments with interventions that could plausibly backfire for some people despite statistically significant positive average effects on takeup of desirable health behaviors. In both, our 95% credible sets include the estimated Frechet bounds, demonstrating that evidence is weak. Yet, our MLE includes no defiers in one; in the other, the MLE includes a count of defiers equal to the estimated upper Frechet bound, over 18% of the sample. The MLE can support a monotonicity assumption or a specific alternative as a step toward improving the average effect of future interventions by targeting them away from noncompliers. Our dbmle package, compatible with Python and Stata, implements our statistics.

Date: 2024-12, Revised 2025-12
New Economics Papers: this item is included in nep-ecm, nep-exp, nep-hea and nep-upt
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://arxiv.org/pdf/2412.16352 Latest version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:arx:papers:2412.16352

Access Statistics for this paper

More papers in Papers from arXiv.org
Bibliographic data for series maintained by arXiv administrators ().