Using a Satisficing Model of Experimenter Decision-Making to Guide Finite-Sample Inference for Compromised Experiments

Ganesh Karapakula and James Heckman

No 2020-063, Working Papers from Human Capital and Economic Opportunity Working Group

Abstract: This paper presents a simple decision-theoretic economic approach for analyzing social experiments with compromised random assignment protocols that are only partially documented. We model administratively constrained experimenters who satisfice in seeking covariate balance. We develop design-based small-sample hypothesis tests that use worst-case (least favorable) randomization null distributions. Our approach accommodates a variety of compromised experiments, including imperfectly documented re-randomization designs. To make our analysis concrete, we focus much of our discussion on the influential Perry Preschool Project. We reexamine previous estimates of program effectiveness using our methods. The choice of how to model reassignment vitally affects inference.

Keywords: randomized controlled trial; randomization tests; worst-case inference; least favorable null distributions; partial identification; small-sample hypothesis testing (search for similar items in EconPapers)
JEL-codes: C10 C40 I21 (search for similar items in EconPapers)
Date: 2020-08
New Economics Papers: this item is included in nep-exp and nep-ore
Note: ECI
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://humcap.uchicago.edu/RePEc/hka/wpaper/Karapa ... -decision-making.pdf First version, August 10, 2020 (application/pdf)
http://humcap.uchicago.edu/RePEc/hka/wpaper/Karapa ... -making_APPENDIX.pdf Appendix (application/pdf)

Related works:
Journal Article: Using a satisficing model of experimenter decision-making to guide finite-sample inference for compromised experiments (2021)
Working Paper: Using a Satisficing Model of Experimenter Decision-Making to Guide Finite-Sample Inference for Compromised Experiments (2020)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:hka:wpaper:2020-063

Access Statistics for this paper

More papers in Working Papers from Human Capital and Economic Opportunity Working Group Contact information at EDIRC.
Bibliographic data for series maintained by Jennifer Pachon ().