Understanding and Misunderstanding Randomized Controlled Trials

Angus Deaton and Nancy Cartwright

No 22595, NBER Working Papers from National Bureau of Economic Research, Inc

Abstract: RCTs would be more useful if there were more realistic expectations of them and if their pitfalls were better recognized. For example, and contrary to many claims in the applied literature, randomization does not equalize everything but the treatment across treatments and controls, it does not automatically deliver a precise estimate of the average treatment effect (ATE), and it does not relieve us of the need to think about (observed or unobserved) confounders. Estimates apply to the trial sample only, sometimes a convenience sample, and usually selected; justification is required to extend them to other groups, including any population to which the trial sample belongs. Demanding “external validity” is unhelpful because it expects too much of an RCT while undervaluing its contribution. Statistical inference on ATEs involves hazards that are not always recognized. RCTs do indeed require minimal assumptions and can operate with little prior knowledge. This is an advantage when persuading distrustful audiences, but it is a disadvantage for cumulative scientific progress, where prior knowledge should be built upon and not discarded. RCTs can play a role in building scientific knowledge and useful predictions but they can only do so as part of a cumulative program, combining with other methods, including conceptual and theoretical development, to discover not “what works,” but “why things work”.

JEL-codes: C10 C26 C93 O22 (search for similar items in EconPapers)
Date: 2016-09
New Economics Papers: this item is included in nep-hpe, nep-pke and nep-sog
Note: AG DEV EH LS
References: Add references at CitEc
Citations: View citations in EconPapers (47)

Published as Angus Deaton & Nancy Cartwright, 2017. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, .

Downloads: (external link)
http://www.nber.org/papers/w22595.pdf (application/pdf)

Related works:
Journal Article: Understanding and misunderstanding randomized controlled trials (2018)
Working Paper: Understanding and misunderstanding randomized controlled trials (2017)
Working Paper: Understanding and Misunderstanding Randomized Controlled Trials (2016)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nbr:nberwo:22595

Ordering information: This working paper can be ordered from
http://www.nber.org/papers/w22595

Access Statistics for this paper

More papers in NBER Working Papers from National Bureau of Economic Research, Inc National Bureau of Economic Research, 1050 Massachusetts Avenue Cambridge, MA 02138, U.S.A.. Contact information at EDIRC.
Bibliographic data for series maintained by ().