Bottlenecks for Evidence Adoption

Stefano DellaVigna, Woojin Kim and Elizabeth Linos

No 30144, NBER Working Papers from National Bureau of Economic Research, Inc

Abstract: Governments increasingly use RCTs to test innovations before scale up. Yet, we know little about whether and how they incorporate the results of the experiments into policy-making. We follow up with 67 U.S. city departments which collectively ran 73 RCTs in collaboration with a national Nudge Unit. Compared to most contexts, the barriers to adoption are low. Yet, city departments adopt a nudge treatment in follow-on communication in 27% of cases. As potential determinants of adoption we consider (i) the strength of the evidence, as determined by the RCT itself, (ii) features of the organization, such as “organizational capacity” of the city and whether the city staff member working on the RCT has been retained, and (iii) the experimental design, such as whether the RCT was implemented as part of pre-existing communication. We find (i) a limited impact of strength of the evidence and (ii) some impact of city features, especially the retention of the original staff member. By far, the largest predictor of adoption is (iii) whether the communication was pre-existing, as opposed to a new communication. We consider two main interpretations of this finding: organizational inertia, in that changes to pre-existing communications are more naturally folded into year-to-year city processes, and costs, since new communications may require additional funding. We find the same pattern for electronic communications, with zero marginal costs, supporting the organizational inertia explanation. The pattern of results differs from the predictions of both experts and practitioners, who over-estimate the extent of evidence-based adoption. Our results underline the importance of considering the barriers to evidence adoption, beginning at the stage of experimental design and continuing after the RCT completion.

JEL-codes: D72 D9 H1 H70 (search for similar items in EconPapers)
Date: 2022-06
New Economics Papers: this item is included in nep-dem and nep-exp
Note: CH DEV ED EH LE LS PE POL PR
References: Add references at CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://www.nber.org/papers/w30144.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nbr:nberwo:30144

Ordering information: This working paper can be ordered from
http://www.nber.org/papers/w30144

Access Statistics for this paper

More papers in NBER Working Papers from National Bureau of Economic Research, Inc National Bureau of Economic Research, 1050 Massachusetts Avenue Cambridge, MA 02138, U.S.A.. Contact information at EDIRC.
Bibliographic data for series maintained by ().