A checklist to guide sensitivity analyses and replications of impact evaluations

Sridevi K. Prasad, Fiona Kastel, Suvarna Pande, Chenxiao Zhang and Douglas M. Glandon

Journal of Development Effectiveness, 2024, vol. 16, issue 3, 332-348

Abstract: The transparency, reproducibility, and ethics movement has helped promote efforts to improve the rigour and validity of social science research. However, replication research, where the original data is used to check the validity and robustness of the estimations and recommendations in the original paper, has not been widely used as a tool in this field. This article outlines a checklist and standardised process that can guide replication researchers in creating and justifying their replication plans. To create the checklist, we reviewed 31 resources that were identified through unstructured keyword searches. We also reviewed and coded replication studies from 3ie’s Replication Program which includes more than 20 3ie-funded replication papers and guidance on our replication process. There were five categories of potential validation tests and robustness checks that researchers could include. The checklist attributes range from testing specific assumptions for select impact evaluation methods to data transformations or dealing with heterogeneity. We hope that this checklist helps stimulate demand for replication research and promotes the use of replication research as a tool within transparency, reproducibility and ethics.

Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/19439342.2024.2318695 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:jdevef:v:16:y:2024:i:3:p:332-348

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/RJDE20

DOI: 10.1080/19439342.2024.2318695

Access Statistics for this article

Journal of Development Effectiveness is currently edited by Howard White

More articles in Journal of Development Effectiveness from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().