How accurate are retrospective survey data? Evidence from rural Ethiopia

Rob Fuller, Simone Lombardini and Cecilia Poggi

Journal of Development Effectiveness, 2024, vol. 16, issue 1, 142-158

Abstract: Development actors are always seeking reliable and cost-effective methods to assess the impact of their programmes. In particular, there are frequently calls to evaluate programmes for which no pre-intervention (or ‘baseline’) data are available. In these cases, evaluators often rely on retrospective survey questions to reconstruct the baseline situation. This article explores the accuracy of such retrospective survey data, using data from two surveys carried out nearly six years apart among women in rural Ethiopia. We find that the proportion of survey items for which baseline data and retrospective data do not agree is 22%. Responses to the retrospective questions are more closely associated with respondents’ situation at the time of the survey than with their situation at the time they were being asked to recall. Consequently, 72% of respondents were allocated to different quintiles of household wealth, depending on whether the true baseline or the retrospective baseline data were used. We show that controlling for retrospective baseline data can considerably underestimate the impact of the intervention being evaluated. This suggests that there is a need for caution in interpreting the findings of evaluations based on such data and in drawing policy conclusions from them.

Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/19439342.2023.2190602 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:jdevef:v:16:y:2024:i:1:p:142-158

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/RJDE20

DOI: 10.1080/19439342.2023.2190602

Access Statistics for this article

Journal of Development Effectiveness is currently edited by Howard White

More articles in Journal of Development Effectiveness from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().