Impacts Assessments without True Baselines: Assessing the Relative Effects of Training on the Performance of Water User Associations in Southern Tajikistan

Soumya Balasubramanya, Joseph P. G. Price and Theodore Horbulyk
Additional contact information
Soumya Balasubramanya: International Water Management Institute, 127 Sunil Mawatha, Pelawatte, Sri Lanka
Joseph P. G. Price: International Water Management Institute, 127 Sunil Mawatha, Pelawatte, Sri Lanka

Water Economics and Policy (WEP), 2018, vol. 04, issue 03, 1-28

Abstract: Conducting rigorous evaluations of whether the process of creating new institutions affects their performance of mandated duties presents several challenges. Not only is assignment to process often not random, but when the process of creating new institutions starts, outcomes and other performance-influencing covariates are not measurable because the yet-to-be created institutions are not functioning at baseline. This paper compares the performance of 74 ‘treated’ water user associations (WUAs) in Tajikistan that were created using a longer training process with 67 ‘control’ WUAs that were created using shorter training, to assess the impact of training on WUA performance of mandated duties. First, propensity scores were constructed to estimate the probabilities of being ‘treated’ by treatment status. These results guided the application of the difference-in-difference technique with right-hand side covariates in a context where field measures of outcomes and other performance-influencing covariates were made after the new institutions were created and functioning. The first measures were taken within 12–18 months of the new institutions being functional and the second measures were taken 24 months after the first. This choice of methods introduces a bias due to measurement error causing an underestimate of the treatment effects, while controlling for biases due to time-invariant and time-varying unobservables. An alternative method that only compared the differences in outcomes at a single point in time after the new institutions were created would have provided an inaccurate estimate of the effects of the intervention. This is a context in which methods such as synthetic controls are impossible to employ due to the nature of the intervention, other macroeconomic structural changes, and severe data restrictions. The methodology employed here generates evidence that, while biased toward generating an underestimate of effect, can still be useful and informative for policy and management purposes, and for evaluating the impact of process on the functioning of new institutions in transition settings.

Keywords: Water user associations; performance; difference-in-difference; propensity scores; Tajikistan (search for similar items in EconPapers)
Date: 2018
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (4)

Downloads: (external link)
http://www.worldscientific.com/doi/abs/10.1142/S2382624X18500078
Access to full text is restricted to subscribers

Related works:
Journal Article: Impacts assessments without true baselines: assessing the relative effects of training on the performance of water user associations in southern Tajikistan (2018)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wsi:wepxxx:v:04:y:2018:i:03:n:s2382624x18500078

Ordering information: This journal article can be ordered from

DOI: 10.1142/S2382624X18500078

Access Statistics for this article

Water Economics and Policy (WEP) is currently edited by Ariel Dinar

More articles in Water Economics and Policy (WEP) from World Scientific Publishing Co. Pte. Ltd.
Bibliographic data for series maintained by Tai Tone Lim ().