Measuring Stakeholder Participation in Evaluation

Pierre-Marc Daigneault, Steve Jacob and Joël Tremblay

Evaluation Review, 2012, vol. 36, issue 4, 243-271

Abstract: Background: Stakeholder participation is an important trend in the field of program evaluation. Although a few measurement instruments have been proposed, they either have not been empirically validated or do not cover the full content of the concept. Objectives: This study consists of a first empirical validation of a measurement instrument that fully covers the content of participation, namely the Participatory Evaluation Measurement Instrument (PEMI). It specifically examines (1) the intercoder reliability of scores derived by two research assistants on published evaluation cases; (2) the convergence between the scores of coders and those of key respondents (i.e., authors); and (3) the convergence between the authors’ scores on the PEMI and the Evaluation Involvement Scale (EIS). Sample: A purposive sample of 40 cases drawn from the evaluation literature was used to assess reliability. One author per case in this sample was then invited to participate in a survey; 25 fully usable questionnaires were received. Measures: Stakeholder participation was measured on nominal and ordinal scales. Cohen’s κ, the intraclass correlation coefficient, and Spearman’s Ï were used to assess reliability and convergence. Results: Reliability results ranged from fair to excellent. Convergence between coders’ and authors’ scores ranged from poor to good. Scores derived from the PEMI and the EIS were moderately associated. Conclusions: Evidence from this study is strong in the case of intercoder reliability and ranges from weak to strong in the case of convergent validation. Globally, this suggests that the PEMI can produce scores that are both reliable and valid.

Keywords: participatory evaluation; stakeholder involvement; stakeholder participation; collaborative inquiry; measurement instrument; empirical validation (search for similar items in EconPapers)
Date: 2012
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.sagepub.com/doi/10.1177/0193841X12458103 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:evarev:v:36:y:2012:i:4:p:243-271

DOI: 10.1177/0193841X12458103

Access Statistics for this article

More articles in Evaluation Review
Bibliographic data for series maintained by SAGE Publications ().