 Incorporating public values into evaluative criteria: Using crowdsourcing to identify criteria and standardsElena Harman and Tarek Azzam Evaluation and Program Planning, 2018, vol. 71, issue C, 68-82 Abstract: At its core, evaluation involves the generation of value judgments. These evaluative judgments are based on comparing an evaluand’s performance to what the evaluand is supposed to do (criteria) and how well it is supposed to do it (standards). The aim of this four-phase study was to test whether criteria and standards can be set via crowdsourcing, a potentially cost- and time-effective approach to collecting public opinion data. In the first three phases, participants were presented with a program description, then asked to complete a task to either identify criteria (phase one), weigh criteria (phase two), or set standards (phase three). Phase four found that the crowd-generated criteria were high quality; more specifically, that they were clear and concise, complete, non-overlapping, and realistic. Overall, the study concludes that crowdsourcing has the potential to be used in evaluation for setting stable, high-quality criteria and standards. Keywords: Crowdsourcing; Criteria; Standards; Value Judgements; Evaluation-Specific Methodology; Stability; Mechanical Turk; MTurk (search for similar items in EconPapers) Downloads: (external link) Related works: Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text Persistent link: https://EconPapers.repec.org/RePEc:eee:epplan:v:71:y:2018:i:c:p:68-82 DOI: 10.1016/j.evalprogplan.2018.08.004 Access Statistics for this article Evaluation and Program Planning is currently edited by Jonathan A. Morell More articles in Evaluation and Program Planning from Elsevier |
Is your work missing from RePEc? Here is how to contribute.
Questions or problems? Check the EconPapers FAQ or send mail to .
EconPapers is hosted by the
School of Business at Örebro University.