Understanding the use and usability of research evaluation studies1,2

Katrin Milzow, Anke Reinhardt, Sten Söderberg and Klaus ZinöckerDeceased

Research Evaluation, 2019, vol. 28, issue 1, 94-107

Abstract: Many research funding and research performing organizations evaluate their operations or are subject to evaluations. A Working Group within Science Europe undertook to illustrate how ex post evaluation of research is actually used. Based on 11 case studies from seven organizations and countries, we explore the use of evaluation studies commissioned or conducted by member organizations. We draw on a theoretical framework that categorizes types of evaluation use, including legitimation, information, learning, steering, mediating, tactical, and ritual use. To explore the factors influencing the use of evaluations, we draw on a ‘logic model of evaluation use’ developed by Mark and Henry (2004). The study shows that many uses of evaluation studies can (co-)exist. In particular, direct influence on decision-making (steering use) is not the only type of use that practitioners see in their daily work. Equally important types of use include contribution to the knowledge base and cognitive resources within organizations and beyond. Our study highlights that a methodologically rigorous and high-quality evaluation study is a necessary, but not sufficient, condition to use evaluation results. It draws attention to the fact that the context and other factors external to the evaluation itself—but often within the realm of the commissioning organization—determine the actual use of the evaluation. The findings suggest applying logic models not only to evaluation objects but also to the evaluation exercise itself, so as to consider the underlying mechanisms and chains of influence through which evaluation may have its effects.

Keywords: logic model; ex post evaluation; research funding; stakeholder; involvement; evaluation influence (search for similar items in EconPapers)
Date: 2019
References: View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://hdl.handle.net/10.1093/reseval/rvy040 (application/pdf)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:oup:rseval:v:28:y:2019:i:1:p:94-107.

Access Statistics for this article

Research Evaluation is currently edited by Julia Melkers, Emanuela Reale and Thed van Leeuwen

More articles in Research Evaluation from Oxford University Press
Bibliographic data for series maintained by Oxford University Press ().