What can we learn from sharing experience about evaluation practices?

Jean-Sébastien Broc (), Gregor Thenius, Dario Di Santo, Barbara Schlomann (), Barbara Breitschopf, Jamilja van Der Meulen, Paul van den Oosterkamp, Lovorko Marić and Marko Matosović
Additional contact information
Jean-Sébastien Broc: IEECP - Institute for European Energy and Climate Policy
Gregor Thenius: Austrian Energy Agency
Dario Di Santo: FIRE - FIRE (Italian Federation for energy efficiency)
Barbara Schlomann: Fraunhofer ISI - Fraunhofer Institute for Systems and Innovation Research - Fraunhofer-Gesellschaft - Fraunhofer
Barbara Breitschopf: Fraunhofer ISI - Fraunhofer Institute for Systems and Innovation Research - Fraunhofer-Gesellschaft - Fraunhofer
Jamilja van Der Meulen: TNO - The Netherlands Organisation for Applied Scientific Research
Paul van den Oosterkamp: TNO - The Netherlands Organisation for Applied Scientific Research
Lovorko Marić: EIHP - Energy Institute Hrvoje Požar
Marko Matosović: EIHP - Energy Institute Hrvoje Požar

Post-Print from HAL

Abstract: Many resources are available about state-of-the-art or best examples of evaluations and evaluation guidelines, like in the IEPEC and IEPPEC proceedings. However, what do we know about daily evaluation practices? Is it always easy to find examples of evaluations about a given country, type of policy instrument, etc.? Are evaluations used to improve policies? And finally, how could evaluation practices be improved? This is the kind of issues that the European project EPATEE aims at tackling, focusing on impact evaluations. About 60 stakeholders from 13 European countries were interviewed or surveyed to better know their priorities about evaluation issues, how they would define the level of evaluation practices in their country and which barriers might impede effective evaluation practices. In parallel, references were collected and coded to build a knowledge base gathering already more than 170 evaluation reports, papers or guidebooks. Case studies are analyzing more than 20 evaluations to provide concrete and detailed experience feedback about why evaluation is used, how it is performed, what difficulties are encountered, etc. The objective of the project is not to provide an exhaustive or representative picture of the evaluation practices in Europe, but to gather and develop materials that can be used as a basis for experience sharing activities, as well as to develop an online tool box that will make these resources available in a user-friendly way. The key assumption of the project is that concrete examples and guidance can help overcome barriers that currently limit the use of evaluation. Stakeholders' feedback indeed shows that evaluation can be a very effective tool to improve policies, thereby achieving higher energy savings at lower costs. This paper presents the results of the first phase of the project, focusing on the main conclusions from the stakeholders' survey, the knowledge base, and the case studies. Feedbacks gathered remind usual no-brainers (e.g., anticipating data collection). It also shows that evaluation is not only a technical issue but that organizational issues (e.g., cooperation between institutions) are critical as well. Learning by doing can help tackle some of the issues (e.g., optimizing data collection), but some issues remain difficult challenges (e.g., getting robust results about net impacts).

Date: 2018-06-25
Note: View the original document on HAL open archive server: https://hal.science/hal-02425109
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Published in IEPPEC 2018 (International Energy Policy & Programme Evaluation Conference), IEPPEC, Jun 2018, Vienna, Austria

Downloads: (external link)
https://hal.science/hal-02425109/document (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:hal:journl:hal-02425109

Access Statistics for this paper

More papers in Post-Print from HAL
Bibliographic data for series maintained by CCSD ().