Does the inclusion of non-academic reviewers make any difference for grant impact panels?

Understanding the Long Term Impact of the Framework Programme, European Policy Evaluation Consortium (EPEC) Report

Junwen Luo, Lai Ma and Kalpana Shankar

Science and Public Policy, 2021, vol. 48, issue 6, 763-775

Abstract: Broader impact of scientific research beyond academia has become increasingly important in research evaluation. To evaluate broader impact of research proposals, some funding agencies compose mixed panels that include peer experts and non-academic stakeholders. Whether and how non-academic reviewers bring any difference to panel discussions has been understudied. We analysed 164 review reports (2014–6) from the Investigators Programme (funding Science, Technology, Engineering, and Mathematics fields) at Science Foundation Ireland, where two types of panels, with and without non-academics, were composed for impact assessments. We find that the mixed panel reviews were longer and touched upon broader and more concrete impact topics. Also, mixed panels commented on causality and attribution of impact towards characteristics of applicants and research process more than scientific excellence. A survey of the same reviewer pool supplements our understanding of the pros and cons of the inclusion of non-academic reviewers. We discuss some policy recommendations for funding agencies to organise review panels.

Keywords: grant evaluation; peer review; societal impact; panel discussion; criteria; user (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://hdl.handle.net/10.1093/scipol/scab046 (application/pdf)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:oup:scippl:v:48:y:2021:i:6:p:763-775.

Access Statistics for this article

Science and Public Policy is currently edited by Nicoletta Corrocher, Jeong-Dong Lee, Mireille Matt and Nicholas Vonortas

More articles in Science and Public Policy from Oxford University Press
Bibliographic data for series maintained by Oxford University Press ().