How unpredictable is research impact? Evidence from the UK’s Research Excellence Framework

Ohid Yaqub, Dmitry Malkov and Josh Siepel

No qw873, SocArXiv from Center for Open Science

Abstract: There is growing interest in the wider impact of research outside academia. In particular, there is concern about the extent to which the topic and principal beneficiaries of such impact can be foreseen before research is even funded. Does research impact tend to emerge largely as planned, or are eventual impacts unrecognisable from initial plans? To explore the unexpectedness of impact, we draw on one of the largest research impact assessment exercises in the world - the UK’s Research Excellence Framework. We exploit REF impact traced back to research funding applications, as a dataset of 2,194 case-grant pairs, to compare impact topics with funder remits. For 209 of those pairs, we directly compare their descriptions of ex-ante and ex-post impact. We found impact claims in these case-grant pairs are often congruent with each other, with 76% showing alignment. This indicates that, conditional on REF-inclusion, research delivers impact with largely the same orientation as envisaged at the outset of their projects. This is especially so when research is co-produced with focal stakeholders. These results offer a distinct complement to other preliminary studies on the REF, which have tended to emphasise the vast diversity and complexity of research impact. Our findings suggest that research impact is far from random, and that there is wide scope for policy intervention. In particular, we highlight the potentially important role that collaboration plays in setting the direction of research impact. However, the findings may also signal that the REF is preferentially excluding unpredicted impacts and indirect impacts. This selection effect, where a qualitatively different class of impacts are crowded out, may well be exacerbated as the share of REF rewards is weighted further towards the impact criterion.

Date: 2022-03-09
New Economics Papers: this item is included in nep-ppm and nep-sog
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://osf.io/download/6227aae7558e6006742a06de/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:socarx:qw873

DOI: 10.31219/osf.io/qw873

Access Statistics for this paper

More papers in SocArXiv from Center for Open Science
Bibliographic data for series maintained by OSF ().