The 2015–2019 VQR, Italy’s Third National Research Assessment: an intended informed peer review or a DIY bibliometric exercise?

Ciriaco Andrea D’Angelo

Research Evaluation, 2025, vol. 34

Abstract: Aiming to enhance university performance and maximize public investment returns, many governments conduct national research assessment exercises to allocate core funding based on research quality, in turn measured by ‘informed peer review’, i.e. combining quantitative indicators with peer reviews. This paper examines Italy’s latest VQR, intended to be based on such an approach, in STEMM fields. We analyze the final report of the 2015–9 VQR exercise and metadata of products submitted by universities to address key questions about the role of metrics in informing peer reviewers. In STEMM, 99% of submitted research products are indexed in bibliometric repositories; the few products not indexed in WoS/Scopus obtained very low quality scores. On average, each reviewer evaluated the originality, methodological rigor and impact of 600 products. Correlating the VQR outcomes with those of a similar purely bibliometric exercise based on an experimental dataset reveals the predictability of the outcomes of the ‘official’ exercise, although ‘locally’ there are important differences between the two rankings. We provide insights for reflections that are all the more urgent, given that the next Italian research evaluation exercise is already underway.

Keywords: research evaluation; research performance assessment; peer review; bibliometrics; universities; Italy (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1093/reseval/rvaf045 (application/pdf)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:oup:rseval:v:34:y:2025:i::p:rvaf045.

Access Statistics for this article

Research Evaluation is currently edited by Julia Melkers, Emanuela Reale and Thed van Leeuwen

More articles in Research Evaluation from Oxford University Press
Bibliographic data for series maintained by Oxford University Press ().