Creating evaluative homogeneity: Experience of constructing a national journal ranking

Jakub Krzeski, Krystian Szadkowski and Emanuel Kulczycki

Research Evaluation, 2022, vol. 31, issue 3, 410-422

Abstract: The study investigates the process of creating evaluative homogeneity within a performance-based research funding system. It does so in reference to the experience of creating a national journal ranking in Poland in 2019. It refers to the cases of two disciplines: biology and history. The use of this case is justified by its unique character, that is creating the ranking through means of both bibliometric indicators and expert panels. We therefore asked: What guided the actors who participated in the process of creating the ranking through the use of bibliometric indicators when introducing changes to its initial form? To answer this question, a mixed-methods approach was used. First, we conducted a quantitative analysis of changes made to the ranking during the consecutive steps of its creation. Second, we conducted a qualitative analysis of semi-structured interviews regarding actors’ motivations. By reflecting on the qualitative analysis along with the results of the quantitative part, this study reveals the extent to which actors impacted the form evaluative homogeneity took within the system. It further argues that this is dictated by how actors position themselves in relation to two opposing forces: those favouring the homogenization of research and those maintaining its heterogeneity. As these forces remain imbalanced, the study concludes with a call for further exploration of the interplay between the forces of homogenization and heterogenization and how the tensions between them are mediated within a performance-based research funding system.

Keywords: evaluative homogeneity; heterogeneity of science; epistemic cultures; national journal ranking; performance-based research funding systems (search for similar items in EconPapers)
Date: 2022
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://hdl.handle.net/10.1093/reseval/rvac011 (application/pdf)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:oup:rseval:v:31:y:2022:i:3:p:410-422.

Access Statistics for this article

Research Evaluation is currently edited by Julia Melkers, Emanuela Reale and Thed van Leeuwen

More articles in Research Evaluation from Oxford University Press
Bibliographic data for series maintained by Oxford University Press ().