Is the panel fair? Evaluating panel compositions through network analysis. The case of research assessments in Italy

Alberto Baccini () and Cristina Re ()
Additional contact information
Alberto Baccini: Università degli Studi di Siena
Cristina Re: Università degli Studi di Siena

Scientometrics, 2025, vol. 130, issue 4, No 4, 2093-2135

Abstract: Abstract Research evaluation is usually governed by panels of peers. Procedural fairness refers to the principles that ensures decisions are made through a fair and transparent process. It requires that the composition of panels is fair. A fair panel is usually defined in terms of observable characteristics of scholars such as gender or affiliations. The formal adherence to these criteria is not sufficient to guarantee a fair composition in terms of scholarly thinking, background, or policy orientation. An empirical strategy for exploring the fairness in the intellectual composition of panels is proposed, based on the observation of links between panellists. The case study regards the three panels selected to evaluate research in economics, statistics and business during the Italian research assessment exercises. The first two panels were appointed directly by the governmental agency responsible for the evaluation, while the third was randomly selected. Hence the third panel can be considered as a control for evaluating about the fairness of the others. The fair representation is explored by comparing the networks of panellists based on their co-authorship relations, the networks based on journals in which they published and the networks based on their affiliated institutions (universities, research centres and newspapers). The results show that the members of the first two panels had connections much higher than the members of the control group. Hence the composition of the first two panels should be considered as unfair, as the results of the research assessments.

Keywords: Procedural fairness; Research assessment; Gatekeepers of economics; Social network analysis (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s11192-025-05255-9 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:scient:v:130:y:2025:i:4:d:10.1007_s11192-025-05255-9

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11192

DOI: 10.1007/s11192-025-05255-9

Access Statistics for this article

Scientometrics is currently edited by Wolfgang Glänzel

More articles in Scientometrics from Springer, Akadémiai Kiadó
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().