Is the panel fair? Evaluating panel compositions through network analysis. The case of research assessments in Italy

Alberto Baccini and Cristina Re

Papers from arXiv.org

Abstract: Research evaluation is usually governed by panels of peers. Procedural fairness refers to the principles that ensures decisions are made through a fair and transparent process. It requires that the composition of panels is fair. A fair panel is usually defined in terms of observable characteristics of scholars such as gender or affiliations. The formal adherence to these criteria is not sufficient to guarantee a fair composition in terms of scholarly thinking, background, or policy orientation. An empirical strategy for exploring the fairness in the intellectual composition of panels is proposed, based on the observation of links between panellists. The case study regards the three panels selected to evaluate research in economics, statistics and business during the Italian research assessment exercises. The first two panels were appointed directly by the governmental agency responsible for the evaluation, while the third was randomly selected. Hence the third panel can be considered as a control for evaluating about the fairness of the others. The fair representation is explored by comparing the networks of panellists based on their co-authorship relations, the networks based on journals in which they published and the networks based on their affiliated institutions (universities, research centres and newspapers). The results show that the members of the first two panels had connections much higher than the members of the control group. Hence the composition of the first two panels should be considered as unfair, as the results of the research assessments.

Date: 2024-05, Revised 2025-01
New Economics Papers: this item is included in nep-net and nep-sog
References: Add references at CitEc
Citations:

Published in Scientometrics 2025

Downloads: (external link)
http://arxiv.org/pdf/2405.06476 Latest version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:arx:papers:2405.06476

Access Statistics for this paper

More papers in Papers from arXiv.org
Bibliographic data for series maintained by arXiv administrators ().