Investigating the effect of publication text similarity between reviewers and authors on the rigor of peer review: An intellectual proximity perspective

Yanlan Kang, Chenwei Zhang, Zhuanlan Sun and Yiwei Li

Journal of Informetrics, 2025, vol. 19, issue 3

Abstract: The involvement of experienced peers as reviewers plays a crucial role in manuscript evaluation during the peer review process. Nonetheless, concerns have arisen regarding potential cognitive bias when reviewers assess research that is outside their areas of expertise. Despite these concerns, quantitative analysis of this issue remains limited. This study aims to empirically investigate whether submissions reviewed by peers with academic backgrounds similar to the authors’ research areas correlate with more rigorous comments during the peer review process. Utilizing a dataset of 2,147 papers published in the journal eLife, along with their publicly available peer review reports and reviewers’ publication records, we employed natural language processing techniques to measure the publication text similarity of reviewers to that of the manuscript’s authors, representing a minuscule part of intellectual proximity. We then used a linear regression model to examine whether such similarity was associated with review rigor, quantified by the frequency of statistical terms from two well-known glossaries. We observed no statistically significant differences in the rigor of comments made by peers with varying levels of publication text similarity in the constructed dataset and setting. The findings remained consistent across several robustness checks and alternative specifications. This suggests that no discernible cognitive bias is introduced by the reviewers’ academic background during the peer review process, enriching the extant literature and offering important insights into understanding the role of reviewers in maintaining fairness.

Keywords: Peer review; Reviewer expertise; Academic background; Publication records; Text similarity (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S1751157725000732
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:infome:v:19:y:2025:i:3:s1751157725000732

DOI: 10.1016/j.joi.2025.101709

Access Statistics for this article

Journal of Informetrics is currently edited by Leo Egghe

More articles in Journal of Informetrics from Elsevier
Bibliographic data for series maintained by Catherine Liu ().