An overview of assessing the quality of peer review reports of scientific articles

Amanda Sizo, Adriano Lino, Luis Paulo Reis and Álvaro Rocha

International Journal of Information Management, 2019, vol. 46, issue C, 286-293

Abstract: Assuring the quality control of publications in the scientific literature is one of the main challenges of the peer review process. Consequently, there has been an increasing demand for computing solutions that will help to maintain the quality of this process. Recently, the use of Artificial Intelligence techniques has been highlighted, applied in the detection of plagiarism, bias, among other functions. The assessment of the reviewer’s review has also been considered as important in the process, but, little is known about it, for instance, which techniques have been applied in this assessment or which criteria have been assessed. Therefore, this systematic literature review aims to find evidence regarding the computational approaches that have been used to evaluate reviewers' reports. In order to achieve this, five online databases were selected, from which 72 articles were identified that met the inclusion criteria of this review, all of which have been published since 2000. The result returned 10 relevant studies meeting the evaluation requirements of scientific article reviews. The review revealed that mechanisms to rank review reports according to a score, as well as the word analysis, are the most common tools, and that there is no consensus on quality criteria. The systematic literature review has shown that reviewers’ report assessment is a valid tool for maintaining quality throughout the process. However, it still needs to be further developed if it is to be used as a resource which surpass a single conference or journal, making the peer review process more rigorous and less based on random choice.

Keywords: Systematic literature review; Peer review; Reviewers’ report assessment; Reviewers’ report quality (search for similar items in EconPapers)
Date: 2019
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0268401218304857
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:ininma:v:46:y:2019:i:c:p:286-293

DOI: 10.1016/j.ijinfomgt.2018.07.002

Access Statistics for this article

International Journal of Information Management is currently edited by Yogesh K. Dwivedi

More articles in International Journal of Information Management from Elsevier
Bibliographic data for series maintained by Catherine Liu ().