Confirmatory bias in peer review

J. A. Garcia (), Rosa Rodriguez-Sánchez and J. Fdez-Valdivia
Additional contact information
J. A. Garcia: Universidad de Granada
Rosa Rodriguez-Sánchez: Universidad de Granada
J. Fdez-Valdivia: Universidad de Granada

Scientometrics, 2020, vol. 123, issue 1, No 25, 517-533

Abstract: Abstract A reduction in reviewer’s recommendation quality may be caused by a limitation of time or cognitive overload that comes from the level of redundancy, contradiction and inconsistency in the research. Some adaptive mechanisms by reviewers who deal with situations of information overload may be chunking single pieces of manuscript information into generic terms, unsystematic omission of research details, queuing of information processing, and prematurely stop the manuscript evaluation. Then, how would a reviewer optimize attention to positive and negative attributes of a manuscript before making a recommendation? How a reviewer’s characteristics such as her prior belief about the manuscript quality and manuscript evaluation cost, affect her attention allocation and final recommendation? To answer these questions, we use a probabilistic model in which a reviewer chooses the optimal evaluation strategy by trading off the value and cost of review information about the manuscript quality. We find that a reviewer could exhibit a confirmatory behavior under which she pays more attention to the type of manuscript attributes that favor her prior belief about the manuscript quality. Then, confirmatory bias could be an optimal behavior of the reviewers that optimize attention to positive and negative manuscript attributes under information overload. We also show that reviewer’s manuscript evaluation cost plays a key role in determining whether she may exhibit confirmatory bias. Moreover, when the reviewer’s prior belief about the manuscript quality is low enough, the probability of obtaining a positive review signal decreases with reviewer’s manuscript evaluation cost, for a sufficiently high cost.

Keywords: Peer review; Confirmatory bias; Optimal behavior; Evaluation cost; Reviewer recommendation (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (4)

Downloads: (external link)
http://link.springer.com/10.1007/s11192-020-03357-0 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:scient:v:123:y:2020:i:1:d:10.1007_s11192-020-03357-0

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11192

DOI: 10.1007/s11192-020-03357-0

Access Statistics for this article

Scientometrics is currently edited by Wolfgang Glänzel

More articles in Scientometrics from Springer, Akadémiai Kiadó
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().