Does citation polarity help evaluate the quality of academic papers?

Linhong Xu (), Kun Ding (), Yuan Lin () and Chunbo Zhang
Additional contact information
Linhong Xu: Dalian University of Technology
Kun Ding: Dalian University of Technology
Yuan Lin: Dalian University of Technology
Chunbo Zhang: Dalian University of Technology

Scientometrics, 2023, vol. 128, issue 7, No 11, 4065-4087

Abstract: Abstract Citation frequency is an important metric for the evaluation of academic papers, but it assumes that all citations are of equal value. The purpose of this study is to determine the validity of citation polarity, which contains evaluative information such as criticism or praise, in the evaluation of paper quality. In this paper, 3538 citation sentences in papers from ACL conferences were selected and manually annotated for citation polarity. They were divided into best paper group and matching paper group, and tested in heterologous pairs to determine whether there were differences in the positive and negative citations of the two groups, and to further investigate the trend of citation polarity with the increase of citation window. The results of the study showed that the best paper and the matching paper had significant differences in the number of positive and negative citations, and the mean and median values of positive citations in the best group were about 1.5 times higher than those in the matching group. As the citation window increased, the best papers maintained both positive and negative citation dominance over 5 years, and the peak citation in the best group was about three times higher than that in the matching group. Therefore, the metric of citation polarity can help evaluate the quality of papers and provide new ideas for scientific and objective evaluation of academic papers.

Keywords: Citation polarity; Quality of academic papers; Positive citation; Negative citation; Research evaluation (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://link.springer.com/10.1007/s11192-023-04734-1 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:scient:v:128:y:2023:i:7:d:10.1007_s11192-023-04734-1

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11192

DOI: 10.1007/s11192-023-04734-1

Access Statistics for this article

Scientometrics is currently edited by Wolfgang Glänzel

More articles in Scientometrics from Springer, Akadémiai Kiadó
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().