An integrated indicator for evaluating scientific papers: considering academic impact and novelty

Zhaoping Yan () and Kaiyu Fan ()
Additional contact information
Zhaoping Yan: Nanjing University
Kaiyu Fan: The Chinese University of Hong Kong - Shenzhen

Scientometrics, 2024, vol. 129, issue 11, No 17, 6909-6929

Abstract: Abstract The assessment of scientific papers has long been a challenging issue. Although numerous studies have proposed quantitative indicators for assessing scientific papers, these studies overlooked the citation characteristics and the novelty of scientific knowledge implied in the textual information of papers. Therefore, this paper constructs an integrated indicator to evaluate scientific papers from both citation and semantic perspectives. Firstly, we propose weighted citations to measure the academic impact of scientific papers, which takes time heterogeneity and citation sentiment factors into consideration. Secondly, we capture the novelty of scientific papers from a semantic perspective, utilizing FastText to represent papers as text embeddings and applying the local outlier factor to calculate it. To validate the performance of our approach, the bullwhip effect domain and the ACL Anthology corpus are used for case studies. The results demonstrate that our indicator can effectively identify outstanding papers, thus providing a more comprehensive evaluation method for evaluating academic research.

Keywords: Paper assessment; Academic impact; Academic novelty; Weighted citation count; Citation sentiment (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s11192-024-05150-9 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:scient:v:129:y:2024:i:11:d:10.1007_s11192-024-05150-9

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11192

DOI: 10.1007/s11192-024-05150-9

Access Statistics for this article

Scientometrics is currently edited by Wolfgang Glänzel

More articles in Scientometrics from Springer, Akadémiai Kiadó
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().