Data-driven artificial intelligence to automate researcher assessment

Rosina O. Weber () and Kedma B. Duarte ()
Additional contact information
Rosina O. Weber: Drexel University
Kedma B. Duarte: Goias State University

Scientometrics, 2021, vol. 126, issue 4, No 23, 3265-3281

Abstract: Abstract This article describes how to utilize data-driven artificial intelligence (AI) to automate researcher assessment using data from profiling systems. We consider that a researcher assessment is done for a purpose and not divorced from a specific target placement. We formulate researcher assessment as a binary classification task, that is, a candidate researcher is classified as either fit or unfit for a given placement. For classifying researchers, we adopt case-based reasoning, a transparent artificial intelligence methodology that implements analogical reasoning, allows adaptation, machine learning, and explainability. This work addresses a human limitation through AI. Given a small number of candidates for a job or award and a clear job description, even if capable of selecting the best fit candidate, human decisions may be neither transparent nor reproducible. The approach in this article describes how to use AI methods to, from a job description, select the best fit candidate while considering career trajectories, providing explanations, and being reproducible. We describe the implementation of the methodology for a hypothetical placement in a real research institute from real but anonymized curriculum vitae from the Brazilian Lattes Database. We describe an experiment demonstrating that the purpose-oriented approach is more accurate than purpose-independent classifiers. The proposed methodology meets various principles from the Leiden Manifesto.

Keywords: Researcher assessment; Profiling systems; Case-based reasoning; Machine learning; Leiden manifesto; Career trajectory; 03–04 Explicit machine computation and programs; 01–08 Computational methods; 68T05 Learning and adaptive systems; 91E40 Memory and learning (search for similar items in EconPapers)
JEL-codes: C45 C52 C53 (search for similar items in EconPapers)
Date: 2021
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s11192-020-03859-x Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:scient:v:126:y:2021:i:4:d:10.1007_s11192-020-03859-x

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11192

DOI: 10.1007/s11192-020-03859-x

Access Statistics for this article

Scientometrics is currently edited by Wolfgang Glänzel

More articles in Scientometrics from Springer, Akadémiai Kiadó
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().