Scientometric indicators: peer-review, bibliometric methods and conflict of interests

Primož Južnič (), Stojan Pečlin (), Matjaž Žaucer (), Tilen Mandelj (), Miro Pušnik () and Franci Demšar ()
Additional contact information
Primož Južnič: University of Ljubljana
Stojan Pečlin: Slovenian Research Agency (ARRS)
Matjaž Žaucer: Central Technological Library at the University of Ljubljana
Tilen Mandelj: Central Technological Library at the University of Ljubljana
Miro Pušnik: Central Technological Library at the University of Ljubljana
Franci Demšar: Slovenian Research Agency (ARRS)

Scientometrics, 2010, vol. 85, issue 2, No 4, 429-441

Abstract: Abstract The paper discusses the role of scientometric indicators in peer-review selection of research project proposals. An ex post facto evaluation was made of three calls for research project proposals in Slovenia: 2003 with a peer review system designed in a way that conflict of interest was not avoided effectively, 2005 with a sound international peer-review system with minimized conflict of interest influence but a limited number of reviewers, and 2008 with a combination of scientometric indicators and a sound international peer review with minimized conflict of interest influence. The hypothesis was that the three different peer review systems would have different correlations with the same set of scientometric indicators. In the last two decision-making systems (2005 and 2008) where conflict of interest was effectively avoided, we have a high percentage (65%) of projects that would have been selected in the call irrespective of the method (peer review or bibliometrics solely). In contrast, in the 2003 call there is a significantly smaller percentage (49%) of projects that would have been selected in the call irrespective of the method (peer review or bibliometrics solely). It was shown that while scientometric indicators can hardly replace the peer-review system as the ultimate decision-making and support system, they can reveal its weaknesses on one hand and on the other can verify peer-review scores and minimize conflict of interest if necessary.

Keywords: Scientometric indicators; Research project proposals; Ex post evaluation; Peer review systems; Conflict of interests (search for similar items in EconPapers)
Date: 2010
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (9)

Downloads: (external link)
http://link.springer.com/10.1007/s11192-010-0230-8 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:scient:v:85:y:2010:i:2:d:10.1007_s11192-010-0230-8

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11192

DOI: 10.1007/s11192-010-0230-8

Access Statistics for this article

Scientometrics is currently edited by Wolfgang Glänzel

More articles in Scientometrics from Springer, Akadémiai Kiadó
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().