Exploring a prototype framework of web-based and peer-reviewed “European Educational Research Quality Indicators” (EERQI)

Ton Mooij ()
Additional contact information
Ton Mooij: Radboud University Nijmegen (ITS)

Scientometrics, 2015, vol. 102, issue 1, No 52, 1037-1055

Abstract: Abstract Digitization, the Internet, and information or webometric interdisciplinary approaches are affecting the fields of Scientometrics and Library and Information Science. These new approaches can be used to improve citation-only procedures to estimate the quality and impact of research. A European pilot to explore this potential was called “European Educational Research Quality Indicators” (EERQI, FP7 # 217549). An interdisciplinary consortium was involved from 2008 to 2011. Different types of indicators were developed to score 171 educational research documents. Extrinsic bibliometric and citation indicators were collected from the Internet for each document; intrinsic indicators reflecting content-based quality were developed and relevant data gathered by peer review. Exploratory and confirmatory factor analysis and structural modeling were used to explore statistical relationships among latent factors or concepts and their indicators. Three intrinsic and two extrinsic latent factors were found to be relevant. Moreover, the more a document was related to a reviewer’s own area of research, the higher the score the reviewer gave concerning (1) significance, originality, and consistency, and (2) methodological adequacy. The conclusions are that a prototype EERQI framework has been constructed: intrinsic quality indicators add specific information to extrinsic quality or impact indicators, and vice versa. Also, a problem of “objective” impact scores is that they are based on “subjective” or biased peer-review scores. Peer-review, which is foundational to having a work cited, seems biased and this bias should be controlled or improved by more refined estimates of quality and impact of research. Some suggestions are given and limitations of the pilot are discussed. As the EERQI development approach, instruments, and tools are new, they should be developed further.

Keywords: Quality indicators; Impact factor; Multi-method assessment; Educational research; Reviewer’s bias (search for similar items in EconPapers)
Date: 2015
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s11192-014-1429-x Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:scient:v:102:y:2015:i:1:d:10.1007_s11192-014-1429-x

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11192

DOI: 10.1007/s11192-014-1429-x

Access Statistics for this article

Scientometrics is currently edited by Wolfgang Glänzel

More articles in Scientometrics from Springer, Akadémiai Kiadó
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().