Can’t bibliometric analysts do better? How quality assessment without field expertise does not work

Nina Lykke ()
Additional contact information
Nina Lykke: Linköping University

Scientometrics, 2018, vol. 117, issue 1, No 41, 655-666

Abstract: Abstract The article is an invited comment on Guy Madison and Therese Söderlund (M&S): Comparisons of content and scientific quality indicators across peer-reviewed journal articles with more or less gender perspective: Gender studies can do better. Scientometrics 115(3):1161–1183. The article pinpoints a series of serious problems in M&S’s quantitative quality assessment and analysis of the field of gender studies, pertaining to their overall conceptual framework, their general approach and their specific analysis. It is argued that the over-arching problem in M&S’s study is their lack of expert knowledge of the field of gender studies, their lack of respect for differences between qualitative and quantitative research, and their research design, which is biased towards quantitative social and natural science research. Firstly, it is demonstrated that a key concept, ‘gender perspective’, is used in an incoherent and confusing way in M&S’s analysis. Secondly, it is argued that the confusion is not an isolated definitional problem, but related to a series of slippages between M&S’s source of inspiration (Ganetz in Genusvetenskapliga projektansökningar inom humaniora-samhällsvetenskap – en uppföljning av Vetenskapsrådets beredning och utfall år 2004. Vetenskapsrådets rapportserie, Stockholm 15/2005, 2005) and their own adoption of the category. Thirdly, differences between qualitative and quantitative research, and between hermeneutic and explanatory knowledge production, are discussed more broadly to sustain the argument that the mentioned slippages occur, because M&S transfer analytical tools from Ganetz’ qualitative study, based on a peer review methodology, to a quantitative quality assessment, carried out without field specific expert knowledge. It is argued that, to be adequate and relevant, a quality assessment would need to respect these differences, and develop tools and research designs accordingly. Fourthly, the validity of M&S’s content analysis—the core of their study—is questioned in detail because of its use of inadequate analytical categories, and because of its exclusion of central elements from the analysis. Finally, it is argued that the bias in M&S’s research design is reproduced in their results.

Keywords: Gender studies; Quantitative quality assessment; Differences between qualitative and quantitative research; The importance of field specific knowledge (search for similar items in EconPapers)
Date: 2018
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s11192-018-2872-x Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:scient:v:117:y:2018:i:1:d:10.1007_s11192-018-2872-x

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11192

DOI: 10.1007/s11192-018-2872-x

Access Statistics for this article

Scientometrics is currently edited by Wolfgang Glänzel

More articles in Scientometrics from Springer, Akadémiai Kiadó
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().