S&T indicators in the wild: Contextualization and participation for responsible metrics

Ismael Rafols

Research Evaluation, 2019, vol. 28, issue 1, 7-22

Abstract: The use of indicators in research policy and evaluation is widely perceived as problematic. Responding to demands for explicit normative framings in STI governance, I propose an agenda for transforming the place and role of indicators in policy. Given that expert advice should not separate knowledge formation from decision-making under conditions of uncertainty and lack of value consensus, I argue that current scientometrics is too focused on technical issues, too reductionist, and too isolated from the contexts and values of its use. Using Callon’s analytical framework of ‘secluded research’ vs. ‘research in the wild’, I propose three moves for improving design and use of science, technology, and innovation (STI) indicators. First, to continue ongoing trends towards pluralizing the data sources, processing and visualization techniques, and expand the research communities involved in scientometrics. Second, to develop forms of quantitative evidence that can be contextualized with the participation of a more diverse set of stakeholders. Third, to open up the policy framings implicit in measurement, and use quantitative analyses to reveal more balanced perspectives of existing and alternative STI options. I conclude by arguing that these shifts are necessary to preserve epistemic diversity and pluralism in the face of ongoing managerial push for standardization via ‘platforms’ run by commercial oligopolies.

Keywords: indicators; contextualization; responsible metrics; evaluation (search for similar items in EconPapers)
Date: 2019
References: Add references at CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://hdl.handle.net/10.1093/reseval/rvy030 (application/pdf)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:oup:rseval:v:28:y:2019:i:1:p:7-22.

Access Statistics for this article

Research Evaluation is currently edited by Julia Melkers, Emanuela Reale and Thed van Leeuwen

More articles in Research Evaluation from Oxford University Press
Bibliographic data for series maintained by Oxford University Press ().