A systematic method for identifying references to academic research in grey literature

Matthew S. Bickley (), Kayvan Kousha () and Michael Thelwall ()
Additional contact information
Matthew S. Bickley: University of Wolverhampton
Kayvan Kousha: University of Wolverhampton
Michael Thelwall: University of Wolverhampton

Scientometrics, 2022, vol. 127, issue 12, No 7, 6913-6933

Abstract: Abstract Grey literature encompasses documents not published in academic journals or books. Some grey literature has substantial societal importance, such as medical guidelines, government analyses and pressure group reports. Academic research cited in such documents may therefore have had indirect societal impact, such as in policy making, clinical practice or legislation. Identifying citations to academic research from grey literature may therefore help assess its societal impacts. This is difficult, however, due to the variety of document and referencing formats used in grey literature, even from a single organisation. In response, this study introduces and tests a semi-automatic method to match academic journal articles with unstandardised grey literature cited references. For this, the metadata (lead author last name, title, year) of 2.45 million UK Russell Group university outputs was matched against a 100-document sample of UK government grey literature to assess the accuracy of 21 matching heuristics. The optimal method (lead author last name and title in either order, maximum of 200 characters apart) is sufficiently accurate and scalable to make the task of matching research outputs to grey literature references feasible. The method was then applied to 3347 government publications, showing approximately 23% of UK government grey literature in this study contained at least one reference to UK Russell Group university output, and of this grey literature, an average of 3.79 references were present per document. The applied method also shows that economics and environmental science academic research is most cited between 2010 and 2018.

Keywords: Grey literature; Impact assessment; Citation analysis; UK government; Academic references; Bland–Altman analysis (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s11192-022-04408-4 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:scient:v:127:y:2022:i:12:d:10.1007_s11192-022-04408-4

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11192

DOI: 10.1007/s11192-022-04408-4

Access Statistics for this article

Scientometrics is currently edited by Wolfgang Glänzel

More articles in Scientometrics from Springer, Akadémiai Kiadó
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().