Economics at your fingertips  

Flexibility in the selection of patent counts: Implications for p-hacking and evidence-based policymaking

Stephan B. Bruns and Martin Kalthaus ()

Research Policy, 2020, vol. 49, issue 1

Abstract: This study analyzes how researchers’ degrees of freedom in selecting patent counts influence econometrically estimated policy effects. Using the evaluation of solar energy policies as an example, we identify 51 strategies to select solar patents from the literature resulting in 306 different solar patent counts, considering six common quality levels of patents. We replicate two leading studies in this literature and re-estimate their econometric models using all of these patent counts. Our results demonstrate severe uncertainty regarding sizes and even signs of key policy effects, opening up the potential for p-hacking and posing a fundamental challenge for evidence-based policymaking. We recommend that more emphasis should be devoted to patent selection procedures, including careful sensitivity analysis regarding key assumptions, such as search strategy and patent quality level. More research is needed to develop common quality standards in working with patent data.

Keywords: Patent data; Sensitivity analysis; p-hacking; Replication; Power; Solar energy technologies (search for similar items in EconPapers)
JEL-codes: C18 C40 O31 (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (3) Track citations by RSS feed

Downloads: (external link)
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link:

DOI: 10.1016/j.respol.2019.103877

Access Statistics for this article

Research Policy is currently edited by M. Bell, B. Martin, W.E. Steinmueller, A. Arora, M. Callon, M. Kenney, S. Kuhlmann, Keun Lee and F. Murray

More articles in Research Policy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().

Page updated 2021-07-16
Handle: RePEc:eee:respol:v:49:y:2020:i:1:s0048733319301969