Flexibility in the selection of patent counts: Implications for p-hacking and evidence-based policymaking
Stephan B. Bruns and
Martin Kalthaus ()
Research Policy, 2020, vol. 49, issue 1
This study analyzes how researchers’ degrees of freedom in selecting patent counts influence econometrically estimated policy effects. Using the evaluation of solar energy policies as an example, we identify 51 strategies to select solar patents from the literature resulting in 306 different solar patent counts, considering six common quality levels of patents. We replicate two leading studies in this literature and re-estimate their econometric models using all of these patent counts. Our results demonstrate severe uncertainty regarding sizes and even signs of key policy effects, opening up the potential for p-hacking and posing a fundamental challenge for evidence-based policymaking. We recommend that more emphasis should be devoted to patent selection procedures, including careful sensitivity analysis regarding key assumptions, such as search strategy and patent quality level. More research is needed to develop common quality standards in working with patent data.
Keywords: Patent data; Sensitivity analysis; p-hacking; Replication; Power; Solar energy technologies (search for similar items in EconPapers)
JEL-codes: C18 C40 O31 (search for similar items in EconPapers)
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (3) Track citations by RSS feed
Downloads: (external link)
Full text for ScienceDirect subscribers only
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
Persistent link: https://EconPapers.repec.org/RePEc:eee:respol:v:49:y:2020:i:1:s0048733319301969
Access Statistics for this article
Research Policy is currently edited by M. Bell, B. Martin, W.E. Steinmueller, A. Arora, M. Callon, M. Kenney, S. Kuhlmann, Keun Lee and F. Murray
More articles in Research Policy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().