 Flexibility in the selection of patent counts: Implications for p-hacking and evidence-based policymakingStephan B. Bruns and Martin Kalthaus Research Policy, 2020, vol. 49, issue 1 Abstract: This study analyzes how researchers’ degrees of freedom in selecting patent counts influence econometrically estimated policy effects. Using the evaluation of solar energy policies as an example, we identify 51 strategies to select solar patents from the literature resulting in 306 different solar patent counts, considering six common quality levels of patents. We replicate two leading studies in this literature and re-estimate their econometric models using all of these patent counts. Our results demonstrate severe uncertainty regarding sizes and even signs of key policy effects, opening up the potential for p-hacking and posing a fundamental challenge for evidence-based policymaking. We recommend that more emphasis should be devoted to patent selection procedures, including careful sensitivity analysis regarding key assumptions, such as search strategy and patent quality level. More research is needed to develop common quality standards in working with patent data. Keywords: Patent data; Sensitivity analysis; p-hacking; Replication; Power; Solar energy technologies (search for similar items in EconPapers) Downloads: (external link) Related works: Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text Persistent link: https://EconPapers.repec.org/RePEc:eee:respol:v:49:y:2020:i:1:s0048733319301969 DOI: 10.1016/j.respol.2019.103877 Access Statistics for this article Research Policy is currently edited by M. Bell, B. Martin, W.E. Steinmueller, A. Arora, M. Callon, M. Kenney, S. Kuhlmann, Keun Lee and F. Murray More articles in Research Policy from Elsevier |
Is your work missing from RePEc? Here is how to contribute.
Questions or problems? Check the EconPapers FAQ or send mail to .
EconPapers is hosted by the
School of Business at Örebro University.