Computational Reproducibility in Finance: Evidence from 1,000 Tests

Christophe Pérignon, Olivier Akmansoy, Christophe Hurlin, Anna Dreber, Felix Holzmeister, Jürgen Huber, Magnus Johannesson, Michael Kirchler, Albert J Menkveld, Michael Razen and Utz Weitzel

The Review of Financial Studies, 2024, vol. 37, issue 11, 3558-3593

Abstract: We analyze the computational reproducibility of more than 1,000 empirical answers to 6 research questions in finance provided by 168 research teams. Running the researchers’ code on the same raw data regenerates exactly the same results only 52% of the time. Reproducibility is higher for researchers with better coding skills and those exerting more effort. It is lower for more technical research questions, more complex code, and results lying in the tails of the distribution. Researchers exhibit overconfidence when assessing the reproducibility of their own research. We provide guidelines for finance researchers and discuss implementable reproducibility policies for academic journals.

Keywords: C80; C87 (search for similar items in EconPapers)
Date: 2024
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://hdl.handle.net/10.1093/rfs/hhae029 (application/pdf)
Access to full text is restricted to subscribers.

Related works:
Working Paper: Computational Reproducibility in Finance: Evidence from 1,000 Tests (2024)
Working Paper: Computational Reproducibility in Finance: Evidence from 1,000 Tests (2022)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:oup:rfinst:v:37:y:2024:i:11:p:3558-3593.

Ordering information: This journal article can be ordered from
https://academic.oup.com/journals

Access Statistics for this article

The Review of Financial Studies is currently edited by Itay Goldstein

More articles in The Review of Financial Studies from Society for Financial Studies Oxford University Press, Journals Department, 2001 Evans Road, Cary, NC 27513 USA.. Contact information at EDIRC.
Bibliographic data for series maintained by Oxford University Press ().