Unintended Code Errors and Computational Reproducibility

Nicholas Ottone and Limor Peer
Additional contact information
Limor Peer: Yale University

No rv6xd, MetaArXiv from Center for Open Science

Abstract: Transparent and reproducible science requires that data and code underlying published results be made available to allow regeneration and verification of the results. But how easy is it to reproduce computational analyses using these materials? This technical report describes efforts to computationally reproduce results at one established repository as part of a routine internal pre-publication review. We conduct a structured content analysis of twenty-six “author reports'' generated through a standard review process and categorize identified issues. We find that four in five studies include code errors that prevent execution or produce inconsistent results. Although these errors are generally minor, we interpret our findings to indicate that review and correction can help resolve many of the errors and facilitate computational reproducibility.

Date: 2024-11-08
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://osf.io/download/672d7abd5ab313acee8ea4b1/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:metaar:rv6xd

DOI: 10.31219/osf.io/rv6xd

Access Statistics for this paper

More papers in MetaArXiv from Center for Open Science
Bibliographic data for series maintained by OSF ().