reprun, automating complete reproducibility verifications

Benjamin Daniels, Ankriti Singh, Luis Eduardo San Martin and Kristoffer Bjarkefur
Additional contact information
Ankriti Singh: World Bank, DIME
Luis Eduardo San Martin: World Bank
Kristoffer Bjarkefur: The World Bank, DIME/LSMS

2024 Stata Conference from Stata Users Group

Abstract: The reprun command in Stata is designed to automate reproducibility verifications for sets of Stata do-files. This session presents detailed updates to the command in the context of DIME Analytics’s repkit package, which spans a complete workflow for the reproducibility verifications. The repkit package aims to ensure that the outputs of reproducibility packages are stable and reproducible, addressing the common sources of reproducibility failures. By identifying and correcting issues, users can improve the reliability of their statistical analyses, making them suitable for sharing and publication. The reprun command performs two runs of a specified do-file, recording the state of Stata after each line’s execution during the first run and then comparing it with the state after the same line’s execution in the second run. Key states monitored include the random-number generator (RNG) state, data sort order, and data contents. If discrepancies occur between the two runs, reprun flags potential reproducibility errors, reporting mismatches in a table format, which helps in identifying and resolving issues. This tool emphasizes the importance of managing randomness and maintaining consistent data states to avoid reproducibility errors, especially when inconsistent outputs are far downstream in code from their sources.

Date: 2024-08-04
References: Add references at CitEc
Citations:

Downloads: (external link)
http://repec.org/usug2024/US24_Daniels.pdf

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:boc:usug24:08

Access Statistics for this paper

More papers in 2024 Stata Conference from Stata Users Group Contact information at EDIRC.
Bibliographic data for series maintained by Christopher F Baum ().