Statistical reporting errors in economics

Stephan Bruns, Helmut Herwartz, John P.A. Ioannidis, Chris-Gabriel Islam and Fabian H. C. Raters

No mbx62, MetaArXiv from Center for Open Science

Abstract: We developed a tool that scrapes and interprets statistical values (DORIS) to analyze reporting errors, which occur if the eye-catcher depicting the level of statistical significance is inconsistent with the reported statistical values. Using 578,132 tests from the top 50 economics journals, we find that 14.88 % of the articles have at least one strong error in the main tests. Our pre-registered analysis suggests that mandatory data and code availability policies reduce the prevalence of strong errors, while suggestive indication of a reversed effect is found for top 5 journals. Integrating DORIS into the review process can help improving article quality.

Date: 2023-09-06
New Economics Papers: this item is included in nep-sog
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (8)

Downloads: (external link)
https://osf.io/download/64f2317ad9f2c90d92d0493a/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:metaar:mbx62

DOI: 10.31219/osf.io/mbx62

Access Statistics for this paper

More papers in MetaArXiv from Center for Open Science
Bibliographic data for series maintained by OSF ().