Journals in Economic Sciences: Paying Lip Service to Reproducible Research?

Sven Vlaeminck and Felix Podkrajac

EconStor Open Access Articles and Book Chapters, 2017, vol. 41, issue 1-4, 1-16

Abstract: The findings of numerous replication studies in economics have raised serious concerns regarding the credibility and reliability of published applied economic research. Literature suggests that economic research often is not replicable because (i) only a small proportion of journals in the field have implemented functional policies on the disclosure of employed datasets and program code, (ii) authors frequently do not comply with these data policies and (iii) editorial offices do not ensure that these policies are enforced. In this paper, we focus on the aspect last mentioned. We empirically evaluate 599 articles published in 37 journals with a data availability policy. We present the share of articles that fall under a data policy, because replication data is needed to verify the published results. Afterwards, we check the journal data archives and supplemental information section of each article for the availability of replication files. For a reduced sub-sample of 245 data-based articles, we check in depth whether the replication files we found are compliant with the requirements of the journal’s respective data policy. Thereby, we are able to determine how much journals in economic sciences enforce their data policies. Our findings suggest a mixed picture: While some journals achieve high compliance rates, a significant share of journals only sporadically provides replication files for data-based research papers. Overall, 47.5% of all articles analysed honour the data policy of the respective journal. Our findings also provide evidence that voluntary data policies are not effective in fostering replicable research.

Keywords: reproducible research; journals; data policies; data archives (search for similar items in EconPapers)
Date: 2017
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://www.econstor.eu/bitstream/10419/265247/1/I ... igiert_Erratum_F.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zbw:espost:177391

DOI: 10.29173/iq6

Access Statistics for this article

More articles in EconStor Open Access Articles and Book Chapters from ZBW - Leibniz Information Centre for Economics Contact information at EDIRC.
Bibliographic data for series maintained by ZBW - Leibniz Information Centre for Economics ().