New Data, New Results? How Data Sources and Vintages Affect the Replicability of Research

Iasmin Goes

No 23, I4R Discussion Paper Series from The Institute for Replication (I4R)

Abstract: Macroeconomic variables like unemployment, inflation, trade, or GDP are not set in stone: they are preliminary estimates that are constantly revised by statistical agencies. These data revisions, or data vintages, often provide conflicting information about the size of a country's economy or its level of development, reducing our confidence in established findings. Would researchers come to different conclusions if they used different vintages? To answer this question, I survey all articles published in a top political science journal between 2005 and 2020. I replicate three prominent articles and find that the use of different vintages can lead to different statistical results, calling into question the robustness of otherwise rigorous empirical research. These findings have two practical implications. First, researchers should always be transparent about their data sources and vintages. Second, researchers should be more modest about the precision and accuracy of their point estimates, since these estimates can mask large measurement errors.

Date: 2023
New Economics Papers: this item is included in nep-des, nep-mac and nep-sog
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.econstor.eu/bitstream/10419/270603/1/I4R-DP023.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zbw:i4rdps:23

Access Statistics for this paper

More papers in I4R Discussion Paper Series from The Institute for Replication (I4R)
Bibliographic data for series maintained by ZBW - Leibniz Information Centre for Economics ().