There must be an error here! Experimental evidence on coding errors' biases

Bruno Ferman and Lucas Finamor

No 266, I4R Discussion Paper Series from The Institute for Replication (I4R)

Abstract: Quantitative research relies heavily on coding, and coding errors are relatively common even in published research. In this paper, we examine whether individuals are more or less likely to check their code depending on the results they obtain. We test this hypothesis in a randomized experiment embedded in the recruitment process for research positions at a large international economic organization. In a coding task designed to assess candidates' programming abilities, we randomize whether participants obtain an expected or unexpected result if they commit a simple coding error. We find that individuals are almost 20% more likely to detect coding errors when they lead to unexpected results. This asymmetry in error detection depending on the results they generate suggests that coding errors may lead to biased findings in scientific research.

JEL-codes: C80 C81 C93 (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.econstor.eu/bitstream/10419/328046/1/I4R-DP266.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zbw:i4rdps:266

Access Statistics for this paper

More papers in I4R Discussion Paper Series from The Institute for Replication (I4R)
Bibliographic data for series maintained by ZBW - Leibniz Information Centre for Economics ().