Is N-Hacking Ever OK? The consequences of collecting more data in pursuit of statistical significance

Pamela Reinagel

PLOS Biology, 2023, vol. 21, issue 11, 1-15

Abstract: Upon completion of an experiment, if a trend is observed that is “not quite significant,” it can be tempting to collect more data in an effort to achieve statistical significance. Such sample augmentation or “N-hacking” is condemned because it can lead to an excess of false positives, which can reduce the reproducibility of results. However, the scenarios used to prove this rule tend to be unrealistic, assuming the addition of unlimited extra samples to achieve statistical significance, or doing so when results are not even close to significant; an unlikely situation for most experiments involving patient samples, cultured cells, or live animals. If we were to examine some more realistic scenarios, could there be any situations where N-hacking might be an acceptable practice? This Essay aims to address this question, using simulations to demonstrate how N-hacking causes false positives and to investigate whether this increase is still relevant when using parameters based on real-life experimental settings.When an experiment comes to an end, it can be tempting to collect more data if a trend is observed that is “not quite significant”. This Essay uses simulations to investigate if this type of sample augmentation can ever be an acceptable practice and, if so, when it could be beneficial.

Date: 2023
References: View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3002345 (text/html)
https://journals.plos.org/plosbiology/article/file ... 02345&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pbio00:3002345

DOI: 10.1371/journal.pbio.3002345

Access Statistics for this article

More articles in PLOS Biology from Public Library of Science
Bibliographic data for series maintained by plosbiology ().