On the optimistic performance evaluation of newly introduced bioinformatic methods

Stefan Buchka, Alexander Hapfelmeier, Paul P Gardner, Rory Wilson and Anne-Laure Boulesteix

No pkqdx, MetaArXiv from Center for Open Science

Abstract: Most research articles presenting new data analysis methods claim that “the new method performs better than existing methods”, but the veracity of such statements is questionable. Our manuscript discusses and illustrates consequences of the optimistic bias occurring during the evaluation of novel data analysis methods, that is, all biases resulting from, for example, selection of datasets or competing methods; better ability to fix bugs in a preferred method; and selective reporting of method variants. We quantitatively investigate this bias using a topical example from epigenetic analysis: normalization methods for data generated by the Illumina HumanMethylation450K BeadChip microarray.

Date: 2021-01-19
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://osf.io/download/6006eb51ba010908098915d7/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:metaar:pkqdx

DOI: 10.31219/osf.io/pkqdx

Access Statistics for this paper

More papers in MetaArXiv from Center for Open Science
Bibliographic data for series maintained by OSF ().