Analyzing nested experimental designs—A user-friendly resampling method to determine experimental significance

Rishikesh U Kulkarni, Catherine L Wang and Carolyn R Bertozzi

PLOS Computational Biology, 2022, vol. 18, issue 5, 1-26

Abstract: While hierarchical experimental designs are near-ubiquitous in neuroscience and biomedical research, researchers often do not take the structure of their datasets into account while performing statistical hypothesis tests. Resampling-based methods are a flexible strategy for performing these analyses but are difficult due to the lack of open-source software to automate test construction and execution. To address this, we present Hierarch, a Python package to perform hypothesis tests and compute confidence intervals on hierarchical experimental designs. Using a combination of permutation resampling and bootstrap aggregation, Hierarch can be used to perform hypothesis tests that maintain nominal Type I error rates and generate confidence intervals that maintain the nominal coverage probability without making distributional assumptions about the dataset of interest. Hierarch makes use of the Numba JIT compiler to reduce p-value computation times to under one second for typical datasets in biomedical research. Hierarch also enables researchers to construct user-defined resampling plans that take advantage of Hierarch’s Numba-accelerated functions.Author summary: An important step in analyzing experimental data is quantifying uncertainty in the experimenter’s conclusions. One mechanism for doing so is by using a statistical hypothesis test, which allows the experimenter to control what percentage of the time they make erroneous conclusions over the course of their career. Biological experimental designs often have hierarchical data-gathering schemes that traditional hypothesis tests are not well-suited for (for example, an experimenter may make measurements of several tissue samples that were collected from subjects who were given a treatment). While traditional tests can be adapted to hierarchical experimental designs, we propose a simple resampling-based hypothesis test that applies to a variety of experimental designs while maintaining control over error rate. In this manuscript, we describe Hierarch, the Python package that enables users to carry out this test and validate it under several conditions.

Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1010061 (text/html)
https://journals.plos.org/ploscompbiol/article/fil ... 10061&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pcbi00:1010061

DOI: 10.1371/journal.pcbi.1010061

Access Statistics for this article

More articles in PLOS Computational Biology from Public Library of Science
Bibliographic data for series maintained by ploscompbiol ().