Impacts of Predictive Genomic Classifier Performance on Subpopulation-Specific Treatment Effects Assessment

Sue-Jane Wang () and Ming-Chung Li
Additional contact information
Sue-Jane Wang: OTS, CDER, US FDA
Ming-Chung Li: National Cancer Institute, National Institutes of Health

Statistics in Biosciences, 2016, vol. 8, issue 1, No 8, 129-158

Abstract: Abstract We consider three (strong, moderate and mild) predictive biomarker scenarios with varying prevalence. As such, there is no treatment effect in the biomarker negative (g −) patient subpopulation. Relative to g −, there is a four-fold profound treatment effect in the biomarker positive (g +) patient subpopulation, a strongly predictive scenario; a three-fold large g + subpopulation treatment effect, a moderately predictive scenario; and a two-fold modest g + subpopulation treatment effect, a mildly predictive scenario. In this paper, we focus on binary endpoint in prescribing treatment effect. Using a Breiman’s (Mach. Learn. 24:123–140, 1996) machine learning voting algorithm via a k-fold cross-validated approach applied by Freidlin et al. (Clin. Cancer Res. 16:691–698, 2010), a predictive biomarker may be developed. We consider development or discovery of a genomic biomarker using microarray gene expressions data in randomized controlled trials and validate the biomarker’s predictive performance in an independent data set. We investigate the classification performance characteristics of a binary genomic composite biomarker (expected to be predictive of treatment effects) including sensitivity, specificity, accuracy, positive predictive value and negative predictive value as a function of true sensitive prevalence. In doing so, we report the finding based on three representative tuning parameter sets with varying degree of rigor in their choices of the parameters ranging from highly rigorous, moderately rigorous to mildly rigorous. We articulate the rationales on the choices of tuning parameter sets. We also study the impacts of misclassification of genomic biomarker classifiers on their assessment of treatment effects in the positive and negative patient subpopulations, and all-comer patients. We elucidate via simulation studies on approaches to improve sensitivity when a biomarker is highly specific but poorly sensitive, a scenario that is most likely to lead to an incorrect test conclusion of an applicable significant treatment effect in a specific patient subpopulation or both positive and negative subpopulations. We explore when it will be beneficial to develop a binary predictive biomarker and conclude that hypothesis test inferences for the g + subpopulation treatment effect in the dual hypotheses setting (all-comer and g + alone) cannot be relied upon if the biomarker classifier is only highly specific and poorly sensitive or resulting in poor negative predictive value. The converse dual hypotheses (all-comer and g − alone) have the same concern, viz. highly sensitive and poorly specific or resulting in poor positive predictive value. In addition, we compare the predictive performance of a biomarker classifier between use of direct selection and selection from a candidate pool shedding favorable lights of direct selection approach where biological or mechanistic plausibility can be relied upon. Further research is needed if accurate classifier is required irrespective of prevalence level.

Keywords: Composite genomic biomarker; Cross-validated machine learning voting algorithm; Pharmacogenomics; Positive or negative predictive value; Prevalence; Subpopulation treatment effect (search for similar items in EconPapers)
Date: 2016
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s12561-013-9092-y Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:stabio:v:8:y:2016:i:1:d:10.1007_s12561-013-9092-y

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/12561

DOI: 10.1007/s12561-013-9092-y

Access Statistics for this article

Statistics in Biosciences is currently edited by Hongyu Zhao and Xihong Lin

More articles in Statistics in Biosciences from Springer, International Chinese Statistical Association
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().