Identifying and Mitigating Potential Biases in Predicting Drug Approvals

Qingyang Xu, Elaheh Ahmadi, Alexander Amini, Daniela Rus and Andrew Lo ()
Additional contact information
Qingyang Xu: MIT Laboratory for Financial Engineering
Elaheh Ahmadi: MIT Computer Science and Artificial Intelligence Laboratory
Alexander Amini: MIT Computer Science and Artificial Intelligence Laboratory
Daniela Rus: MIT Computer Science and Artificial Intelligence Laboratory

Drug Safety, 2022, vol. 45, issue 5, No 11, 533 pages

Abstract: Abstract Introduction Machine learning models are increasingly applied to predict the drug development outcomes based on intermediary clinical trial results. A key challenge to this task is to address various forms of bias in the historical drug approval data. Objective We aimed to identify and mitigate the bias in drug approval predictions and quantify the impacts of debiasing in terms of financial value and drug safety. Methods We instantiated the Debiasing Variational Autoencoder, the state-of-the-art model for automated debiasing. We trained and evaluated the model on the Citeline dataset provided by Informa Pharma Intelligence to predict the final drug development outcome from phase II trial results. Results The debiased Debiasing Variational Autoencoder model achieved better performance (measured by the $$F_{1}$$ F 1 score 0.48) in predicting the drug development outcomes than its un-debiased baseline ( $$F_{1}$$ F 1 score 0.25). It had a much higher true-positive rate than baseline (60% vs 15%), while its true-negative rate was slightly lower (88% vs 99%). The Debiasing Variational Autoencoder distinguished between drugs developed by large pharmaceutical firms and those by small biotech companies. The model prediction is strongly influenced by multiple factors such as prior approval of the drug for another indication, whether the trial meets the positive/negative endpoints, and the year when the trial is completed. We estimate that the debiased model generates financial value for the drug developer in six major therapeutic areas, with a range of US$763–1,365 million. Conclusions Our analysis shows that debiasing improves the financial efficiency of late-stage drug development. From the pharmacovigilance perspective, the debiased model is more likely to identify drugs that are both safe and effective. Meanwhile, it may predict a higher probability of success for drugs with potential adverse effects (because of its lower true-negative rate), thus it must be used with caution to predict the development outcomes of drug candidates currently in the pipeline.

Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s40264-022-01160-9 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:drugsa:v:45:y:2022:i:5:d:10.1007_s40264-022-01160-9

Ordering information: This journal article can be ordered from
http://www.springer.com/adis/journal/40264

DOI: 10.1007/s40264-022-01160-9

Access Statistics for this article

Drug Safety is currently edited by Nitin Joshi

More articles in Drug Safety from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().