Supervised Machine Learning-Based Decision Support for Signal Validation Classification

Muhammad Imran (), Aasia Bhatti, David M. King, Magnus Lerch, Jürgen Dietrich, Guy Doron and Katrin Manlik
Additional contact information
Muhammad Imran: Bayer AG, Digital Transformation and Information Technology Pharma, Decision Science and Advanced Analytics for Medical Affairs, Pharmacovigilance and Regulatory Affairs
Aasia Bhatti: Bayer US LLC, Pharmaceuticals, Pharmacovigilance, Benefit-Risk Management TA Radiology
David M. King: Bayer US LLC, Digital Transformation and Information Technology Pharma, Adverse Event Management
Magnus Lerch: Lenolution GmbH
Jürgen Dietrich: Bayer AG, Pharmaceuticals, Pharmacovigilance, Innovation and Digitalization
Guy Doron: Bayer AG, Pharmaceuticals, Pharmacovigilance, R&D, Data Sciences
Katrin Manlik: Bayer AG, Pharmaceuticals, Pharmacovigilance, Data Science and Insight Generation

Drug Safety, 2022, vol. 45, issue 5, No 16, 583-596

Abstract: Abstract Introduction Signal validation in pharmacovigilance is the process of evaluating data to decide whether evidence is sufficient to justify further assessment of a detected signal. During the signal validation process, safety experts in our organization are required to review signals of disproportionate reporting (SDRs) and classify them into one of six predefined categories. Objective This experiment explored the extent to which predictive machine learning (ML) models can support the decision making of safety experts by accurately identifying the most appropriate predefined signal validation category. Methods We extracted cumulative data for six medicinal products, consisting of historic SDR validations and Individual Case Safety Reports, from the company’s safety database for training and testing of the ML model. We implemented a decision tree-based supervised multiclass classifier model termed Gradient Boosted Trees followed by a SHapley Additive exPlanations (SHAP) analysis to mitigate the “black box” effect of the ensemble model by identifying the key predicting features in the model. Following a retrospective analysis, a prospective experiment was conducted to test the model accuracy and user acceptance in a real-life setting. Results The prediction accuracy of our ML model ranged from 83 to 86% over 3 months for the six medicinal products. The applicability of the model was confirmed by the company’s safety experts. Additionally, the systematic predictions provided valuable information to the safety experts and assisted them in reviewing the SDRs efficiently and consistently. Conclusions This experiment demonstrated that it is possible to train a multiclass classification model to accurately predict signal validation categories for SDRs. More importantly, the transparency of the predictions provided by the SHAP analysis led to high acceptance by the safety experts.

Date: 2022
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s40264-022-01159-2 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:drugsa:v:45:y:2022:i:5:d:10.1007_s40264-022-01159-2

Ordering information: This journal article can be ordered from
http://www.springer.com/adis/journal/40264

DOI: 10.1007/s40264-022-01159-2

Access Statistics for this article

Drug Safety is currently edited by Nitin Joshi

More articles in Drug Safety from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().