Portability of a text mining algorithm for detecting adverse drug reactions in electronic health records across diverse patient groups in two Dutch hospitals

Britt W M van de Burgt, Loes F C van Dijck, Bjorn Dullemond, Naomi T Jessurun, Minou van Seyen, Rob J van Marum, Remco J A van Wensen, Wai-Yan Liu, Carolien M J van der Linden, Rene J E Grouls, R Arthur Bouwman, Erik H M Korsten and Toine C G Egberts

PLOS Digital Health, 2026, vol. 5, issue 2, 1-14

Abstract: Adverse Drug Reactions (ADRs) pose a significant challenge in healthcare. While structured documentation of ADRs in electronic health records (EHRs) enables automated alerting, many ADRs are recorded as unstructured free-text, limiting detection. Text mining (TM) shows potential for extracting clinically relevant data from unstructured text. However, the portability of TM algorithms across different institutions and departments remains uncertain, due to variations in EHR structures and documentation practices. To enhance these general-purpose algorithms, evaluating their portability is essential for ensuring effective performance across diverse clinical settings. To evaluate the portability of a previously developed TM-based ADR identification algorithm by assessing its performance using EHRs from two different departments in two different hospitals. EHR free-text data from 62 hospitalized patients in the geriatric and orthopedic departments of two Dutch teaching hospitals were reviewed for ADRs via manual review and the TM algorithm. Performance was evaluated using F-score, sensitivity and positive predictive value (PPV), with comparisons across hospitals and departments. Manual review identified 359 unique ADRs. The TM algorithm detected 534 potential ADRs (pADRs), 286 of which overlapped with manual review, yielding an F-score of 0.64, sensitivity of 80% and PPV of 54%. Performance was consistent across hospitals and departments. Notably, 26 pADRs identified by the algorithm were clinically relevant yet missed in manual review. This study demonstrates portability of the TM algorithm by identifying pADRs across different hospitals and departments without adaptations. These findings support its broader implementation potential for ADR detection in diverse healthcare settings.Author summary: Adverse Drug Reactions (ADRs) present a significant challenge in healthcare, with many being recorded as unstructured free-text in Electronic Health Records (EHRs). This study evaluates the portability of a text mining (TM) algorithm developed for ADR identification in EHRs, by assessing its performance across two departments in two Dutch hospitals. EHR free-text data from 62 patients in the geriatric and orthopedic departments were analyzed using manual review and the TM algorithm. The results showed that the TM algorithm demonstrated a good performance, with an F-score of 0.64, sensitivity of 80%, and positive predictive value (PPV) of 54%. Additionally, the algorithm identified clinically relevant ADRs that were missed in manual review. These findings suggest that the TM algorithm is portable across different clinical settings without requiring adaptation, highlighting its potential for broader implementation in ADR detection.

Date: 2026
References: Add references at CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001230 (text/html)
https://journals.plos.org/digitalhealth/article/fi ... 01230&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pdig00:0001230

DOI: 10.1371/journal.pdig.0001230

Access Statistics for this article

More articles in PLOS Digital Health from Public Library of Science
Bibliographic data for series maintained by digitalhealth ().