A Weakly-Supervised Named Entity Recognition Machine Learning Approach for Emergency Medical Services Clinical Audit

Han Wang, Wesley Lok Kin Yeung, Qin Xiang Ng, Angeline Tung, Joey Ai Meng Tay, Davin Ryanputra, Marcus Eng Hock Ong, Mengling Feng and Shalini Arulanandam
Additional contact information
Han Wang: Saw Swee Hock School of Public Health, National University of Singapore, Singapore 117549, Singapore
Wesley Lok Kin Yeung: Singapore Civil Defence Force, Singapore 408827, Singapore
Qin Xiang Ng: Singapore Civil Defence Force, Singapore 408827, Singapore
Angeline Tung: Singapore Civil Defence Force, Singapore 408827, Singapore
Joey Ai Meng Tay: Singapore Civil Defence Force, Singapore 408827, Singapore
Davin Ryanputra: Singapore Civil Defence Force, Singapore 408827, Singapore
Marcus Eng Hock Ong: Health Services Research Centre, Singapore Health Services, Singapore 169856, Singapore
Mengling Feng: Saw Swee Hock School of Public Health, National University of Singapore, Singapore 117549, Singapore
Shalini Arulanandam: Singapore Civil Defence Force, Singapore 408827, Singapore

IJERPH, 2021, vol. 18, issue 15, 1-11

Abstract: Clinical performance audits are routinely performed in Emergency Medical Services (EMS) to ensure adherence to treatment protocols, to identify individual areas of weakness for remediation, and to discover systemic deficiencies to guide the development of the training syllabus. At present, these audits are performed by manual chart review, which is time-consuming and laborious. In this paper, we report a weakly-supervised machine learning approach to train a named entity recognition model that can be used for automatic EMS clinical audits. The dataset used in this study contained 58,898 unlabeled ambulance incidents encountered by the Singapore Civil Defence Force from 1st April 2019 to 30th June 2019. With only 5% labeled data, we successfully trained three different models to perform the NER task, achieving F1 scores of around 0.981 under entity type matching evaluation and around 0.976 under strict evaluation. The BiLSTM-CRF model was 1~2 orders of magnitude lighter and faster than our BERT-based models. Our proposed proof-of-concept approach may improve the efficiency of clinical audits and can also help with EMS database research. Further external validation of this approach is needed.

Keywords: emergency medical services; clinical audit; natural language processing; named entity recognition; weakly-supervised learning (search for similar items in EconPapers)
JEL-codes: I I1 I3 Q Q5 (search for similar items in EconPapers)
Date: 2021
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1660-4601/18/15/7776/pdf (application/pdf)
https://www.mdpi.com/1660-4601/18/15/7776/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jijerp:v:18:y:2021:i:15:p:7776-:d:599290

Access Statistics for this article

IJERPH is currently edited by Ms. Jenna Liu

More articles in IJERPH from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().