Entity-enhanced BERT for medical specialty prediction based on clinical questionnaire data

Soyeon Lee, Ye Ji Han, Hyun Joon Park, Byung Hoon Lee, DaHee Son, SoYeon Kim, HyeonJong Yang, TaeJun Han, EunSun Kim and Sung Won Han

PLOS ONE, 2025, vol. 20, issue 1, 1-18

Abstract: A medical specialty prediction system for remote diagnosis can reduce the unexpected costs incurred by first-visit patients who visit the wrong hospital department for their symptoms. To develop medical specialty prediction systems, several researchers have explored clinical predictive models using real medical text data. Medical text data include large amounts of information regarding patients, which increases the sequence length. Hence, a few studies have attempted to extract entities from the text as concise features and provide domain-specific knowledge for clinical text classification. However, it is still insufficient to inject them into the model effectively. Thus, we propose Entity-enhanced BERT (E-BERT), which utilizes the structural attributes of BERT for medical specialty prediction. E-BERT has an entity embedding layer and entity-aware attention to inject domain-specific knowledge and focus on relationships between medical-related entities within the sequences. Experimental results on clinical questionnaire data demonstrate the superiority of E-BERT over the other benchmark models, regardless of the input sequence length. Moreover, the visualization results for the effects of entity-aware attention prove that E-BERT effectively incorporate domain-specific knowledge and other information, enabling the capture of contextual information in the text. Finally, the robustness and applicability of the proposed method is explored by applying it to other Pre-trained Language Models. These effective medical specialty predictive model can provide practical information to first-visit patients, resulting in streamlining the diagnostic process and improving the quality of medical consultations.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0317795 (text/html)
https://journals.plos.org/plosone/article/file?id= ... 17795&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pone00:0317795

DOI: 10.1371/journal.pone.0317795

Access Statistics for this article

More articles in PLOS ONE from Public Library of Science
Bibliographic data for series maintained by plosone ().