A Clustering Algorithm Based on Document Embedding to Identify Clinical Note Templates

Chunlei Tang, Joseph Michael Plasek, Yun Xiong (), Zhikun Zhang, David Westfall Bates and Li Zhou
Additional contact information
Chunlei Tang: Brigham and Women’s Hospital
Joseph Michael Plasek: Brigham and Women’s Hospital
Yun Xiong: Fudan University
Zhikun Zhang: Fudan University
David Westfall Bates: Brigham and Women’s Hospital
Li Zhou: Brigham and Women’s Hospital

Annals of Data Science, 2021, vol. 8, issue 3, No 4, 497-515

Abstract: Abstract This paper proposes a novel unsupervised document embedding based clustering algorithm to generate clinical note templates. We adapted Charikar’s SimHash to embed each clinical document into a vector representation. We modified the traditional K-means algorithm to merge any two clusters with centroids when they are very close. Under the K-means paradigm, our algorithm designates the cluster representative corresponding to the document vector closest to the centroid as the prototype template. On a corpus of clinical notes, we evaluated the feasibility of utilizing our algorithm at the individual author level. The corpus contains 1,063,893 clinical notes corresponding to 19,146 unique providers between January 2011 and July 2016. Our algorithm achieved more than 80% precision and runs in O(n) time complexity. We further validated our algorithm using human annotators who reported it is able to efficiently detect a real clinical document that can represent the other documents in the same cluster at both the department level and the individual clinician level.

Keywords: Cluster analysis; Medical informatics applications; Electronic health records; Embedding methods; Documentation (search for similar items in EconPapers)
Date: 2021
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s40745-020-00296-8 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:aodasc:v:8:y:2021:i:3:d:10.1007_s40745-020-00296-8

Ordering information: This journal article can be ordered from
https://www.springer ... gement/journal/40745

DOI: 10.1007/s40745-020-00296-8

Access Statistics for this article

Annals of Data Science is currently edited by Yong Shi

More articles in Annals of Data Science from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().