Narrative Search Engine for Case Series Assessment Supported by Artificial Intelligence Query Suggestions

Alem Zekarias (), Eva-Lisa Meldau, Shachi Bista, Joana Félix China and Lovisa Sandberg
Additional contact information
Alem Zekarias: Uppsala Monitoring Centre
Eva-Lisa Meldau: Uppsala Monitoring Centre
Shachi Bista: Uppsala Monitoring Centre
Joana Félix China: Uppsala Monitoring Centre
Lovisa Sandberg: Uppsala Monitoring Centre

Drug Safety, 2025, vol. 48, issue 7, No 5, 767-779

Abstract: Abstract Introduction Manual identification of case narratives with specific relevant information can be challenging when working with large numbers of adverse event reports (case series). The process can be supported with a search engine, but building search queries often remains a manual task. Suggesting terms to add to the search query could support assessors in the identification of case narratives within a case series. Objective The aim of this study is to explore the feasibility of identifying case narratives containing specific characteristics with a narrative search engine supported by artificial intelligence (AI) query suggestions. Methods The narrative search engine uses Best Match 25 (BM25) and suggests additional query terms from two word embedding models providing English and biomedical words to a human in the loop. We calculated the percentage of relevant narratives retrieved by the system (recall) and the percentage of retrieved narratives relevant to the search (precision) on an evaluation dataset including narratives from VigiBase, the World Health Organization global database of adverse event reports for medicines and vaccines. Exact-match search and BM25 search with the Relevance Model (RM3), an alternative way to expand queries, were used as comparators. Results The gold standard included 55/750 narratives labelled as relevant. Our narrative search engine retrieved on average 56.4% of the relevant narratives (recall), which is higher when compared with exact-match search (21.8%), without a significant drop in precision (54.5% to 43.1%). The recall is also higher as compared with RM3 (34.4%). Conclusions Our study demonstrates that a narrative search engine supported by AI query suggestions can be a viable alternative to an exact-match search and BM25 search with RM3, since it can facilitate the retrieval of additional relevant narratives during signal assessments.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s40264-025-01529-6 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:drugsa:v:48:y:2025:i:7:d:10.1007_s40264-025-01529-6

Ordering information: This journal article can be ordered from
http://www.springer.com/adis/journal/40264

DOI: 10.1007/s40264-025-01529-6

Access Statistics for this article

Drug Safety is currently edited by Nitin Joshi

More articles in Drug Safety from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().