Incorporating evidence into mental health Q&A: a novel method to use generative language models for validated clinical content extraction

Ksenia Kharitonova, David Pérez-Fernández, Javier Gutiérrez-Hernando, Asier Gutiérrez-Fandiño, Zoraida Callejas and David Griol

Behaviour and Information Technology, 2025, vol. 44, issue 10, 2333-2350

Abstract: Generative language models have changed the way we interact with computers using natural language. With the release of increasingly advanced GPT models, systems are able to correctly respond to questions in various domains. However, they still have important limitations, such as hallucinations, lack of substance in answers, inability to justify responses, or showing high confidence with fabricated content. In digital mental health, every decision must be traceable and based on scientific evidence and these shortcomings are hindering the integration of LLMs into clinical practice. In this paper, we provide a novel automated method to develop evidence-based question answering systems. Powerful state-of-the-art generalist language models are used and forced to employ only contents in validated clinical guidelines, tracking the source of the evidence for each generated response. This way, the system is able to protect users from hallucinatory responses. As a proof of concept, we present the results obtained building question-answering systems circumscribed to the clinical practice guidelines of the Spanish National Health System about the management of depression and attention deficit hyperactivity disorder. The coherence, veracity, and evidence supporting the responses have been evaluated by human experts obtaining high reliability, clarity, completeness, and traceability of evidence results.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/0144929X.2024.2321959 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:tbitxx:v:44:y:2025:i:10:p:2333-2350

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/tbit20

DOI: 10.1080/0144929X.2024.2321959

Access Statistics for this article

Behaviour and Information Technology is currently edited by Dr Panos P Markopoulos

More articles in Behaviour and Information Technology from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().