Are numerical or verbal explanations of AI the key to appropriate user reliance and error detection? An experimental study with a classification algorithm

Jörg Papenkordt (), Axel-Cyrille Ngonga Ngomo () and Kirsten Thommes ()
Additional contact information
Jörg Papenkordt: Paderborn University
Axel-Cyrille Ngonga Ngomo: Paderborn University
Kirsten Thommes: Paderborn University

No 147, Working Papers Dissertations from Paderborn University, Faculty of Business Administration and Economics

Abstract: Advances in AI and our limited human capabilities have made AI decision-making opaque to humans. One prerequisite for enhancing the transparency of AI recommendations is improving AI explainability as humans need to be enabled to take responsibility for their actions even with AI support. Our study aims to tackle this issue by investigating two basic approaches to explainability: We evaluate numerical explanations, such as certainty measures, against verbal explanations, such as those provided by LLM as explanatory agents. Specifically, we examine whether verbal or numerical (or both) explanations in tasks of high uncertainty lure users into false beliefs or, on the contrary, promote appropriate reliance. Drawing on an experiment with 441 participants, we explore the dynamics of non-expert users' interactions with AI under varying explanatory conditions. Results show that explanations significantly improve reliance and decision accuracy. Numerical explanations aid in identifying uncertainties and errors, but the users' reliance on the advice falls far behind the given numerical certainty. Verbal explanations foster higher reliance while increasing the risk of over-reliance. Combining both explanation types enhances reliance but further amplifies blind trust in AI.

Keywords: explainable AI; artificial intelligence; human-computer interaction (search for similar items in EconPapers)
JEL-codes: C83 C88 D81 O33 (search for similar items in EconPapers)
Pages: 36
Date: 2025-07
New Economics Papers: this item is included in nep-ain, nep-exp and nep-mac
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://groups.uni-paderborn.de/wp-wiwi/RePEc/pdf/dispap/DP147.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:pdn:dispap:147

Access Statistics for this paper

More papers in Working Papers Dissertations from Paderborn University, Faculty of Business Administration and Economics Contact information at EDIRC.
Bibliographic data for series maintained by WP-WiWi-Info ().