What do algorithms explain? The issue of the goals and capabilities of Explainable Artificial Intelligence (XAI)

Moritz Renftle, Holger Trittenbach, Michael Poznic () and Reinhard Heil
Additional contact information
Moritz Renftle: Independent Scholar
Holger Trittenbach: Neurocat GmbH
Michael Poznic: Institute for Technology Assessment and Systems Analysis (ITAS)
Reinhard Heil: Institute for Technology Assessment and Systems Analysis (ITAS)

Palgrave Communications, 2024, vol. 11, issue 1, 1-10

Abstract: Abstract The increasing ubiquity of machine learning (ML) motivates research on algorithms to “explain” models and their predictions—so-called Explainable Artificial Intelligence (XAI). Despite many publications and discussions, the goals and capabilities of such algorithms are far from being well understood. We argue that this is because of a problematic reasoning scheme in the literature: Such algorithms are said to complement machine learning models with desired capabilities, such as interpretability or explainability. These capabilities are in turn assumed to contribute to a goal, such as trust in a system. But most capabilities lack precise definitions and their relationship to such goals is far from obvious. The result is a reasoning scheme that obfuscates research results and leaves an important question unanswered: What can one expect from XAI algorithms? In this paper, we clarify the modest capabilities of these algorithms from a concrete perspective: that of their users. We show that current algorithms can only answer user questions that can be traced back to the question: “How can one represent an ML model as a simple function that uses interpreted attributes?”. Answering this core question can be trivial, difficult or even impossible, depending on the application. The result of the paper is the identification of two key challenges for XAI research: the approximation and the translation of ML models.

Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1057/s41599-024-03277-x Abstract (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:pal:palcom:v:11:y:2024:i:1:d:10.1057_s41599-024-03277-x

Ordering information: This journal article can be ordered from
https://www.nature.com/palcomms/about

DOI: 10.1057/s41599-024-03277-x

Access Statistics for this article

More articles in Palgrave Communications from Palgrave Macmillan
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().