Investigating Self-Rationalizing Models for Commonsense Reasoning

Fanny Rancourt (), Paula Vondrlik, Diego Maupomé () and Marie-Jean Meurs ()
Additional contact information
Fanny Rancourt: Department of Computer Science, Université du Québec à Montréal (UQAM), Montreal, QC H2L 2C4, Canada
Paula Vondrlik: Department of Computer Science, Université du Québec à Montréal (UQAM), Montreal, QC H2L 2C4, Canada
Diego Maupomé: Department of Computer Science, Université du Québec à Montréal (UQAM), Montreal, QC H2L 2C4, Canada
Marie-Jean Meurs: Department of Computer Science, Université du Québec à Montréal (UQAM), Montreal, QC H2L 2C4, Canada

Stats, 2023, vol. 6, issue 3, 1-13

Abstract: The rise of explainable natural language processing spurred a bulk of work on datasets augmented with human explanations, as well as technical approaches to leverage them. Notably, generative large language models offer new possibilities, as they can output a prediction as well as an explanation in natural language. This work investigates the capabilities of fine-tuned text-to-text transfer Transformer (T5) models for commonsense reasoning and explanation generation. Our experiments suggest that while self-rationalizing models achieve interesting results, a significant gap remains: classifiers consistently outperformed self-rationalizing models, and a substantial fraction of model-generated explanations are not valid. Furthermore, training with expressive free-text explanations substantially altered the inner representation of the model, suggesting that they supplied additional information and may bridge the knowledge gap. Our code is publicly available, and the experiments were run on open-access datasets, hence allowing full reproducibility.

Keywords: explainability; generative models; large language models; natural language processing; rationales; X-AI (search for similar items in EconPapers)
JEL-codes: C1 C10 C11 C14 C15 C16 (search for similar items in EconPapers)
Date: 2023
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2571-905X/6/3/56/pdf (application/pdf)
https://www.mdpi.com/2571-905X/6/3/56/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jstats:v:6:y:2023:i:3:p:56-919:d:1227747

Access Statistics for this article

Stats is currently edited by Mrs. Minnie Li

More articles in Stats from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().