Multimodal Deception Detection Using Linguistic and Acoustic Features

Tien Nguyen (), Faranak Abri (), Akbar Siami Namin () and Keith S. Jones ()
Additional contact information
Tien Nguyen: San José State University
Faranak Abri: San José State University
Akbar Siami Namin: Texas Tech University
Keith S. Jones: Texas Tech University

A chapter in Machine Learning, Deep Learning and AI for Cybersecurity, 2025, pp 565-598 from Springer

Abstract: Abstract Recently, there has been a growing interest among researchers in the automatic detection of deceptive behavior, actions, and contents. This surge in attention is driven by the wide-ranging applications of deception detection, particularly in criminology and cybersecurity. To advance this line of research, this study investigates both text and audio data derived from speeches in natural languages. We evaluate traditional linguistic models alongside deep models and advanced Large Language Models (LLMs), utilizing Natural Language Processing (NLP) techniques to model deception detection. Furthermore, we employ various feature selection methods to determine the significance of linguistic features. Through extensive experimentation, we assess the effectiveness of both conventional and advanced deep models on transcribed data while also applying deep models to audio data, thus leveraging both types of data to build a multimodal model for deception and lie detection. Our findings indicate that the Bidirectional Long Short-Term Memory (BiLSTM) model excels in processing textual data. On the other hand, the ResNet50 model performs best with audio data. By combining these models in a late fusion approach, we achieve a model that outperforms individual text and audio models.

Date: 2025
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-3-031-83157-7_20

Ordering information: This item can be ordered from
http://www.springer.com/9783031831577

DOI: 10.1007/978-3-031-83157-7_20

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().