Multimodal Biometric Emotion Recognition for Mental Health Monitoring Using AI-DL Techniques

Sk Md Hasnat () and Tilendra Shishir Sinha
Additional contact information
Sk Md Hasnat: Maulana Abul Kalam Azad University of Technology, Departments of Information Technology
Tilendra Shishir Sinha: Maulana Abul Kalam Azad University of Technology, Faculty of Computer Science and Engineering

A chapter in AI in Smart and Secure Healthcare, 2026, pp 309-328 from Springer

Abstract: Abstract This study shows a better version of a sentiment analysis system that uses different types of body signals with the help of smart AI and deep learning methods. It uses the CMU-MOSEI dataset and mixes face expressions, voice features, and text meaning to understand emotions more deeply. A Modified Cross-Modality Transformer (M-CMT) is used for audio by working with Mel spectrograms. ResNet50 is used to take features from face images, and BERT is used for the text part, which is also a transformer-based model. In this system, a self-attention layer helps to understand better how the features from the same type of input are connected, and cross-attention is used to bring together all the different inputs for more correct emotion recognition. The main goal of this system is to help watch over mental health. Finding emotional problems early can help give treatment on time, and also make the treatment work better. This work helps move forward the field of smart systems that can understand emotions.

Keywords: Multimodal; Emotion recognition; Cross-modality fusion; Mel spectrogram representation; Self-attention; Cross-attention; Emotion-aware (search for similar items in EconPapers)
Date: 2026
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:spochp:978-3-032-15092-9_12

Ordering information: This item can be ordered from
http://www.springer.com/9783032150929

DOI: 10.1007/978-3-032-15092-9_12

Access Statistics for this chapter

More chapters in Springer Optimization and Its Applications from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().