Speech Emotion Recognition Using Attention Model

Jagjeet Singh, Lakshmi Babu Saheer () and Oliver Faust
Additional contact information
Jagjeet Singh: School of Computing and Information Science Research, Anglia Ruskin University, Cambridge CB1 1PT, UK
Lakshmi Babu Saheer: School of Computing and Information Science Research, Anglia Ruskin University, Cambridge CB1 1PT, UK
Oliver Faust: School of Computing and Information Science Research, Anglia Ruskin University, Cambridge CB1 1PT, UK

IJERPH, 2023, vol. 20, issue 6, 1-21

Abstract: Speech emotion recognition is an important research topic that can help to maintain and improve public health and contribute towards the ongoing progress of healthcare technology. There have been several advancements in the field of speech emotion recognition systems including the use of deep learning models and new acoustic and temporal features. This paper proposes a self-attention-based deep learning model that was created by combining a two-dimensional Convolutional Neural Network (CNN) and a long short-term memory (LSTM) network. This research builds on the existing literature to identify the best-performing features for this task with extensive experiments on different combinations of spectral and rhythmic information. Mel Frequency Cepstral Coefficients (MFCCs) emerged as the best performing features for this task. The experiments were performed on a customised dataset that was developed as a combination of RAVDESS, SAVEE, and TESS datasets. Eight states of emotions (happy, sad, angry, surprise, disgust, calm, fearful, and neutral) were detected. The proposed attention-based deep learning model achieved an average test accuracy rate of 90%, which is a substantial improvement over established models. Hence, this emotion detection model has the potential to improve automated mental health monitoring.

Keywords: speech emotion recognition; self-attention models; convolutional neural networks; long short-term memory; RAVDESS; SAVEE; TESS (search for similar items in EconPapers)
JEL-codes: I I1 I3 Q Q5 (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1660-4601/20/6/5140/pdf (application/pdf)
https://www.mdpi.com/1660-4601/20/6/5140/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jijerp:v:20:y:2023:i:6:p:5140-:d:1097389

Access Statistics for this article

IJERPH is currently edited by Ms. Jenna Liu

More articles in IJERPH from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().