Improving Facial Emotion Recognition Using Residual Autoencoder Coupled Affinity Based Overlapping Reduction

Sankhadeep Chatterjee, Asit Kumar Das, Janmenjoy Nayak and Danilo Pelusi
Additional contact information
Sankhadeep Chatterjee: Department of Computer Science and Technology, Indian Institute of Engineering Science and Technology, Shibpur, Howrah 711101, India
Asit Kumar Das: Department of Computer Science and Technology, Indian Institute of Engineering Science and Technology, Shibpur, Howrah 711101, India
Janmenjoy Nayak: Department of Computer Science, Maharaja Sriram Chandra Bhanja Deo (MSCBD) University, Baripada 757003, India
Danilo Pelusi: Department of Communications Sciences, University of Teramo, 64100 Teramo, Italy

Mathematics, 2022, vol. 10, issue 3, 1-18

Abstract: Emotion recognition using facial images has been a challenging task in computer vision. Recent advancements in deep learning has helped in achieving better results. Studies have pointed out that multiple facial expressions may present in facial images of a particular type of emotion. Thus, facial images of a category of emotion may have similarity to other categories of facial images, leading towards overlapping of classes in feature space. The problem of class overlapping has been studied primarily in the context of imbalanced classes. Few studies have considered imbalanced facial emotion recognition. However, to the authors’ best knowledge, no study has been found on the effects of overlapped classes on emotion recognition. Motivated by this, in the current study, an affinity-based overlap reduction technique (AFORET) has been proposed to deal with the overlapped class problem in facial emotion recognition. Firstly, a residual variational autoencoder (RVA) model has been used to transform the facial images to a latent vector form. Next, the proposed AFORET method has been applied on these overlapped latent vectors to reduce the overlapping between classes. The proposed method has been validated by training and testing various well known classifiers and comparing their performance in terms of a well known set of performance indicators. In addition, the proposed AFORET method is compared with already existing overlap reduction techniques, such as the OSM, ν -SVM, and NBU methods. Experimental results have shown that the proposed AFORET algorithm, when used with the RVA model, boosts classifier performance to a greater extent in predicting human emotion using facial images.

Keywords: facial emotion recognition; class overlapping; residual networks; variational autoencoders (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2022
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/10/3/406/pdf (application/pdf)
https://www.mdpi.com/2227-7390/10/3/406/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:10:y:2022:i:3:p:406-:d:736001

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().