Voting based optimized deep ensemble model: an effective visual and textual fusion for recognition of face emotions

Dipti Pandit () and Sangeeta Jadhav ()
Additional contact information
Dipti Pandit: Vishwakarma Institute of Technology
Sangeeta Jadhav: Army Institute of Technology

Computational Statistics, 2025, vol. 40, issue 9, No 25, 5537-5572

Abstract: Abstract Facial emotion detection has been receiving widespread interest due to its broad applications in various fields. Nevertheless, unifying disparate data modalities, like visual and text data, is a complicated procedure. This research introduces a Voting-based Optimized Deep Ensemble (VODE) Model designed to solve these challenges by leveraging base learners and meta learners to enhance facial emotion recognition accuracy. This model incorporates advanced feature extraction modules, involving Scale-Invariant Feature Transform (SIFT) for visual data and FastText for textual data, to effectively capture key emotional cues. A multi-head transformer model is developed to efficiently fuse these features, capable of capturing intricate relationships between different modalities. Additionally, it utilizes an ensemble learning model by combining predictions from multiple deep networks, such as VGG19, DenseNet121, ResNetv2, InceptionResNet, and MobileNet, with fine-tuned support vector machine and gradient boosting algorithms for emotion recognition These ensemble predictions are then optimized through a majority voting mechanism, further enhancing the model's performance. The experimental outcomes establish significant improvements in emotion recognition, also validating the efficiency of the introduced model. The approach attained 98% of accuracy, 98% of precision, and 98.5% of recall. These metrics underscore the higher functionality of the introduced approaches when compared to existing techniques.

Keywords: Facial emotion recognition; Majority voting; VGG19; DenseNet121; ResNetv2; InceptionResNet; MobileNet (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s00180-025-01666-7 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:compst:v:40:y:2025:i:9:d:10.1007_s00180-025-01666-7

Ordering information: This journal article can be ordered from
http://www.springer.com/statistics/journal/180/PS2

DOI: 10.1007/s00180-025-01666-7

Access Statistics for this article

Computational Statistics is currently edited by Wataru Sakamoto, Ricardo Cao and Jürgen Symanzik

More articles in Computational Statistics from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().