Hierarchical-attention-based neural network for gait emotion recognition

Sainan Zhang, Jun Zhang, Weiguo Song, Longnan Yang and Xuedan Zhao

Physica A: Statistical Mechanics and its Applications, 2024, vol. 637, issue C

Abstract: Human gait is an emerging biometric feature and contains important information for long-distance emotion recognition. However, sadness and neutral emotions are easily misjudged during the recognition process, because the body posture of the two emotions is quite similar. Existing methods have difficulty in distinguishing these two emotions satisfactorily since they treat all action features equally without differentiating their contribution to recognition, or they ignore the motion information in gait that can express emotion. In this paper, we propose a novel hierarchical attention neural network, which can automatically learn the affective features contained in human motion and action, and effectively distinguish between sad and neutral emotions. The network consists of three modules: motion sentiment module (MSM), action sentiment module (ASM) and emotion classifier. Specifically, MSM is composed of a position encoder and a velocity encoder. It extracts affective features from motion information and helps to distinguish the sad and neutral emotions based on gait velocity. ASM consists of an action encoder that compresses the discriminative human actions into a latent space. Emotion classifier recognizes the emotion based on the outputs from MSM and ASM. Moreover, we present a feature preprocessing method to deal with the problem of imbalanced data categories. Experiments demonstrate that our approach enhances the discriminability of sad and neutral emotions and performs better than many state-of-the-art methods. In addition, ablation experiments further verify that both velocity and action features are important for the gait-based emotion recognition task.

Keywords: Emotion recognition; Gait; Hierarchical attention network; Deep learning; Feature fusion (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0378437124001080
Full text for ScienceDirect subscribers only. Journal offers the option of making the article available online on Science direct for a fee of $3,000

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:phsmap:v:637:y:2024:i:c:s0378437124001080

DOI: 10.1016/j.physa.2024.129600

Access Statistics for this article

Physica A: Statistical Mechanics and its Applications is currently edited by K. A. Dawson, J. O. Indekeu, H.E. Stanley and C. Tsallis

More articles in Physica A: Statistical Mechanics and its Applications from Elsevier
Bibliographic data for series maintained by Catherine Liu ().