Optical Flow Magnification and Cosine Similarity Feature Fusion Network for Micro-Expression Recognition

Heyou Chang, Jiazheng Yang, Kai Huang, Wei Xu, Jian Zhang and Hao Zheng ()
Additional contact information
Heyou Chang: School of Information Engineering, Nanjing Xiaozhuang University, Nanjing 211171, China
Jiazheng Yang: School of Information Engineering, Nanjing Xiaozhuang University, Nanjing 211171, China
Kai Huang: School of Computer Engineering, Jiangsu Ocean University, Lianyungang 222005, China
Wei Xu: School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China
Jian Zhang: School of Computer Engineering, Jiangsu Ocean University, Lianyungang 222005, China
Hao Zheng: School of Information Engineering, Nanjing Xiaozhuang University, Nanjing 211171, China

Mathematics, 2025, vol. 13, issue 15, 1-18

Abstract: Recent advances in deep learning have significantly advanced micro-expression recognition, yet most existing methods process the entire facial region holistically, struggling to capture subtle variations in facial action units, which limits recognition performance. To address this challenge, we propose the Optical Flow Magnification and Cosine Similarity Feature Fusion Network (MCNet). MCNet introduces a multi-facial action optical flow estimation module that integrates global motion-amplified optical flow with localized optical flow from the eye and mouth–nose regions, enabling precise capture of facial expression nuances. Additionally, an enhanced MobileNetV3-based feature extraction module, incorporating Kolmogorov–Arnold networks and convolutional attention mechanisms, effectively captures both global and local features from optical flow images. A novel multi-channel feature fusion module leverages cosine similarity between Query and Key token sequences to optimize feature integration. Extensive evaluations on four public datasets—CASME II, SAMM, SMIC-HS, and MMEW—demonstrate MCNet’s superior performance, achieving state-of-the-art results with 92.88% UF1 and 86.30% UAR on the composite dataset, surpassing the best prior method by 1.77% in UF1 and 6.0% in UAR.

Keywords: micro-expression recognition; KAN; optical flow; motion magnification; feature fusion (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/13/15/2330/pdf (application/pdf)
https://www.mdpi.com/2227-7390/13/15/2330/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:13:y:2025:i:15:p:2330-:d:1707173

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().