Video-based hand gesture recognition via SPD manifold spatial representation and optical flow motion features

Zhonghai Bai, Václav Snášel, Seyedali Mirjalili, Bay Vo, Jeng-Shyang Pan, Lingping Kong and Crina Grosan

PLOS ONE, 2026, vol. 21, issue 5, 1-24

Abstract: Hand gesture recognition plays an important role in human–computer interaction, yet accurately modeling both spatial structure and temporal motion patterns in video-based vision systems remains challenging. Many existing approaches focus on either spatial appearance or motion information, which can limit their ability to capture the full complexity of dynamic hand gestures evolving over time. In this work, we present a unified feature representation framework that combines spatial descriptors modeled on the Symmetric Positive Definite (SPD) manifold with temporal motion features extracted from gesture video sequences using grid-based optical flow histograms in Euclidean space. Spatial covariance descriptors are mapped from the SPD manifold to a Euclidean space through the Log-Euclidean metric, enabling effective feature fusion while preserving intrinsic geometric properties. The resulting representation captures complementary spatial and temporal information in a compact and interpretable form. We evaluate the proposed framework on two publicly available video-based benchmark datasets for dynamic hand gesture recognition, the Cambridge Hand Gesture dataset and the Northwestern University Hand Gesture dataset. Experimental results demonstrate that the combined representation consistently improves classification performance compared to using spatial or temporal features alone, achieving 99.31% accuracy on the Cambridge dataset and 97.23% on the Northwestern dataset. These findings indicate that integrating manifold-aware spatial features with motion-based temporal cues provides a practical and effective solution for robust dynamic hand gesture recognition.

Date: 2026
References: Add references at CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0348122 (text/html)
https://journals.plos.org/plosone/article/file?id= ... 48122&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pone00:0348122

DOI: 10.1371/journal.pone.0348122

Access Statistics for this article

More articles in PLOS ONE from Public Library of Science
Bibliographic data for series maintained by plosone ().