Hand-Guiding Gesture-Based Telemanipulation with the Gesture Mode Classification and State Estimation Using Wearable IMU Sensors

Haegyeom Choi, Haneul Jeon, Donghyeon Noh, Taeho Kim and Donghun Lee ()
Additional contact information
Haegyeom Choi: Mechanical Engineering Department, Soongsil University, Seoul 06978, Republic of Korea
Haneul Jeon: Mechanical Engineering Department, Soongsil University, Seoul 06978, Republic of Korea
Donghyeon Noh: Mechanical Engineering Department, Soongsil University, Seoul 06978, Republic of Korea
Taeho Kim: Mechanical Engineering Department, Soongsil University, Seoul 06978, Republic of Korea
Donghun Lee: Mechanical Engineering Department, Soongsil University, Seoul 06978, Republic of Korea

Mathematics, 2023, vol. 11, issue 16, 1-16

Abstract: This study proposes a telemanipulation framework with two wearable IMU sensors without human skeletal kinematics. First, the states (intensity and direction) of spatial hand-guiding gestures are separately estimated through the proposed state estimator, and the states are also combined with the gesture’s mode (linear, angular, and via) obtained with the bi-directional LSTM-based mode classifier. The spatial pose of the 6-DOF manipulator’s end-effector (EEF) can be controlled by combining the spatial linear and angular motions based on integrating the gesture’s mode and state. To validate the significance of the proposed method, the teleoperation of the EEF to the designated target poses was conducted in the motion-capture space. As a result, it was confirmed that the mode could be classified with 84.5% accuracy in real time, even during the operator’s dynamic movement; the direction could be estimated with an error of less than 1 degree; and the intensity could be successfully estimated with the gesture speed estimator and finely tuned with the scaling factor. Finally, it was confirmed that a subject could place the EEF within the average range of 83 mm and 2.56 degrees in the target pose with only less than ten consecutive hand-guiding gestures and visual inspection in the first trial.

Keywords: hand-guiding gesture; gesture recognition; gesture state estimation; real-time remote control; bi-directional LSTM; wearable sensor (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.mdpi.com/2227-7390/11/16/3514/pdf (application/pdf)
https://www.mdpi.com/2227-7390/11/16/3514/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:11:y:2023:i:16:p:3514-:d:1217192

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().