BeyondVision: An EMG-driven Micro Hand Gesture Recognition Based on Dynamic Segmentation
BeyondVision: An EMG-driven Micro Hand Gesture Recognition Based on Dynamic Segmentation
Nana Wang, Jianwei Niu, Xuefeng Liu, Dongqin Yu, Guogang Zhu, Xinghao Wu, Mingliang Xu, Hao Su
Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence
Main Track. Pages 6044-6052.
https://doi.org/10.24963/ijcai.2024/668
Hand gesture recognition (HGR) plays a pivotal role in natural and intuitive human-computer interactions. Recent HGR methods focus on recognizing gestures from vision-based images or videos. However, vision-based methods are limited in recognizing micro hand gestures (MHGs) (e.g., pinch within 1cm) and gestures with occluded fingers. To address these issues, combined with the electromyography (EMG) technique, we propose BeyondVision, an EMG-driven MHG recognition system based on deep learning. BeyondVision consists of a wristband-style EMG sampling device and a tailored lightweight neural network BV-Net that can accurately translate EMG signals of MHGs to control commands in real-time. Moreover, we propose a post-processing mechanism and a weight segmentation algorithm to effectively improve the accuracy rate of MHG recognition. Subjective and objective experimental results show that our approach achieves over 95% average recognition rate, 2000Hz sampling frequency, and real-time micro gesture recognition. Our technique has been applied in a commercially available product, introduced at: https://github.com/tyc333/NoBarriers.
Keywords:
Multidisciplinary Topics and Applications: MTA: AI hardware
Computer Vision: CV: Biometrics, face, gesture and pose recognition
Machine Learning: ML: Applications
Multidisciplinary Topics and Applications: MTA: Interactive entertainment