{"type":"ScholarlyArticle","title":"Semi-Supervised Learning for Surface EMG-based Gesture Recognition","authors":[{"name":"Yu Du","slug":"yu-du","given":"Yu","family":"Du","affiliation":"College of Computer Science , Zhejiang University"},{"name":"Yongkang Wong","slug":"yongkang-wong","given":"Yongkang","family":"Wong","affiliation":"Smart Systems Institute , National University of Singapore"},{"name":"Wenguang Jin","slug":"wenguang-jin","given":"Wenguang","family":"Jin","affiliation":"College of Information Science and Electronic Engineering , Zhejiang University"},{"name":"Wentao Wei","slug":"wentao-wei","given":"Wentao","family":"Wei","affiliation":"College of Computer Science , Zhejiang University"},{"name":"Yu Hu","slug":"yu-hu","given":"Yu","family":"Hu","affiliation":"College of Computer Science , Zhejiang University"},{"name":"Mohan Kankanhalli","slug":"mohan-kankanhalli","given":"Mohan","family":"Kankanhalli","affiliation":"School of Computing , National University of Singapore"},{"name":"Weidong Geng","slug":"weidong-geng","given":"Weidong","family":"Geng","affiliation":"College of Computer Science , Zhejiang University"}],"abstract":"Conventionally, gesture recognition based on non-intrusive muscle-computer interfaces required a strongly-supervised learning algorithm and a large amount of labeled training signals of surface electromyography (sEMG). In this work, we show that temporal relationship of sEMG signals and data glove provides implicit supervisory signal for learning the gesture recognition model. To demonstrate this, we present a semi-supervised learning framework with a novel Siamese architecture for sEMG-based gesture recognition. Specifically, we employ auxiliary tasks to learn visual representation; predicting the temporal order of two consecutive sEMG frames; and, optionally, predicting the statistics of 3D hand pose with a sEMG frame. Experiments on the NinaPro, CapgMyo and csl-hdemg datasets validate the efficacy of our proposed approach, especially when the labeled samples are very scarce.","doi":"10.24963/ijcai.2017/225","url":"https://ijcai.org/proceedings/2017/225","pdf":"https://ijcai.org/proceedings/2017/0225.pdf","markdown":"https://ijcai.org/proceedings/2017/225.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/225","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":225,"pages":{"first":1624,"last":1630},"keywords":["Machine Learning: Deep Learning","Multidisciplinary Topics and Applications: Human-Computer Interaction","Machine Learning: Semi-Supervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Amma et al., 2015] Christoph Amma, Thomas Krings, Jonas Böer, and Tanja Schultz. Advancing muscle- computer interfaces with high-density electromyography. In SIGCHI, pages 929-938, 2015.","title":"Advancing musclecomputer interfaces with high-density electromyography","authors":"Christoph Amma, Thomas Krings, Jonas Böer, Tanja Schultz","year":"2015","venue":"SIGCHI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Atzori et al., 2012] M Atzori, A Gijsberts, S Heynen, A M Hager, O Deriaz, P van der Smagt, C Castellini, B Caputo, and H Muller. Building the Ninapro database: a resource for the biorobotics community. In IEEE RAS EMBS Inter- national Conference on BioRob, pages 1258-1265, 2012.","title":"Building the Ninapro database: a resource for the biorobotics community","authors":"Atzori, S Gijsberts, A M Heynen, O Hager, P Deriaz, C Van Der Smagt, B Castellini, H Caputo, Muller","year":"2012","venue":"IEEE RAS EMBS International Conference on BioRob","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Atzori et al., 2014] Manfredo Atzori, Arjan Gijsberts, Clau- dio Castellini, Barbara Caputo, Anne-Gabrielle Mittaz Hager, Simone Elsig, Giorgio Giatsidis, Franco Bassetto, and Henning Müller. Electromyography data for non- invasive naturally-controlled robotic hand prostheses. Sci- entific Data, 1, 2014.","title":"Electromyography data for noninvasive naturally-controlled robotic hand prostheses","authors":"Manfredo Atzori, Arjan Gijsberts, Claudio Castellini, Barbara Caputo, Anne-Gabrielle Mittaz Hager, Simone Elsig, Giorgio Giatsidis, Franco Bassetto, Henning Müller","year":"2014","venue":"Scientific Data","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Atzori et al., 2016] Manfredo Atzori, Matteo Cognolato, and Henning Müller. deep learning with convolutional neural networks applied to electromyography data: a re- source for the classification of movements for prosthetic hands. Frontiers in Neurorobotics, 10, 2016.","title":"deep learning with convolutional neural networks applied to electromyography data: a resource for the classification of movements for prosthetic hands","authors":"Manfredo Atzori, Matteo Cognolato, Henning Müller","year":"2016","venue":"Frontiers in Neurorobotics","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bengio et al., 2014] Yoshua Bengio, Eric Laufer, Guil- laume Alain, and Jason Yosinski. Deep generative stochas- tic networks trainable by backprop. In ICML, 2014.","title":"Deep generative stochastic networks trainable by backprop","authors":"Yoshua Bengio, Eric Laufer, Guillaume Alain, Jason Yosinski","year":"2014","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Caruana, 1997] Rich Caruana. Multitask learning. Machine Learning, 28:41-75, 1997.","title":"Multitask learning","authors":"Caruana","year":"1997","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2015] Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, and Zheng Zhang. MxNet: a flexible and efficient machine learning library for heterogeneous dis- tributed systems. In NIPS Workshop, 2015.","title":"MxNet: a flexible and efficient machine learning library for heterogeneous distributed systems","authors":"Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, Zheng Zhang","year":"2015","venue":"NIPS Workshop","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cheng et al., 2016] Yanhua Cheng, Xin Zhao, Rui Cai, Zhi- wei Li, Kaiqi Huang, and Yong Rui. Semi-supervised mul- timodal deep learning for RGB-D object recognition. In IJCAI, pages 3345-3351, 2016.","title":"Semi-supervised multimodal deep learning for RGB-D object recognition","authors":"Yanhua Cheng, Xin Zhao, Rui Cai, Zhiwei Li, Kaiqi Huang, Yong Rui","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/473"},{"position":9,"raw":"[Doersch et al., 2015] Carl Doersch, Abhinav Gupta, and Alexei A Efros. Unsupervised visual representation learn- ing by context prediction. In ICCV, pages 1422-1430, 2015.","title":"Unsupervised visual representation learning by context prediction","authors":"Carl Doersch, Abhinav Gupta, Alexei A Efros","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Farina et al., 2014] Dario Farina, Ning Jiang, Hubertus Re- hbaum, Aleš Holobar, Bernhard Graimann, Hans Dietl, and Oskar C Aszmann. The extraction of neural infor- mation from the surface EMG for the control of upper- limb prostheses: emerging avenues and challenges. IEEE Trans. Neural Syst. Rehabil. Eng., 22(4):797-809, 2014.","title":"The extraction of neural information from the surface EMG for the control of upperlimb prostheses: emerging avenues and challenges","authors":"Dario Farina, Ning Jiang, Hubertus Rehbaum, Aleš Holobar, Bernhard Graimann, Hans Dietl, Oskar C Aszmann","year":"2014","venue":"IEEE Trans. Neural Syst. Rehabil. Eng","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fowler, 1983] Carol A Fowler. Converging sources of evi- dence on spoken and perceived rhythms of speech: cyclic production of vowels in monosyllabic stress feet. Journal of Experimental Psychology: General, 112(3):386, 1983.","title":"Converging sources of evidence on spoken and perceived rhythms of speech: cyclic production of vowels in monosyllabic stress feet","authors":"Carol A Fowler","year":"1983","venue":"Journal of Experimental Psychology: General","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Geng et al., 2016] Weidong Geng, Yu Du, Wenguang Jin, Wentao Wei, Yu Hu, and Jiajun Li. Gesture recognition by instantaneous surface EMG images. Scientific Reports, 6:36571, 2016.","title":"Gesture recognition by instantaneous surface EMG images","authors":"Weidong Geng, Yu Du, Wenguang Jin, Wentao Wei, Yu Hu, Jiajun Li","year":"2016","venue":"Scientific Reports","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Guo et al., 2016] Yuchen Guo, Guiguang Ding, Yue Gao, and Jianmin Wang. Semi-supervised active learning with cross-class sample transfer. In IJCAI, 2016.","title":"Semi-supervised active learning with cross-class sample transfer","authors":"Yuchen Guo, Guiguang Ding, Yue Gao, Jianmin Wang","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/219"},{"position":14,"raw":"[He et al., 2015] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: surpass- ing human-level performance on imagenet classification. In ICCV, pages 1026-1034, 2015.","title":"Delving deep into rectifiers: surpassing human-level performance on imagenet classification","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: accelerating deep network training by reducing internal covariate shift. In ICML, pages 448-456, 2015.","title":"Batch normalization: accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe, Sergey Ioffe, Christian Szegedy","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Joachims, 2002] Thorsten Joachims. Optimizing search en- gines using clickthrough data. In SIGKDD, 2002.","title":"Optimizing search engines using clickthrough data","authors":"Thorsten Joachims, Joachims","year":"2002","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Konrad, 2005] Peter Konrad. The ABC of EMG, 2005.","title":"The ABC of EMG","authors":"Peter Konrad","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet classification with deep convolutional neural networks. In NIPS, pages 1097- 1105. 2012.","title":"ImageNet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Leng et al., 2013] Yan Leng, Xinyan Xu, and Guanghui Qi. Combining active learning and semi-supervised learning to construct SVM classifier. Knowl.-Based Syst., 44:121- 131, 2013.","title":"Combining active learning and semi-supervised learning to construct SVM classifier","authors":"Yan Leng, Xinyan Xu, Guanghui Qi","year":"2013","venue":"Knowl.-Based Syst","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mobahi et al., 2009] Hossein Mobahi, Ronan Collobert, and Jason Weston. Deep learning from temporal coher- ence in video. In ICML, pages 737-744, 2009.","title":"Deep learning from temporal coherence in video","authors":"Hossein Mobahi, Ronan Collobert, Jason Weston","year":"2009","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Owens et al., 2016] Andrew Owens, Jiajun Wu, Josh H Mc- Dermott, William T Freeman, and Antonio Torralba. Am- bient sound provides supervision for visual learning. In LNCS, volume 9905, pages 801-816, 2016.","title":"Ambient sound provides supervision for visual learning","authors":"Andrew Owens, Jiajun Wu, Josh H Mc-Dermott, William T Freeman, Antonio Torralba","year":"2016","venue":"LNCS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pinto et al., 2016] Lerrel Pinto, Dhiraj Gandhi, Yuanfeng Han, Yong-Lae Park, and Abhinav Gupta. The curious robot: learning visual representations via physical interac- tions. In LNCS, volume 9906, pages 3-18, 2016.","title":"The curious robot: learning visual representations via physical interactions","authors":"Pinto","year":"2016","venue":"LNCS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rasmus et al., 2015] Antti Rasmus, Mathias Berglund, Mikko Honkala, Harri Valpola, and Tapani Raiko. Semi- supervised learning with ladder networks. In NIPS, pages 3546-3554, 2015.","title":"Semisupervised learning with ladder networks","authors":"Antti Rasmus, Mathias Berglund, Mikko Honkala, Harri Valpola, Tapani Raiko","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Saponas et al., 2008] T Scott Saponas, Desney S Tan, Dan Morris, and Ravin Balakrishnan. Demonstrating the fea- sibility of using forearm electromyography for muscle- computer interfaces. In SIGCHI, pages 515-524, 2008.","title":"Demonstrating the feasibility of using forearm electromyography for musclecomputer interfaces","authors":"Scott Saponas, Dan Desney S Tan, Ravin Morris, Balakrishnan","year":"2008","venue":"SIGCHI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Saric, 2011] Marin Saric. Libhand: a library for hand artic- ulation, 2011.","title":"Marin Saric. Libhand: a library for hand articulation","authors":"Saric","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Shahshahani and Landgrebe, 1994] Behzad M Shahshahani and David A Landgrebe. The effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon. IEEE Trans. Geosci. Remote Sens., 32(5):1087-1095, 1994.","title":"The effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon","authors":"M Behzad, David A Shahshahani, Landgrebe","year":"1994","venue":"IEEE Trans. Geosci. Remote Sens","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey E Hin- ton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. JMLR, 15(1):1929-1958, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Stewart and Ermon, 2017] Russell Stewart and Stefano Er- mon. Label-free supervision of neural networks with physics and domain knowledge. In IJCAI, 2017.","title":"Label-free supervision of neural networks with physics and domain knowledge","authors":"Russell Stewart, Stefano Ermon","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Taigman et al., 2014] Y Taigman, Ming Yang, M Ranzato, and L Wolf. DeepFace: closing the gap to human-level performance in face verification. In CVPR, 2014.","title":"DeepFace: closing the gap to human-level performance in face verification","authors":"Ming Taigman, M Yang, L Ranzato, Wolf","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang and Gupta, 2015] Xiaolong Wang and Abhinav Gupta. Unsupervised learning of visual representations using videos. In ICCV, pages 2794-2802, 2015.","title":"Unsupervised learning of visual representations using videos","authors":"Wang, Gupta, Xiaolong Wang, Abhinav Gupta","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2016] Xibin Wang, Junhao Wen, Shafiq Alam, Zhuo Jiang, and Yingbo Wu. Semi-supervised learning combining transductive support vector machine with ac- tive learning. Neurocomputing, 173:1288-1298, 2016.","title":"Semi-supervised learning combining transductive support vector machine with active learning","authors":"Wang","year":"2016","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}