{"type":"ScholarlyArticle","title":"A Novel Distribution-Embedded Neural Network for Sensor-Based Activity Recognition","authors":[{"name":"Hangwei Qian","slug":"hangwei-qian","given":"Hangwei","family":"Qian","affiliation":"School of Computer Science and Engineering; Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly; Interdisciplinary Graduate School Nanyang Technological University , Singapore"},{"name":"Sinno Jialin Pan","slug":"sinno-jialin-pan","given":"Sinno Jialin","family":"Pan","affiliation":"School of Computer Science and Engineering"},{"name":"Bingshui Da","slug":"bingshui-da","given":"Bingshui","family":"Da","affiliation":"School of Computer Science and Engineering"},{"name":"Chunyan Miao","slug":"chunyan-miao","given":"Chunyan","family":"Miao","affiliation":"School of Computer Science and Engineering; Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly"}],"abstract":"Feature-engineering-based machine learning models and deep learning models have been explored for wearable-sensor-based human activity recognition. For both types of methods, one crucial research issue is how to extract proper features from the partitioned segments of multivariate sensor readings. Existing methods have different drawbacks: 1) feature-engineering-based methods are able to extract meaningful features, such as statistical or structural information underlying the segments, but usually require manual designs of features for different applications, which is time consuming, and 2) deep learning models are able to learn temporal and/or spatial features from the sensor data automatically, but fail to capture statistical information. In this paper, we propose a novel deep learning model to automatically learn meaningful features including statistical features, temporal features and spatial correlation features for activity recognition in a unified framework. Extensive experiments are conducted on four datasets to demonstrate the effectiveness of our proposed method compared with state-of-the-art baselines.","doi":"10.24963/ijcai.2019/779","url":"https://ijcai.org/proceedings/2019/779","pdf":"https://ijcai.org/proceedings/2019/0779.pdf","markdown":"https://ijcai.org/proceedings/2019/779.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/779","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Planning and Scheduling","track":"Main track","number":779,"pages":{"first":5614,"last":5620},"keywords":["Planning and Scheduling: Activity and Plan Recognition","Machine Learning Applications: Other Applications"],"videos":[],"references":[{"position":1,"raw":"[Anguita et al., 2012] Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, and Jorge Luis Reyes-Ortiz. Human activity recognition on smartphones using a mul- ticlass hardware-friendly support vector machine. In IWAAL, pages 216-223. Springer, 2012.","title":"Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine","authors":"Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, Jorge Luis, Reyes-Ortiz","year":"2012","venue":"IWAAL","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bächlin et al., 2010] Marc Bächlin, Meir Plotnik, Daniel Roggen, Inbal Maidan, Jeffrey M. Hausdorff, Nir Gi- ladi, and Gerhard Tröster. Wearable assistant for parkin- son's disease patients with the freezing of gait symp- tom. IEEE Trans. Information Technology in Biomedicine, 14(2):436-446, 2010.","title":"Wearable assistant for parkinson's disease patients with the freezing of gait symptom","authors":"Marc Bächlin, Meir Plotnik, Daniel Roggen, Inbal Maidan, Jeffrey M Hausdorff, Nir Giladi, Gerhard Tröster","year":"2010","venue":"IEEE Trans. Information Technology in Biomedicine","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bulling et al., 2014] Andreas Bulling, Ulf Blanke, and Bernt Schiele. A tutorial on human activity recognition using body-worn inertial sensors. ACM Comput. Surv., 46(3):33:1-33:33, 2014.","title":"A tutorial on human activity recognition using body-worn inertial sensors","authors":"Bulling","year":"2014","venue":"ACM Comput. Surv","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chavarriaga et al., 2013] Ricardo Chavarriaga, Hesam Sagha, Alberto Calatroni, Sundara Tejaswi Digumarti, Gerhard Tröster, José del R. Millán, and Daniel Roggen. The opportunity challenge: A benchmark database for on-body sensor-based activity recognition. Pattern Recognition Letters, 34(15):2033-2042, 2013.","title":"The opportunity challenge: A benchmark database for on-body sensor-based activity recognition","authors":"Ricardo Chavarriaga, Hesam Sagha, Alberto Calatroni, Tejaswi Sundara, Gerhard Digumarti, José Tröster, Daniel Del R. Millán, Roggen","year":"2013","venue":"Pattern Recognition Letters","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2012] Liming Chen, Jesse Hoey, Chris D. Nu- gent, Diane J. Cook, and Zhiwen Yu. Sensor-based activity recognition. IEEE Trans. Systems, Man, and Cybernetics, Part C, 42(6):790-808, 2012.","title":"Sensor-based activity recognition","authors":"Liming Chen, Jesse Hoey, Chris D Nugent, Diane J Cook, Zhiwen Yu","year":"2012","venue":"IEEE Trans. Systems, Man, and Cybernetics, Part C","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hammerla et al., 2013] Nils Y. Hammerla, Reuben Kirkham, Peter Andras, and Thomas Ploetz. On preserv- ing statistical characteristics of accelerometry data using their empirical cumulative distribution. In ISWC, pages 65-68, 2013.","title":"On preserving statistical characteristics of accelerometry data using their empirical cumulative distribution","authors":"Nils Y Hammerla, Reuben Kirkham, Peter Andras, Thomas Ploetz","year":"2013","venue":"ISWC","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hammerla et al., 2016] Nils Y. Hammerla, Shane Halloran, and Thomas Plötz. Deep, convolutional, and recurrent models for human activity recognition using wearables. In IJCAI, pages 1533-1540. IJCAI/AAAI Press, 2016.","title":"Deep, convolutional, and recurrent models for human activity recognition using wearables","authors":"Nils Y Hammerla, Shane Halloran, Thomas Plötz","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/220"},{"position":8,"raw":"[Ignatov, 2018] Andrey Ignatov. Real-time human activity recognition from accelerometer data using convolutional neural networks. Appl. Soft Comput., 62:915-922, 2018.","title":"Real-time human activity recognition from accelerometer data using convolutional neural networks","authors":"Andrey Ignatov","year":"2018","venue":"Appl. Soft Comput","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Janidarmian et al., 2017] Majid Janidarmian, Atena Roshan Fekr, Katarzyna Radecka, and Zeljko Zilic. A comprehen- sive analysis on wearable acceleration sensors in human activity recognition. Sensors, 17(3):529, 2017.","title":"A comprehensive analysis on wearable acceleration sensors in human activity recognition","authors":"Majid Janidarmian, Katarzyna Atena Roshan Fekr, Zeljko Radecka, Zilic","year":"2017","venue":"Sensors","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lara and Labrador, 2013] Oscar D. Lara and Miguel A. Labrador. A survey on human activity recognition using wearable sensors. IEEE Communications Surveys and Tu- torials, 15(3):1192-1209, 2013.","title":"A survey on human activity recognition using wearable sensors","authors":"Oscar D Lara, Miguel A Labrador","year":"2013","venue":"IEEE Communications Surveys and Tutorials","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2015] Yujia Li, Kevin Swersky, and Richard S. Zemel. Generative moment matching networks. In ICML, volume 37 of JMLR Workshop and Conference Proceed- ings, pages 1718-1727. JMLR.org, 2015.","title":"Generative moment matching networks","authors":"Yujia Li, Kevin Swersky, Richard S Zemel","year":"2015","venue":"JMLR Workshop and Conference Proceedings","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2017] Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, and Barnabás Póczos. MMD GAN: towards deeper understanding of moment matching network. In NIPS, pages 2200-2210, 2017.","title":"MMD GAN: towards deeper understanding of moment matching network","authors":"Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, Barnabás Póczos","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin et al., 2007] Jessica Lin, Eamonn J. Keogh, Li Wei, and Stefano Lonardi. Experiencing SAX: a novel symbolic representation of time series. Data Min. Knowl. Discov., 15(2):107-144, 2007.","title":"Experiencing SAX: a novel symbolic representation of time series","authors":"Jessica Lin, Eamonn J Keogh, Li Wei, Stefano Lonardi","year":"2007","venue":"Data Min. Knowl. Discov","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Morales and Roggen, 2016] Francisco Javier Ordóñez Morales and Daniel Roggen. Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors, 16(1):115, 2016.","title":"Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition","authors":"Roggen Morales, Francisco Javier, Ordóñez Morales, Daniel Roggen","year":"2016","venue":"Sensors","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Muandet et al., 2017] Krikamol Muandet, Kenji Fukumizu, Bharath K. Sriperumbudur, and Bernhard Schölkopf. Ker- nel mean embedding of distributions: A review and be- yond. Foundations and Trends in Machine Learning, 10(1- 2):1-141, 2017.","title":"Kernel mean embedding of distributions: A review and beyond","authors":"Muandet","year":"2017","venue":"Foundations and Trends in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Plötz et al., 2011] Thomas Plötz, Nils Y. Hammerla, and Patrick Olivier. Feature learning for activity recognition in ubiquitous computing. In IJCAI, pages 1729-1734, 2011.","title":"Feature learning for activity recognition in ubiquitous computing","authors":"Thomas Plötz, Nils Y Hammerla, Patrick Olivier","year":"2011","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2011/290"},{"position":17,"raw":"[Qian et al., 2018] Hangwei Qian, Sinno Jialin Pan, and Chunyan Miao. Sensor-based activity recognition via learning from distributions. In AAAI, 2018.","title":"Sensor-based activity recognition via learning from distributions","authors":"Qian","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Qian et al., 2019] Hangwei Qian, Sinno Jialin Pan, and Chunyan Miao. Distribution-based semi-supervised learn- ing for activity recognition. In AAAI, 2019.","title":"Distribution-based semi-supervised learning for activity recognition","authors":"Qian","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rahimi and Recht, 2007] Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. In NIPS, pages 1177-1184, 2007.","title":"Random features for large-scale kernel machines","authors":"Recht Rahimi, Ali Rahimi, Benjamin Recht","year":"2007","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ravì et al., 2017] Daniele Ravì, Charence Wong, Benny Lo, and Guang-Zhong Yang. A deep learning approach to on- node sensor data analytics for mobile or wearable devices. IEEE J. Biomedical and Health Informatics, 21(1):56-64, 2017.","title":"A deep learning approach to onnode sensor data analytics for mobile or wearable devices","authors":"Daniele Ravì, Charence Wong, Benny Lo, Guang-Zhong Yang","year":"2017","venue":"IEEE J. Biomedical and Health Informatics","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Reiss and Stricker, 2012] Attila Reiss and Didier Stricker. Introducing a new benchmarked dataset for activity moni- toring. In ISWC, pages 108-109. IEEE Computer Society, 2012.","title":"Introducing a new benchmarked dataset for activity monitoring","authors":"Stricker Reiss, Attila Reiss, Didier Stricker","year":"2012","venue":"ISWC","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Smola et al., 2007] Alexander J. Smola, Arthur Gretton, Le Song, and Bernhard Schölkopf. A hilbert space em- bedding for distributions. In ALT, pages 13-31, 2007.","title":"A hilbert space embedding for distributions","authors":"Alexander J Smola, Arthur Gretton, Le Song, Bernhard Schölkopf","year":"2007","venue":"ALT","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sriperumbudur et al., 2009] Bharath K. Sriperumbudur, Kenji Fukumizu, Arthur Gretton, Gert R. G. Lanckriet, and Bernhard Schölkopf. Kernel choice and classifiability for RKHS embeddings of probability distributions. In NIPS, pages 1750-1758, 2009.","title":"Kernel choice and classifiability for RKHS embeddings of probability distributions","authors":"K Bharath, Kenji Sriperumbudur, Arthur Fukumizu, Gert R G Gretton, Bernhard Lanckriet, Schölkopf","year":"2009","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2017] Jindong Wang, Yiqiang Chen, Shuji Hao, Xiaohui Peng, and Lisha Hu. Deep learning for sensor-based activity recognition: A survey. CoRR, abs/1707.03502, 2017.","title":"Deep learning for sensor-based activity recognition: A survey","authors":"Jindong Wang, Yiqiang Chen, Shuji Hao, Xiaohui Peng, Lisha Hu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yang et al., 2015] Jianbo Yang, Minh Nhut Nguyen, Phyo Phyo San, Xiaoli Li, and Shonali Krishnaswamy. Deep convolutional neural networks on multichannel time series for human activity recognition. In IJCAI, pages 3995-4001. AAAI Press, 2015.","title":"Deep convolutional neural networks on multichannel time series for human activity recognition","authors":"Jianbo Yang, Minh Nhut Nguyen, Phyo Phyo San, Xiaoli Li, Shonali Krishnaswamy","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/561"},{"position":26,"raw":"[Zeng et al., 2014] Ming Zeng, Le T. Nguyen, Bo Yu, Ole J. Mengshoel, Jiang Zhu, Pang Wu, and Joy Zhang. Convo- lutional neural networks for human activity recognition us- ing mobile sensors. In MobiCASE, pages 197-205. IEEE, 2014.","title":"Convolutional neural networks for human activity recognition using mobile sensors","authors":"Ming Zeng, Le T Nguyen, Bo Yu, Ole J Mengshoel, Jiang Zhu, Pang Wu, Joy Zhang","year":"2014","venue":"MobiCASE","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}