{"type":"ScholarlyArticle","title":"SparseSense: Human Activity Recognition from Highly Sparse Sensor Data-streams Using Set-based Neural Networks","authors":[{"name":"Alireza Abedin","slug":"alireza-abedin","given":"Alireza","family":"Abedin","affiliation":"School of Computer Science , The University of Adelaide , Australia"},{"name":"S. Hamid Rezatofighi","slug":"s-hamid-rezatofighi","given":"S. Hamid","family":"Rezatofighi","affiliation":"School of Computer Science , The University of Adelaide , Australia"},{"name":"Qinfeng Shi","slug":"qinfeng-shi","given":"Qinfeng","family":"Shi","affiliation":"School of Computer Science , The University of Adelaide , Australia"},{"name":"Damith C. Ranasinghe","slug":"damith-c-ranasinghe","given":"Damith C.","family":"Ranasinghe","affiliation":"School of Computer Science , The University of Adelaide , Australia"}],"abstract":"Batteryless or so called passive wearables are providing new and innovative methods for human activity recognition (HAR), especially in healthcare applications for older people. Passive sensors are low cost, lightweight, unobtrusive and desirably disposable; attractive attributes for healthcare applications in hospitals and nursing homes. Despite the compelling propositions for sensing applications, the data streams from these sensors are characterised by high sparsity---the time intervals between sensor readings are irregular while the number of readings per unit time are often limited. In this paper, we rigorously explore the problem of learning activity recognition models from temporally sparse data. We describe how to learn directly from sparse data using a deep learning paradigm in an end-to-end manner. We demonstrate significant classification performance improvements on real-world passive sensor datasets from older people over the state-of-the-art deep learning human activity recognition models. Further, we provide insights into the model's behaviour through complementary experiments on a benchmark dataset and visualisation of the learned activity feature spaces.","doi":"10.24963/ijcai.2019/801","url":"https://ijcai.org/proceedings/2019/801","pdf":"https://ijcai.org/proceedings/2019/0801.pdf","markdown":"https://ijcai.org/proceedings/2019/801.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/801","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"AI for Improving Human Well-being","track":"AI for Improving Human Well-being","number":801,"pages":{"first":5780,"last":5786},"keywords":["Special Track on AI for Improving Human-Well Being: AI applications for Improving Human-Well Being (Special Track on AI and Human Wellbeing)","Special Track on AI for Improving Human-Well Being: Health applications (Special Track on AI and Human Wellbeing)"],"videos":[],"references":[{"position":1,"raw":"[Alsheikh et al., 2016] Mohammad Abu Alsheikh, Ahmed Selim, Dusit Niyato, Linda Doyle, Shaowei Lin, and Hwee-Pink Tan. Deep activity recognition models with tri- axial accelerometers. In Workshops at the Thirtieth AAAI Conference on Artificial Intelligence, 2016.","title":"Deep activity recognition models with triaxial accelerometers","authors":"Mohammad Abu, Alsheikh, Ahmed Selim, Dusit Niyato, Linda Doyle, Shaowei Lin, Hwee-Pink Tan","year":"2016","venue":"Workshops at the Thirtieth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bulling et al., 2014] Andreas Bulling, Ulf Blanke, and Bernt Schiele. A tutorial on human activity recognition using body-worn inertial sensors. ACM Computing Sur- veys, 46(3):33, 2014.","title":"A tutorial on human activity recognition using body-worn inertial sensors","authors":"Bulling","year":"2014","venue":"ACM Computing Surveys","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2015] Shengjian Jammy Chen, Christophe Fumeaux, Damith Chinthana Ranasinghe, and Thomas Kaufmann. Paired snap-on buttons connections for bal- anced antennas in wearable systems. IEEE Antennas and Wireless Propagation Letters, 14:1498-1501, 2015.","title":"Paired snap-on buttons connections for balanced antennas in wearable systems","authors":"Chen","year":"2015","venue":"IEEE Antennas and Wireless Propagation Letters","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gövercin et al., 2010] Mehmet Gövercin, Y Költzsch, M Meis, S Wegel, M Gietzelt, J Spehr, S Winkelbach, M Marschollek, and E Steinhagen-Thiessen. Defining the user requirements for wearable and optical fall prediction and fall detection devices for home use. Informatics for health and social care, 35(3-4):177-187, 2010.","title":"Defining the user requirements for wearable and optical fall prediction and fall detection devices for home use","authors":"Mehmet Gövercin, M Költzsch, S Meis, M Wegel, J Gietzelt, S Spehr, M Winkelbach, E Marschollek, Steinhagen-Thiessen","year":"2010","venue":"Informatics for health and social care","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gu et al., 2018] Fuqiang Gu, Kourosh Khoshelham, Shahrokh Valaee, Jianga Shang, and Rui Zhang. Locomo- tion activity recognition using stacked denoising autoen- coders. IEEE Internet of Things Journal, 5(3):2085-2093, 2018.","title":"Locomotion activity recognition using stacked denoising autoencoders","authors":"Fuqiang Gu, Kourosh Khoshelham, Shahrokh Valaee, Jianga Shang, Rui Zhang","year":"2018","venue":"IEEE Internet of Things Journal","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Guan and Plötz, 2017] Yu Guan and Thomas Plötz. En- sembles of deep lstm learners for activity recognition us- ing wearables. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1(2):11, 2017.","title":"Ensembles of deep lstm learners for activity recognition using wearables","authors":"Yu Guan, Thomas Plötz","year":"2017","venue":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hammerla et al., 2016] Nils Y. Hammerla, Shane Halloran, and Thomas Plötz. Deep, convolutional, and recurrent models for human activity recognition using wearables. In Proceedings of the Twenty-Fifth International Joint Con- ference on Artificial Intelligence, pages 1533-1540, 2016.","title":"Deep, convolutional, and recurrent models for human activity recognition using wearables","authors":"Nils Y Hammerla, Shane Halloran, Thomas Plötz","year":"2016","venue":"Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2016/220"},{"position":8,"raw":"[Kwapisz et al., 2011] Jennifer R Kwapisz, Gary M Weiss, and Samuel A Moore. Activity recognition using cell phone accelerometers. ACM SigKDD Explorations Newsletter, 12(2):74-82, 2011.","title":"Activity recognition using cell phone accelerometers","authors":"Gary M Jennifer R Kwapisz, Samuel A Weiss, Moore","year":"2011","venue":"ACM SigKDD Explorations Newsletter","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lemey et al., 2016] Sam Lemey, Sam Agneessens, Patrick Van Torre, Kristof Baes, Jan Vanfleteren, and Hendrik Ro- gier. Wearable flexible lightweight modular rfid tag with integrated energy harvester. IEEE Transactions on Mi- crowave Theory and Techniques, 64(7):2304-2314, 2016.","title":"Wearable flexible lightweight modular rfid tag with integrated energy harvester","authors":"Sam Lemey, Sam Agneessens, Patrick Van Torre, Kristof Baes, Jan Vanfleteren, Hendrik Rogier","year":"2016","venue":"IEEE Transactions on Microwave Theory and Techniques","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Maaten and Hinton, 2008] Laurens van der Maaten and Ge- offrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9:2579-2605, 2008.","title":"Laurens van der Maaten and Geoffrey Hinton","authors":"Hinton Maaten","year":"2008","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ordóñez and Roggen, 2016] Francisco Ordóñez and Daniel Roggen. Deep convolutional and lstm recurrent neural net- works for multimodal wearable activity recognition. Sen- sors, 16(1):115, 2016.","title":"Francisco Ordóñez and Daniel Roggen. Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition","authors":"Roggen Ordóñez","year":"2016","venue":"Sensors","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Paszke et al., 2017] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary De- Vito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In NIPS Autodiff Workshop, 2017.","title":"Automatic differentiation in pytorch","authors":"Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary De-Vito, Zeming Lin, Alban Desmaison, Luca Antiga, Adam Lerer","year":"2017","venue":"NIPS Autodiff Workshop","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Qi et al., 2017] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 652-660, 2017.","title":"Pointnet: Deep learning on point sets for 3d classification and segmentation","authors":"Hao Charles R Qi, Kaichun Su, Leonidas J Mo, Guibas","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Tieleman and Hinton, 2012] Tijmen Tieleman and Geoffrey Hinton. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning, 4(2):26-31, 2012.","title":"Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude","authors":"Hinton Tieleman, Tijmen Tieleman, Geoffrey Hinton","year":"2012","venue":"COURSERA: Neural Networks for Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Torres et al., 2013] Roberto L Shinmoto Torres, Damith C Ranasinghe, Qinfeng Shi, and Alanson P Sample. Sensor enabled wearable rfid technology for mitigating the risk of falls near beds. In IEEE International Conference on RFID, pages 191-198, 2013.","title":"Sensor enabled wearable rfid technology for mitigating the risk of falls near beds","authors":"Roberto L Shinmoto Torres, Qinfeng Damith C Ranasinghe, Alanson P Shi, Sample","year":"2013","venue":"IEEE International Conference on RFID","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Torres et al., 2017] Roberto L Shinmoto Torres, Renuka Visvanathan, Derek Abbott, Keith D Hill, and Damith C Ranasinghe. A battery-less and wireless wearable sensor system for identifying bed and chair exits in a pilot trial in hospitalized older people. PloS one, 12(10):1-25, 2017.","title":"A battery-less and wireless wearable sensor system for identifying bed and chair exits in a pilot trial in hospitalized older people","authors":"Roberto L Shinmoto Torres, Renuka Visvanathan, Derek Abbott, Keith D Hill, Damith C Ranasinghe","year":"2017","venue":"PloS one","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2019] Jindong Wang, Yiqiang Chen, Shuji Hao, Xiaohui Peng, and Lisha Hu. Deep learning for sensor-based activity recognition: A survey. Pattern Recognition Letters, 119:3 -11, 2019.","title":"Deep learning for sensor-based activity recognition: A survey","authors":"Jindong Wang, Yiqiang Chen, Shuji Hao, Xiaohui Peng, Lisha Hu","year":"2019","venue":"Pattern Recognition Letters","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wickramasinghe and Ranasinghe, 2015] Asanga Wickra- masinghe and Damith Ranasinghe. Recognising activities in real time using body worn passive sensors with sparse data streams: To interpolate or not to interpolate? In proceedings of the 12th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services, pages 21-30, 2015.","title":"Asanga Wickramasinghe and Damith Ranasinghe. Recognising activities in real time using body worn passive sensors with sparse data streams: To interpolate or not to interpolate","authors":"Ranasinghe Wickramasinghe","year":"2015","venue":"proceedings of the 12th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Yang et al., 2015] Jian Bo Yang, Minh Nhut Nguyen, Phyo Phyo San, Xiao Li Li, and Shonali Krishnaswamy. Deep convolutional neural networks on multichannel time series for human activity recognition. In Proceedings of the 24th International Conference on Artificial Intelli- gence, pages 3995-4001, 2015.","title":"Deep convolutional neural networks on multichannel time series for human activity recognition","authors":"Bo Jian, Minh Nhut Yang, Phyo Nguyen, Xiao Li Phyo San, Shonali Li, Krishnaswamy","year":"2015","venue":"Proceedings of the 24th International Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2015/561"},{"position":20,"raw":"[Zaheer et al., 2017] Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan R Salakhutdinov, and Alexander J Smola. Deep sets. In Advances in neural information processing systems, pages 3391-3401, 2017.","title":"Deep sets","authors":"Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan R Salakhutdinov, Alexander J Smola","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zeng et al., 2014] Ming Zeng, Le T Nguyen, Bo Yu, Ole J Mengshoel, Jiang Zhu, Pang Wu, and Joy Zhang. Con- volutional neural networks for human activity recognition using mobile sensors. In 6th International Conference on Mobile Computing, Applications and Services, pages 197- 205, 2014.","title":"Convolutional neural networks for human activity recognition using mobile sensors","authors":"Ming Zeng, Le T Nguyen, Bo Yu, Ole J Mengshoel, Jiang Zhu, Pang Wu, Joy Zhang","year":"2014","venue":"6th International Conference on Mobile Computing, Applications and Services","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}