{"type":"ScholarlyArticle","title":"Mindful Active Learning","authors":[{"name":"Zhila Esna Ashari","slug":"zhila-esna-ashari","given":"Zhila","family":"Esna Ashari","affiliation":"School of Electrical Engineering & Computer Science , Washington State University , Pullman , WA"},{"name":"Hassan Ghasemzadeh","slug":"hassan-ghasemzadeh","given":"Hassan","family":"Ghasemzadeh","affiliation":"School of Electrical Engineering & Computer Science , Washington State University , Pullman , WA"}],"abstract":"We propose a novel active learning framework for activity recognition using wearable sensors. Our work is unique in that it takes physical and cognitive limitations of the oracle into account when selecting sensor data to be annotated by the oracle. Our approach is inspired by human-beings' limited capacity to respond to external stimulus such as responding to a prompt on their mobile devices. This capacity constraint is manifested not only in the number of queries that a person can respond to in a given time-frame but also in the lag between the time that a query is made and when it is responded to. We introduce the notion of mindful active learning and propose a computational framework, called EMMA, to maximize the active learning performance taking informativeness of sensor data, query budget, and human memory into account. We formulate this optimization problem, propose an approach to model memory retention, discuss complexity of the problem, and propose a greedy heuristic to solve the problem. We demonstrate the effectiveness of our approach on three publicly available datasets and by simulating oracles with various memory strengths. We show that the activity recognition accuracy ranges from 21% to 97% depending on memory strength, query budget, and difficulty of the machine learning task. Our results also indicate that EMMA achieves an accuracy level that is, on average, 13.5% higher than the case when only informativeness of the sensor data is considered for active learning. Additionally, we show that the performance of our approach is at most 20% less than experimental upper-bound and up to 80% higher than experimental lower-bound. We observe that mindful active learning is most beneficial when query budget is small and/or oracle's memory is weak, thus emphasizing contributions of our work in human-centered mobile health settings and for elderly with cognitive impairments.","doi":"10.24963/ijcai.2019/314","url":"https://ijcai.org/proceedings/2019/314","pdf":"https://ijcai.org/proceedings/2019/0314.pdf","markdown":"https://ijcai.org/proceedings/2019/314.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/314","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":314,"pages":{"first":2265,"last":2271},"keywords":["Machine Learning: Active Learning","Multidisciplinary Topics and Applications: Ubiquitous Computing Systems","Machine Learning: Classification"],"videos":[],"references":[{"position":1,"raw":"[Altun and Barshan, 2010] Kerem Altun and Billur Barshan. Human activity recognition using inertial/magnetic sensor units. In First International Workshop on Human Behavior Understanding, Istanbul, Turkey, August 2010.","title":"Human activity recognition using inertial/magnetic sensor units","authors":"Altun, Barshan, Kerem Altun, Billur Barshan","year":"2010","venue":"First International Workshop on Human Behavior Understanding","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Altun et al., 2010] Kerem Altun, Billur Barshan, and Orkun Tunc ¸el. Comparative study on classifying human activi- ties with miniature inertial and magnetic sensors. Pattern Recognition, 43(10):3605-3620, October 2010.","title":"Comparative study on classifying human activities with miniature inertial and magnetic sensors","authors":"Altun","year":"2010","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Anguita et al., 2013] Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, and Jorge L. Reyes-Ortiz. A public domain dataset for human activity recognition us- ing smartphones. In 21th European Symposium on Ar- tificial Neural Networks, Computational Intelligence and Machine Learning, ESANN, Bruges, Belgium, April 2013.","title":"A public domain dataset for human activity recognition using smartphones","authors":"Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, Jorge L Reyes-Ortiz","year":"2013","venue":"21th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, ESANN","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bachman et al., 2017] Philip Bachman, Alessandro Sor- doni, and Adam Trischler. Learning algorithms for active learning. In Proceedings of the 34th International Confer- ence on Machine Learning, Sydney, Australia, 2017.","title":"Learning algorithms for active learning","authors":"Philip Bachman, Alessandro Sordoni, Adam Trischler","year":"2017","venue":"Proceedings of the 34th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bao and Intille, 2004] Ling Bao and Stephen S. Intille. Ac- tivity recognition from user-annotated acceleration data. In International Conference on Pervasive Computing, Linz/Vienna, Austria, April 2004.","title":"Activity recognition from user-annotated acceleration data","authors":"Ling Bao, Stephen S Intille","year":"2004","venue":"International Conference on Pervasive Computing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Barshan and Yüksek, 2014] Billur Barshan and Murat Ci- han Yüksek. Recognizing daily and sports activities in two open source machine learning environments using body- worn sensor units. The Computer Journal, 57(11), 2014.","title":"Billur Barshan and Murat Cihan Yüksek. Recognizing daily and sports activities in two open source machine learning environments using bodyworn sensor units","authors":"Yüksek Barshan","year":"2014","venue":"The Computer Journal","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Diethe et al., 2016] Tom Diethe, Niall Twomey, and Peter Flach. Active transfer learning for activity recognition. In Proceedings of proceedings, European Symposium on Ar- tificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, Belgium, April 2016.","title":"Active transfer learning for activity recognition","authors":"Diethe","year":"2016","venue":"Proceedings of proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ebbinghaus, 1885] Hermann Ebbinghaus. Memory: A con- tribution to experimental psychology. Annals of Neuro- sciences, 1885.","title":"Memory: A contribution to experimental psychology","authors":"Hermann Ebbinghaus, Ebbinghaus","year":"1885","venue":"Annals of Neurosciences","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fallahzadeh and Ghasemzadeh, 2017] R. Fallahzadeh and H. Ghasemzadeh. Personalization without user interrup- tion: Boosting activity recognition in new subjects using unlabeled data. In 2017 ACM/IEEE 8th International Con- ference on Cyber-Physical Systems (ICCPS), April 2017.","title":"Personalization without user interruption: Boosting activity recognition in new subjects using unlabeled data","authors":"R Fallahzadeh, H Ghasemzadeh","year":"2017","venue":"ACM/IEEE 8th International Conference on Cyber-Physical Systems (ICCPS)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hoi et al., 2006] Steven C. H. Hoi, Rong Jin, Jianke Zhu, and Michael R. Lyu. Batch mode active learning and its application to medical image classification. In 23rd In- ternational Conference on Machine Learning, Pittsburgh, PA, 2006.","title":"Batch mode active learning and its application to medical image classification","authors":"C H Steven, Rong Hoi, Jianke Jin, Michael R Zhu, Lyu","year":"2006","venue":"23rd International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Huang et al., 2014] Sheng-Jun Huang, Rong Jin, and Zhi- Hua Zhou. Active learning by querying informative and representative examples. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(10), October 2014.","title":"Active learning by querying informative and representative examples","authors":"Sheng-Jun Huang, Rong Jin, Zhi-Hua Zhou","year":"2014","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jun and Nowak, 2016] Kwang-Sung Jun and Robert Nowak. Graph-based active learning: A new look at expected error minimization. In IEEE Global Conference on Signal and Information Processing (GlobalSIP), Washington, DC, USA, December 2016.","title":"Graph-based active learning: A new look at expected error minimization","authors":"Kwang-Sung Jun, Robert Nowak","year":"2016","venue":"IEEE Global Conference on Signal and Information Processing (GlobalSIP)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ma et al., 2019] Yuchao Ma, Zhila Esna Ashari, Mahdi Pe- dram, Navid Amini, Daniel Tarquinio, Kouros Nouri- Mahdavi, Mohammad Pourhomayoun, Robert D Catena, and Hassan Ghasemzadeh. Cyclepro: A robust framework for domain-agnostic gait cycle detection. IEEE Sensors Journal, 19(10):3751-3762, 2019.","title":"Cyclepro: A robust framework for domain-agnostic gait cycle detection","authors":"Ma","year":"2019","venue":"IEEE Sensors Journal","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Murukannaiah and Singh, 2015] Pradeep K. Murukannaiah and Munindar P. Singh. Platys: An active learning frame- work for place-aware application development and its evaluation. ACM Transactions on Software Engineering and Methodology, 24(3), May 2015.","title":"Platys: An active learning framework for place-aware application development and its evaluation","authors":"K Pradeep, Munindar P Murukannaiah, Singh","year":"2015","venue":"ACM Transactions on Software Engineering and Methodology","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Palumboa et al., 2016] Filippo Palumboa, Claudio Gallic- chiob, Rita Pucciband, and Alessio Micheli. Human ac- tivity recognition using multisensor data fusion based on reservoir computing. Journal of Ambient Intelligence and Smart Environments, 8(2):87-107, October 2016.","title":"Human activity recognition using multisensor data fusion based on reservoir computing","authors":"Filippo Palumboa, Claudio Gallicchiob, Rita Pucciband, Alessio Micheli","year":"2016","venue":"Journal of Ambient Intelligence and Smart Environments","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pisinger, 2005] David Pisinger. Where are the hard knap- sack problems? Computers & Operations Research, 32(9), 2005.","title":"Where are the hard knapsack problems","authors":"David Pisinger","year":"2005","venue":"Computers & Operations Research","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Redmond et al., 2014] SJ Redmond, NH Lovell, GZ Yang, A Horsch, P Lukowicz, L Murrugarra, and M Marschollek. What does big data mean for wear- able sensor systems? Yearbook of medical informatics, 23(01):135-142, 2014.","title":"What does big data mean for wearable sensor systems?","authors":"Redmond","year":"2014","venue":"Yearbook of medical informatics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sabato and Hess, 2016] Sivan Sabato and Tom Hess. Inter- active algorithms: from pool to stream. In Proceedings of the 29th Annual Conference on Learning Theory (COLT), JMLR Workshop and Conference Proceedings, New-York City, USA, June 2016.","title":"Interactive algorithms: from pool to stream","authors":"Hess Sabato, Sivan Sabato, Tom Hess","year":"2016","venue":"Proceedings of the 29th Annual Conference on Learning Theory (COLT), JMLR Workshop and Conference Proceedings","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sener and Savarese, 2018] Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core- set approach. In ICLR 2018, Vancouver, Canada, May 2018.","title":"Active learning for convolutional neural networks: A coreset approach","authors":"Ozan Sener, Silvio Savarese","year":"2018","venue":"ICLR 2018","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Settles, 2012] Burr Settles. Active learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 6(1), 2012.","title":"Burr Settles. Active learning","authors":"Settles","year":"2012","venue":"Synthesis Lectures on Artificial Intelligence and Machine Learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shen and Zhai, 2005] Xuehua Shen and ChengXiang Zhai. Active feedback in ad hoc information retrieval. In 28th Annual International ACM SIGIR Conference on Re- search and Development in Information Retrieval, Sal- vador, Brazil, August 2005.","title":"Xuehua Shen and ChengXiang Zhai. Active feedback in ad hoc information retrieval","authors":"Zhai Shen","year":"2005","venue":"28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Stikic and Schiele, 2009] Maja Stikic and Bernt Schiele. Activity recognition from sparsely labeled data using multi-instance learning. In Proceedings of 4th Interna- tional Symposium on Location-and Context-Awareness, Tokyo, Japan, May 2009.","title":"Activity recognition from sparsely labeled data using multi-instance learning","authors":"Schiele Stikic, Maja Stikic, Bernt Schiele","year":"2009","venue":"Proceedings of 4th International Symposium on Location-and Context-Awareness","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vijayanarasimhan et al., 2016] Sudheendra Vijaya- narasimhan, Prateek Jain, and Kristen Grauman. Far-sighted active learning on a budget for image and video recognition. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, February 2016.","title":"Far-sighted active learning on a budget for image and video recognition","authors":"Sudheendra Vijayanarasimhan, Prateek Jain, Kristen Grauman","year":"2016","venue":"Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xu et al., 2007] Zuobing Xu, Ram Akella, and Yi Zhang. Incorporating diversity and density in active learning for relevance feedback. In Proceedings of 29th European Conference on IR Research, Rome, Italy, April 2007.","title":"Incorporating diversity and density in active learning for relevance feedback","authors":"Xu","year":"2007","venue":"Proceedings of 29th European Conference on IR Research","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}