{"type":"ScholarlyArticle","title":"Uncertainty Sampling for Action Recognition via Maximizing Expected Average Precision","authors":[{"name":"Hanmo Wang","slug":"hanmo-wang","given":"Hanmo","family":"Wang","affiliation":"State Key Lab. of Computer Science , Institute of Software , Chinese Academy of Sciences , China; University of Chinese Academy of Sciences , Beijing 100049 , China"},{"name":"Xiaojun Chang","slug":"xiaojun-chang","given":"Xiaojun","family":"Chang","affiliation":"School of Computer Science , Carnegie Mellon University , Pittsburgh , USA"},{"name":"Lei Shi","slug":"lei-shi","given":"Lei","family":"Shi","affiliation":"State Key Lab. of Computer Science , Institute of Software , Chinese Academy of Sciences , China; University of Chinese Academy of Sciences , Beijing 100049 , China"},{"name":"Yi Yang","slug":"yi-yang","given":"Yi","family":"Yang","affiliation":"Centre for Artificial Intelligence , University of Technology Sydney , Sydney , Australia"},{"name":"Yi-Dong Shen","slug":"yi-dong-shen","given":"Yi-Dong","family":"Shen","affiliation":"State Key Lab. of Computer Science , Institute of Software , Chinese Academy of Sciences , China"}],"abstract":"Recognizing human actions in video clips has been an important topic in computer vision. Sufficient labeled data is one of the prerequisites for the good performance of action recognition algorithms. However, while abundant videos can be collected from the Internet, categorizing each video clip is tedious and even time-consuming. Active learning is one way to alleviate the labeling labor by allowing the classifier to choose the most informative unlabeled instances for manual annotation. Among various active learning algorithms, uncertainty sampling is arguably the most widely-used strategy. Conventional uncertainty sampling strategies such as entropy-based methods are usually  tested under accuracy. However, in action recognition Average Precision (AP) is an acknowledged evaluation metric, which is somehow ignored in the active learning community. It is defined as the area under the precision-recall curve.  In this paper, we propose a novel uncertainty sampling algorithm for action recognition using expected AP. We conduct experiments on three real-world action recognition datasets and show that our algorithm outperforms other uncertainty-based active learning algorithms.","doi":"10.24963/ijcai.2018/134","url":"https://ijcai.org/proceedings/2018/134","pdf":"https://ijcai.org/proceedings/2018/0134.pdf","markdown":"https://ijcai.org/proceedings/2018/134.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/134","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":134,"pages":{"first":964,"last":970},"keywords":["Machine Learning: Classification","Machine Learning: Active Learning","Computer Vision: Action Recognition"],"videos":[],"references":[{"position":1,"raw":"[Behl et al., 2014] Aseem Behl, CV Jawahar, and M Pawan Kumar. Optimizing average precision us- ing weakly supervised data. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1011-1018, 2014.","title":"Optimizing average precision using weakly supervised data","authors":"Aseem Behl, M Pawan Jawahar, Kumar","year":"2014","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chakraborty et al., 2015] Shayok Chakraborty, Vineeth Balasubramanian, Qian Sun, Sethuraman Panchanathan, and Jieping Ye. Active batch selection via convex relaxations with guaranteed solution bounds. IEEE transactions on pattern analysis and machine intelligence, 37(10):1945-1958, 2015.","title":"Active batch selection via convex relaxations with guaranteed solution bounds","authors":"Shayok Chakraborty, Vineeth Balasubramanian, Qian Sun, Sethuraman Panchanathan, Jieping Ye","year":"2015","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Culver et al., 2006] Matt Culver, Deng Kun, and Stephen Scott. Active learning to maximize area under the roc curve. In Data Mining, 2006. ICDM'06. Sixth Interna- tional Conference on, pages 149-158. IEEE, 2006.","title":"Active learning to maximize area under the roc curve","authors":"Matt Culver, Deng Kun, Stephen Scott","year":"2006","venue":"ICDM'06. Sixth International Conference on","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dutt Jain and Grauman, 2016] Suyog Dutt Jain and Kristen Grauman. Active image segmentation propagation. In Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2864-2873, 2016.","title":"Active image segmentation propagation","authors":"Dutt Jain, Grauman ; Suyog Dutt Jain, Kristen Grauman","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Kuehne et al., 2011] Hildegard Kuehne, Hueihan Jhuang, Estíbaliz Garrote, Tomaso Poggio, and Thomas Serre. Hmdb: a large video database for human motion recogni- tion. In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 2556-2563. IEEE, 2011.","title":"Hmdb: a large video database for human motion recognition","authors":"Kuehne","year":"2011","venue":"Computer Vision (ICCV), 2011 IEEE International Conference on","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Lan et al., 2015] Zhengzhong Lan, Ming Lin, Xuanchong Li, Alex G Hauptmann, and Bhiksha Raj. Beyond gaussian pyramid: Multi-skip feature stacking for action recogni- tion. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 204-212, 2015.","title":"Beyond gaussian pyramid: Multi-skip feature stacking for action recognition","authors":"Lan","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Lewis and Gale, 1994] David D Lewis and William A Gale. A sequential algorithm for training text classifiers. In Pro- ceedings of the 17th annual international ACM SIGIR con- ference on Research and development in information re- trieval, pages 3-12. Springer-Verlag New York, Inc., 1994.","title":"A sequential algorithm for training text classifiers","authors":"D David, William A Lewis, Gale","year":"1994","venue":"Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Liu and Ferrari, 2017] Buyu Liu and Vittorio Ferrari. Ac- tive learning for human pose estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4363-4372, 2017.","title":"Active learning for human pose estimation","authors":"Ferrari Liu, Buyu Liu, Vittorio Ferrari","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Long et al., 2010] Bo Long, Olivier Chapelle, Ya Zhang, Yi Chang, Zhaohui Zheng, and Belle Tseng. Active learn- ing for ranking through expected loss optimization. In Proceedings of the 33rd international ACM SIGIR con- ference on Research and development in information re- trieval, pages 267-274. ACM, 2010.","title":"Active learning for ranking through expected loss optimization","authors":"Bo Long, Olivier Chapelle, Ya Zhang, Yi Chang, Zhaohui Zheng, Belle Tseng","year":"2010","venue":"Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Marszalek et al., 2009] Marcin Marszalek, Ivan Laptev, and Cordelia Schmid. Actions in context. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Confer- ence on, pages 2929-2936. IEEE, 2009.","title":"Actions in context","authors":"Marcin Marszalek, Ivan Laptev, Cordelia Schmid","year":"2009","venue":"Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Perronnin et al., 2010] Florent Perronnin, Jorge Sánchez, and Thomas Mensink. Improving the fisher kernel for large-scale image classification. Computer Vision-ECCV 2010, pages 143-156, 2010.","title":"Improving the fisher kernel for large-scale image classification","authors":"Florent Perronnin, Jorge Sánchez, Thomas Mensink","year":"2010","venue":"Computer Vision-ECCV 2010","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ramirez-Loaiza et al., 2017] Maria E Ramirez-Loaiza, Manali Sharma, Geet Kumar, and Mustafa Bilgic. Active learning: an empirical study of common baselines. Data Mining and Knowledge Discovery, 31(2):287-313, 2017.","title":"Active learning: an empirical study of common baselines","authors":"Maria E Ramirez-Loaiza, Manali Sharma, Geet Kumar, Mustafa Bilgic","year":"2017","venue":"Data Mining and Knowledge Discovery","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Reddy and Shah, 2013] Kishore K Reddy and Mubarak Shah. Recognizing 50 human action categories of web videos. Machine Vision and Applications, 24(5):971-981, 2013.","title":"Recognizing 50 human action categories of web videos","authors":"K Kishore, Mubarak Reddy, Shah","year":"2013","venue":"Machine Vision and Applications","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Roy and McCallum, 2001] Nicholas Roy and Andrew Mc- Callum. Toward optimal active learning through monte carlo estimation of error reduction. ICML, Williamstown, pages 441-448, 2001.","title":"Toward optimal active learning through monte carlo estimation of error reduction","authors":"Roy, Mccallum, Nicholas Roy, Andrew Mc-Callum","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Settles, 2010] Burr Settles. Active learning literature sur- vey. University of Wisconsin, Madison, 52(55-66):11, 2010.","title":"Burr Settles. Active learning literature survey","authors":"Settles","year":"2010","venue":"University of Wisconsin","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Seung et al., 1992] H Sebastian Seung, Manfred Opper, and Haim Sompolinsky. Query by committee. In Proceedings of the fifth annual workshop on Computational learning theory, pages 287-294. ACM, 1992.","title":"Query by committee","authors":"Sebastian Seung, Manfred Opper, Haim Sompolinsky","year":"1992","venue":"Proceedings of the fifth annual workshop on Computational learning theory","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sharma and Bilgic, 2017] Manali Sharma and Mustafa Bil- gic. Evidence-based uncertainty sampling for active learn- ing. Data Mining and Knowledge Discovery, 31(1):164- 202, 2017.","title":"Manali Sharma and Mustafa Bilgic. Evidence-based uncertainty sampling for active learning","authors":"Bilgic Sharma","year":"2017","venue":"Data Mining and Knowledge Discovery","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Triantafillou et al., 2017] Eleni Triantafillou, Richard Zemel, and Raquel Urtasun. Few-shot learning through an information retrieval lens. In Advances in Neural Information Processing Systems, pages 2252-2262, 2017.","title":"Few-shot learning through an information retrieval lens","authors":"Eleni Triantafillou, Richard Zemel, Raquel Urtasun","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang and Schmid, 2013] Heng Wang and Cordelia Schmid. Action recognition with improved trajectories. In Proceedings of the IEEE international conference on computer vision, pages 3551-3558, 2013.","title":"Action recognition with improved trajectories","authors":"Wang, Schmid ; ] Heng, Wang, Cordelia Schmid","year":"2013","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2011] Heng Wang, Alexander Kläser, Cordelia Schmid, and Cheng-Lin Liu. Action recognition by dense trajectories. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 3169-3176. IEEE, 2011.","title":"Action recognition by dense trajectories","authors":"Heng Wang, Alexander Kläser, Cordelia Schmid, Cheng-Lin Liu","year":"2011","venue":"Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yang et al., 2015] Yi Yang, Zhigang Ma, Feiping Nie, Xiao- jun Chang, and Alexander G Hauptmann. Multi-class ac- tive learning by uncertainty sampling with diversity max- imization. International Journal of Computer Vision, 113(2):113-127, 2015.","title":"Multi-class active learning by uncertainty sampling with diversity maximization","authors":"Yi Yang, Zhigang Ma, Feiping Nie, Xiaojun Chang, Alexander G Hauptmann","year":"2015","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yu et al., 2006] Kai Yu, Jinbo Bi, and Volker Tresp. Ac- tive learning via transductive experimental design. In Pro- ceedings of the 23rd international conference on Machine learning, pages 1081-1088. ACM, 2006.","title":"Active learning via transductive experimental design","authors":"Kai Yu, Jinbo Bi, Volker Tresp","year":"2006","venue":"Proceedings of the 23rd international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yue et al., 2007] Yisong Yue, Thomas Finley, Filip Radlin- ski, and Thorsten Joachims. A support vector method for optimizing average precision. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pages 271-278. ACM, 2007.","title":"A support vector method for optimizing average precision","authors":"Yue","year":"2007","venue":"Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhou et al., 2017] Zongwei Zhou, Jae Shin, Lei Zhang, Suryakanth Gurudu, Michael Gotway, and Jianming Liang. Fine-tuning convolutional neural networks for biomedical image analysis: actively and incrementally. In IEEE conference on computer vision and pattern recogni- tion, Hawaii, pages 7340-7349, 2017.","title":"Fine-tuning convolutional neural networks for biomedical image analysis: actively and incrementally","authors":"Zhou","year":"2017","venue":"IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}