{"type":"ScholarlyArticle","title":"Cost-Effective Active Learning for Hierarchical Multi-Label Classification","authors":[{"name":"Yi-Fan Yan","slug":"yi-fan-yan","given":"Yi-Fan","family":"Yan","affiliation":"College of Computer Science & Technology , Nanjing University of Aeronautics & Astronautics; Collaborative Innovation Center of Novel Software Technology and Industrialization"},{"name":"Sheng-Jun Huang","slug":"sheng-jun-huang","given":"Sheng-Jun","family":"Huang","affiliation":"College of Computer Science & Technology , Nanjing University of Aeronautics & Astronautics"}],"abstract":"Active learning reduces the labeling cost by actively querying labels for the most valuable data. It is particularly important for multi-label learning, where the annotation cost is rather high because each instance may have multiple labels simultaneously. In many multi-label tasks, the labels are organized into hierarchies from coarse to fine. The labels at different levels of the hierarchy contribute differently to the model training, and also have diverse annotation costs. In this paper, we propose a multi-label active learning approach to exploit the label hierarchies for cost-effective queries. By incorporating the potential contribution of ancestor and descendant labels, a novel criterion is proposed to estimate the informativeness of each candidate query. Further, a subset selection method is introduced to perform active batch selection by balancing the informativeness and cost of each instance-label pair. Experimental results validate the effectiveness of both the proposed criterion and the selection method.","doi":"10.24963/ijcai.2018/411","url":"https://ijcai.org/proceedings/2018/411","pdf":"https://ijcai.org/proceedings/2018/0411.pdf","markdown":"https://ijcai.org/proceedings/2018/411.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/411","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":411,"pages":{"first":2962,"last":2968},"keywords":["Machine Learning: Active Learning","Machine Learning: Semi-Supervised Learning","Machine Learning: Machine Learning","Machine Learning: Multi-instance;Multi-label;Multi-view learning"],"videos":[],"references":[{"position":1,"raw":"[Barutcuoglu et al., 2006] Zafer Barutcuoglu, Robert E Schapire, and Olga G Troyanskaya. Hierarchical multi- label prediction of gene function. Bioinformatics, 22(7):830-836, 2006.","title":"Hierarchical multilabel prediction of gene function","authors":"Zafer Barutcuoglu, Robert E Schapire, Olga G Troyanskaya","year":"2006","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Blockeel et al., 2002] Hendrik Blockeel, Maurice Bruynooghe, Sašo Džeroski, Jan Ramon, and Jan Struyf. Hierarchical multi-classification. In Workshop Notes of the KDD'02 Workshop on Multi-Relational Data Mining, pages 21-35, 2002.","title":"Hierarchical multi-classification","authors":"Hendrik Blockeel, Maurice Bruynooghe, Sašo Džeroski, Jan Ramon, Jan Struyf","year":"2002","venue":"Workshop Notes of the KDD'02 Workshop on Multi-Relational Data Mining","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Brinker, 2003] Klaus Brinker. Incorporating diversity in ac- tive learning with support vector machines. In Proceedings of the 20th International Conference on Machine Learn- ing, pages 59-66, 2003.","title":"Incorporating diversity in active learning with support vector machines","authors":"Klaus Brinker, Brinker","year":"2003","venue":"Proceedings of the 20th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Burred and Lerch, 2003] Juan José Burred and Alexander Lerch. A hierarchical approach to automatic musical genre classification. In Proceedings of the 6th International Con- ference on Digital Audio Effects, pages 8-11, 2003.","title":"Juan José Burred and Alexander Lerch. A hierarchical approach to automatic musical genre classification","authors":"Lerch Burred","year":"2003","venue":"Proceedings of the 6th International Conference on Digital Audio Effects","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chakraborty et al., 2015] Shayok Chakraborty, Vineeth Balasubramanian, Adepu Ravi Sankar, Sethuraman Panchanathan, and Jieping Ye. Batchrank: A novel batch mode active learning framework for hierarchical classification. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 99-108, 2015.","title":"Batchrank: A novel batch mode active learning framework for hierarchical classification","authors":"Shayok Chakraborty, Vineeth Balasubramanian, Adepu Ravi Sankar, Sethuraman Panchanathan, Jieping Ye","year":"2015","venue":"Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chang and Lin, 2011] Chih-Chung Chang and Chih-Jen Lin. Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3):27, 2011.","title":"Libsvm: a library for support vector machines","authors":"Chang, Lin ; ] Chih-Chung Chang, Chih-Jen Lin","year":"2011","venue":"ACM Transactions on Intelligent Systems and Technology","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cheng et al., 2012] Yu Cheng, Kunpeng Zhang, Yusheng Xie, Ankit Agrawal, and Alok Choudhary. On active learn- ing in hierarchical classification. In Proceedings of the 21st ACM International Conference on Information and knowledge Management, pages 2467-2470, 2012.","title":"On active learning in hierarchical classification","authors":"Yu Cheng, Kunpeng Zhang, Yusheng Xie, Ankit Agrawal, Alok Choudhary","year":"2012","venue":"Proceedings of the 21st ACM International Conference on Information and knowledge Management","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cheng et al., 2014] Yu Cheng, Zhengzhang Chen, Hongliang Fei, Fei Wang, and Alok Choudhary. Batch mode active learning with hierarchical-structured embed- ded variance. In Proceedings of the SIAM International Conference on Data Mining, pages 10-18, 2014.","title":"Batch mode active learning with hierarchical-structured embedded variance","authors":"Yu Cheng, Zhengzhang Chen, Hongliang Fei, Fei Wang, Alok Choudhary","year":"2014","venue":"Proceedings of the SIAM International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Clare, 2003] Amanda Clare. Machine learning and data mining for yeast functional genomics. Aberystwyth: The University of Wales.(Doctor of Philosophy), 2003.","title":"Machine learning and data mining for yeast functional genomics","authors":"Amanda Clare","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Eisner et al., 2005] Roman Eisner, Brett Poulin, Duane Szafron, Paul Lu, and Russell Greiner. Improving protein function prediction using the hierarchical structure of the gene ontology. In Proceedings of the IEEE Symposium on Computational Intelligence in Bioinformatics and Compu- tational Biology, pages 1-10, 2005.","title":"Improving protein function prediction using the hierarchical structure of the gene ontology","authors":"Eisner","year":"2005","venue":"Proceedings of the IEEE Symposium on Computational Intelligence in Bioinformatics and Computational Biology","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Freitas and de Carvalho, 2007] Alex A Freitas and An- dre CPFL de Carvalho. A tutorial on hierarchical clas- sification with applications in bioinformatics. In D. Taniar (Ed.) Research and Trends in Data Mining Technologies and Applications, Idea Group, 2007.","title":"Andre CPFL de Carvalho. A tutorial on hierarchical classification with applications in bioinformatics","authors":"Alex A Freitas","year":"2007","venue":"Research and Trends in Data Mining Technologies and Applications","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huang and Zhou, 2013] Sheng-Jun Huang and Zhi-Hua Zhou. Active query driven by uncertainty and diversity for incremental multi-label learning. In Proceedings of the 13th IEEE International Conference on Data Mining, pages 1079-1084, 2013.","title":"Active query driven by uncertainty and diversity for incremental multi-label learning","authors":"Zhou ; Sheng-Jun Huang, Zhi-Hua Huang, Zhou","year":"2013","venue":"Proceedings of the 13th IEEE International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Huang et al., 2014a] Sheng-Jun Huang, Wei Gao, and Zhi- Hua Zhou. Fast multi-instance multi-label learning. In Proceedings of the Twenty-Eighth AAAI Conference on Ar- tificial Intelligence, pages 1868-1874, 2014.","title":"Fast multi-instance multi-label learning","authors":"Huang","year":"2014","venue":"Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Huang et al., 2014b] Sheng-Jun Huang, Rong Jin, and Zhi- Hua Zhou. Active learning by querying informative and representative examples. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(10):1936-1949, 2014.","title":"Active learning by querying informative and representative examples","authors":"Huang","year":"2014","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Huang et al., 2015] Sheng-Jun Huang, Songcan Chen, and Zhi-Hua Zhou. Multi-label active learning: Query type matters. In Proceedings of the 24th International Joint Conference on Artificial Intelligence, pages 946-952, 2015.","title":"Multi-label active learning: Query type matters","authors":"Sheng-Jun Huang, Songcan Chen, Zhi-Hua Zhou","year":"2015","venue":"Proceedings of the 24th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2015/138"},{"position":16,"raw":"[Hung and Lin, 2011] Chen Wei Hung and Hsuan Tien Lin. Multi-label active learning with auxiliary learner. In Asian Conference on Machine Learning, pages 315-332, 2011.","title":"Multi-label active learning with auxiliary learner","authors":"Chen Wei, Hung, Hsuan Tien, Lin","year":"2011","venue":"Asian Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Koller and Sahami, 1997] Daphne Koller and Mehran Sa- hami. Hierarchically classifying documents using very few words. In Proceedings of the 14th International Confer- ence on Machine Learning, pages 170-178, 1997.","title":"Hierarchically classifying documents using very few words","authors":"Sahami Koller","year":"1997","venue":"Proceedings of the 14th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2004] Xuchun Li, Lei Wang, and Eric Sung. Mul- tilabel svm active learning for image classification. In In- ternational Conference on Image Processing, pages 2207- 2210, 2004.","title":"Multilabel svm active learning for image classification","authors":"Li","year":"2004","venue":"International Conference on Image Processing","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2012] Xiao Li, Da Kuang, and Charles X Ling. Active learning for hierarchical text classification. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 14-25, 2012.","title":"Active learning for hierarchical text classification","authors":"Xiao Li, Da Kuang, Charles X Ling","year":"2012","venue":"Pacific-Asia Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Li et al., 2013] Xiao Li, Charles X Ling, and Huaimin Wang. Effective top-down active learning for hierarchical text classification. In Pacific-Asia Conference on Knowl- edge Discovery and Data Mining, pages 233-244, 2013.","title":"Effective top-down active learning for hierarchical text classification","authors":"Xiao Li, Charles X Ling, Huaimin Wang","year":"2013","venue":"Pacific-Asia Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Nguyen and Smeulders, 2004] Hieu T Nguyen and Arnold Smeulders. Active learning using pre-clustering. In Pro- ceedings of the 21st International Conference on Machine Learning, page 79, 2004.","title":"Active learning using pre-clustering","authors":"T Hieu, Arnold Nguyen, Smeulders","year":"2004","venue":"Proceedings of the 21st International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Qian et al., 2015] Chao Qian, Yang Yu, and Zhi-Hua Zhou. Subset selection by pareto optimization. In Advances in Neural Information Processing Systems, pages 1774- 1782, 2015.","title":"Subset selection by pareto optimization","authors":"Yang Chao Qian, Zhi-Hua Yu, Zhou","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Secker et al., 2007] Andrew D Secker, Matthew N Davies, Alex A Freitas, Jon Timmis, Miguel Mendao, and Dar- ren R Flower. An experimental comparison of classi- fication algorithms for hierarchical prediction of protein function. Prediction of Protein Function Expert Update, 9(3):17-22, 2007.","title":"An experimental comparison of classification algorithms for hierarchical prediction of protein function","authors":"Matthew N Andrew D Secker, Alex A Davies, Jon Freitas, Miguel Timmis, Darren R Mendao, Flower","year":"2007","venue":"Prediction of Protein Function Expert Update","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Tong and Koller, 2001] Simon Tong and Daphne Koller. Support vector machine active learning with applications to text classification. Journal of Machine Learning Re- search, 2(Nov):45-66, 2001.","title":"Support vector machine active learning with applications to text classification","authors":"Koller Tong, Simon Tong, Daphne Koller","year":"2001","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}