{"type":"ScholarlyArticle","title":"Privileged Multi-label Learning","authors":[{"name":"Shan You","slug":"shan-you","given":"Shan","family":"You","affiliation":"Key Lab. of Machine Perception (MOE) , School of EECS , Peking University , P. R. China; Cooperative Medianet Innovation Center , Peking University , P. R. China"},{"name":"Chang Xu","slug":"chang-xu","given":"Chang","family":"Xu","affiliation":"UBTech Sydney AI Institute , School of IT , FEIT , The University of Sydney , Australia"},{"name":"Yunhe Wang","slug":"yunhe-wang","given":"Yunhe","family":"Wang","affiliation":"Key Lab. of Machine Perception (MOE) , School of EECS , Peking University , P. R. China; Cooperative Medianet Innovation Center , Peking University , P. R. China"},{"name":"Chao Xu","slug":"chao-xu","given":"Chao","family":"Xu","affiliation":"Key Lab. of Machine Perception (MOE) , School of EECS , Peking University , P. R. China; Cooperative Medianet Innovation Center , Peking University , P. R. China"},{"name":"Dacheng Tao","slug":"dacheng-tao","given":"Dacheng","family":"Tao","affiliation":"UBTech Sydney AI Institute , School of IT , FEIT , The University of Sydney , Australia"}],"abstract":"This paper presents privileged multi-label learning (PrML) to explore and exploit the relationship between labels in multi-label learning problems. We suggest that for each individual label, it cannot only be implicitly connected with other labels via the low-rank constraint over label predictors, but also its performance on examples can receive the explicit comments from other labels together acting as an Oracle teacher. We generate privileged label feature for each example and its individual label, and then integrate it into the framework of low-rank based multi-label learning. The proposed algorithm can therefore comprehensively explore and exploit label relationships by inheriting all the merits of privileged information and low-rank constraints. We show that PrML can be efficiently solved by dual coordinate descent algorithm using iterative optimization strategy with cheap updates. Experiments on benchmark datasets show that through privileged label features, the performance can be significantly improved and PrML is superior to several competing methods in most cases.","doi":"10.24963/ijcai.2017/466","url":"https://ijcai.org/proceedings/2017/466","pdf":"https://ijcai.org/proceedings/2017/0466.pdf","markdown":"https://ijcai.org/proceedings/2017/466.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/466","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":466,"pages":{"first":3336,"last":3342},"keywords":["Machine Learning: Classification","Machine Learning: Multi-instance/Multi-label/Multi-view learning","Machine Learning: Transfer, Adaptation, Multi-task Learning"],"videos":[],"references":[{"position":1,"raw":"[Bhatia et al., 2015] Kush Bhatia, Himanshu Jain, Purushot- tam Kar, Manik Varma, and Prateek Jain. Sparse local embeddings for extreme multi-label classification. In Ad- vances in Neural Information Processing Systems, pages 730-738, 2015.","title":"Sparse local embeddings for extreme multi-label classification","authors":"Bhatia","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chiang et al., 2016] Wei-Lin Chiang, Mu-Chu Lee, and Chih-Jen Lin. Parallel dual coordinate descent method for large-scale linear classification in multi-core environments. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2016.","title":"Parallel dual coordinate descent method for large-scale linear classification in multi-core environments","authors":"Wei-Lin Chiang, Mu-Chu Lee, Chih-Jen Lin","year":"2016","venue":"Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Hou et al., 2016] Peng Hou, Xin Geng, and Min-Ling Zhang. Multi-label manifold learningn. In Thirtieth AAAI Confer- ence on Artificial Intelligence, 2016.","title":"Multi-label manifold learningn","authors":"Peng Hou, Xin Geng, Min-Ling Zhang","year":"2016","venue":"Thirtieth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Hsieh et al., 2008] Cho-Jui Hsieh, Kai-Wei Chang, Chih-Jen Lin, S Sathiya Keerthi, and Sellamanickam Sundararajan. A dual coordinate descent method for large-scale linear svm. In Proceedings of the 25th international conference on Machine learning, pages 408-415. ACM, 2008.","title":"A dual coordinate descent method for large-scale linear svm","authors":"Cho-Jui Hsieh, Kai-Wei Chang, Chih-Jen Lin, S Sathiya Keerthi, Sellamanickam Sundararajan","year":"2008","venue":"Proceedings of the 25th international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Li et al., 2015] Ximing Li, Jihong Ouyang, and Xiaotang Zhou. Supervised topic models for multi-label classifica- tion. Neurocomputing, 149:811-819, 2015.","title":"Supervised topic models for multi-label classification","authors":"Ximing Li, Jihong Ouyang, Xiaotang Zhou","year":"2015","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Li et al., 2016] Wen Li, Dengxin Dai, Mingkui Tan, Dong Xu, and Luc Van Gool. Fast algorithms for linear and kernel svm+. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2258-2266, 2016.","title":"Fast algorithms for linear and kernel svm+","authors":"Wen Li, Dengxin Dai, Mingkui Tan, Dong Xu, Luc Van Gool","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Liu et al., 2017] Tongliang Liu, Dacheng Tao, Mingli Song, and Stephen J Maybank. Algorithm-dependent generaliza- tion bounds for multi-task learning. IEEE transactions on pattern analysis and machine intelligence, 39(2):227-241, 2017.","title":"Algorithm-dependent generalization bounds for multi-task learning","authors":"Liu","year":"2017","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Pechyony and Vapnik, 2010] Dmitry Pechyony and Vladimir Vapnik. On the theory of learnining with privileged information. In Advances in neural information processing systems, pages 1894-1902, 2010.","title":"On the theory of learnining with privileged information","authors":"Dmitry Pechyony, Vladimir Vapnik","year":"2010","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Pechyony et al., 2010] Dmitry Pechyony, Rauf Izmailov, Akshay Vashist, and Vladimir Vapnik. Smo-style algo- rithms for learning using privileged information. In DMIN, pages 235-241, 2010.","title":"Smo-style algorithms for learning using privileged information","authors":"Dmitry Pechyony, Rauf Izmailov, Akshay Vashist, Vladimir Vapnik","year":"2010","venue":"DMIN","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Read et al., 2011] Jesse Read, Bernhard Pfahringer, Geoff Holmes, and Eibe Frank. Classifier chains for multi-label classification. Machine learning, 85(3):333-359, 2011.","title":"Classifier chains for multi-label classification","authors":"Read","year":"2011","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Sharmanska et al., 2013] Viktoriia Sharmanska, Novi Quadrianto, and Christoph H Lampert. Learning to rank using privileged information. In Proceedings of the IEEE International Conference on Computer Vision, pages 825-832, 2013.","title":"Viktoriia Sharmanska, Novi Quadrianto, and Christoph H Lampert. Learning to rank using privileged information","authors":"Sharmanska","year":"2013","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Sun et al., 2014] Fuming Sun, Jinhui Tang, Haojie Li, Guo- Jun Qi, and Thomas S Huang. Multi-label image categoriza- tion with sparse factor representation. Image Processing, IEEE Transactions on, 23(3):1028-1037, 2014.","title":"Multi-label image categorization with sparse factor representation","authors":"Fuming Sun, Jinhui Tang, Haojie Li, Guo-Jun Qi, Thomas S Huang","year":"2014","venue":"IEEE Transactions on","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Tai and Lin, 2012] Farbound Tai and Hsuan-Tien Lin. Mul- tilabel classification with principal label space transforma- tion. Neural Computation, 24(9):2508-2542, 2012.","title":"Farbound Tai and Hsuan-Tien Lin. Multilabel classification with principal label space transformation","authors":"Lin Tai","year":"2012","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Takac et al., 2015] Martin Takac, Peter Richtarik, and Nathan Srebro. Distributed mini-batch sdca. arXiv preprint arXiv:1507.08322, 2015.","title":"Distributed mini-batch sdca","authors":"Takac","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Tsoumakas et al., 2010] Grigorios Tsoumakas, Ioannis Katakis, and Ioannis Vlahavas. Mining multi-label data. In Data mining and knowledge discovery handbook, pages 667-685. Springer, 2010.","title":"Mining multi-label data","authors":"Grigorios Tsoumakas, Ioannis Katakis, Ioannis Vlahavas","year":"2010","venue":"Data mining and knowledge discovery handbook","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tsoumakas et al., 2011] Grigorios Tsoumakas, Ioannis Katakis, and Ioannis Vlahavas. Random k-labelsets for multilabel classification. IEEE Transactions on Knowledge and Data Engineering, 23(7):1079-1089, 2011.","title":"Random k-labelsets for multilabel classification","authors":"Grigorios Tsoumakas, Ioannis Katakis, Ioannis Vlahavas","year":"2011","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Vapnik and Izmailov, 2015] Vladimir Vapnik and Rauf Iz- mailov. Learning using privileged information: Similarity control and knowledge transfer. Journal of Machine Learn- ing Research, 16:2023-2049, 2015.","title":"Vladimir Vapnik and Rauf Izmailov. Learning using privileged information: Similarity control and knowledge transfer","authors":"Izmailov Vapnik","year":"2015","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Vapnik and Vashist, 2009] Vladimir Vapnik and Akshay Vashist. A new learning paradigm: Learning using privi- leged information. Neural Networks, 22(5):544-557, 2009.","title":"Vladimir Vapnik and Akshay Vashist. A new learning paradigm: Learning using privileged information","authors":"Vashist Vapnik","year":"2009","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Vapnik et al., 2009] Vladimir Vapnik, Akshay Vashist, and Natalya Pavlovitch. Learning using hidden information (learning with teacher). In 2009 International Joint Confer- ence on Neural Networks, pages 3188-3195. IEEE, 2009.","title":"Learning using hidden information (learning with teacher)","authors":"Vladimir Vapnik, Akshay Vashist, Natalya Pavlovitch","year":"2009","venue":"2009 International Joint Conference on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xu et al., 2015] Chang Xu, Dacheng Tao, and Chao Xu. Large-margin multi-label causal feature learning. In AAAI, pages 1924-1930, 2015.","title":"Large-margin multi-label causal feature learning","authors":"Chang Xu, Dacheng Tao, Chao Xu","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xu et al., 2016a] Chang Xu, Dacheng Tao, and Chao Xu. Robust extreme multi-label learning. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowl- edge Discovery and Data Mining, San Francisco, CA, USA August, pages 13-17, 2016.","title":"Robust extreme multi-label learning","authors":"Xu","year":"2016","venue":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xu et al., 2016b] Xinxing Xu, Joey Tianyi Zhou, IvorW Tsang, Zheng Qin, Rick Siow Mong Goh, and Yong Liu. Simple and efficient learning using privileged information. arXiv preprint arXiv:1604.01518, 2016.","title":"Simple and efficient learning using privileged information","authors":"Xu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yang et al., 2016] Hao Yang, Joey Tianyi Zhou, Yu Zhang, Bin-Bin Gao, Jianxin Wu, and Jianfei Cai. Exploit bound- ing box annotations for multi-label object recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.","title":"Exploit bounding box annotations for multi-label object recognition","authors":"Hao Yang, Joey Tianyi Zhou, Yu Zhang, Bin-Bin Gao, Jianxin Wu, Jianfei Cai","year":"2016","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yu et al., 2014] Hsiang-Fu Yu, Prateek Jain, Purushottam Kar, and Inderjit Dhillon. Large-scale multi-label learning with missing labels. In Proceedings of The 31st Interna- tional Conference on Machine Learning, pages 593-601, 2014.","title":"Large-scale multi-label learning with missing labels","authors":"Hsiang-Fu Yu, Prateek Jain, Purushottam Kar, Inderjit Dhillon","year":"2014","venue":"Proceedings of The 31st International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang and Wu, 2015] Min-Ling Zhang and Lei Wu. Lift: Multi-label learning with label-specific features. IEEE transactions on pattern analysis and machine intelligence, 37(1):107-120, 2015.","title":"Lift: Multi-label learning with label-specific features","authors":"Wu ; ] Min-Ling Zhang, Lei Zhang, Wu","year":"2015","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2017] Hongyang Zhang, Shan You, Zhouchen Lin, and Chao Xu. Fast compressive phase retrieval under bounded noise. In Thirty-first AAAI Conference on Artificial Intelligence, 2017.","title":"Fast compressive phase retrieval under bounded noise","authors":"Hongyang Zhang, Shan You, Zhouchen Lin, Chao Xu","year":"2017","venue":"Thirty-first AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}