{"type":"ScholarlyArticle","title":"Learning for Tail Label Data: A Label-Specific Feature Approach","authors":[{"name":"Tong Wei","slug":"tong-wei","given":"Tong","family":"Wei","affiliation":"National Key Laboratory for Novel Software Technology Nanjing University , Nanjing 210023 , China"},{"name":"Wei-Wei Tu","slug":"wei-wei-tu","given":"Wei-Wei","family":"Tu","affiliation":"4Paradigm Inc. , Beijing , China"},{"name":"Yu-Feng Li","slug":"yu-feng-li","given":"Yu-Feng","family":"Li","affiliation":"National Key Laboratory for Novel Software Technology Nanjing University , Nanjing 210023 , China"}],"abstract":"Tail label data (TLD) is prevalent in real-world tasks, and large-scale multi-label learning (LMLL) is its major learning scheme. Previous LMLL studies typically need to additionally take into account extensive head label data (HLD), and thus fail to guide the learning behavior of TLD. In many applications such as recommender systems, however, the prediction of tail label is very necessary, since it provides very important supplementary information. We call this kind of problem as \\emph{tail label learning}. In this paper, we propose a novel method for the tail label learning problem. Based on the observation that the raw feature representation in LMLL data usually benefits HLD, which may not be suitable for TLD, we construct effective and rich label-specific features through exploring labeled data distribution and leveraging label correlations. Specifically, we employ clustering analysis to explore discriminative features for each tail label replacing the original high-dimensional and sparse features. In addition, due to the scarcity of positive examples of TLD, we encode knowledge from HLD by exploiting label correlations to enhance the label-specific features. Experimental results verify the superiority of the proposed method in terms of performance on TLD.","doi":"10.24963/ijcai.2019/533","url":"https://ijcai.org/proceedings/2019/533","pdf":"https://ijcai.org/proceedings/2019/0533.pdf","markdown":"https://ijcai.org/proceedings/2019/533.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/533","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":533,"pages":{"first":3842,"last":3848},"keywords":["Machine Learning: Multi-instance;Multi-label;Multi-view learning","Machine Learning: Classification"],"videos":[],"references":[{"position":1,"raw":"[Babbar and Schölkopf, 2017] Rohit Babbar and Bernhard Schölkopf. Dismec: Distributed sparse machines for ex- treme multi-label classification. In Proceedings of the 10th ACM International Conference on Web Search and Data Mining, pages 721-729, Cambridge, UK, 2017.","title":"Dismec: Distributed sparse machines for extreme multi-label classification","authors":"Schölkopf Babbar, Rohit Babbar, Bernhard Schölkopf","year":"2017","venue":"Proceedings of the 10th ACM International Conference on Web Search and Data Mining","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Babbar and Schölkopf, 2018] Rohit Babbar and Bernhard Schölkopf. Adversarial extreme multi-label classification. arXiv preprint arXiv:1803.01570, 2018.","title":"Adversarial extreme multi-label classification","authors":"Rohit Babbar, Bernhard Schölkopf","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bhatia et al., 2015] Kush Bhatia, Himanshu Jain, Purushot- tam Kar, Manik Varma, and Prateek Jbabain. Sparse local embeddings for extreme multi-label classification. In Ad- vances in Neural Information Processing Systems 28, pages 730-738. Montreal, Canada, 2015.","title":"Sparse local embeddings for extreme multi-label classification","authors":"Bhatia","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of the 22nd IEEE Conference on Computer Vision and Pattern Recog- nition, pages 248-255, Miami, FL, 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"Proceedings of the 22nd IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ebert et al., 2010] Sandra Ebert, Diane Larlus, and Bernt Schiele. Extracting structures in image collections for ob- ject recognition. In Proceedings of European Conference on Computer Vision, pages 720-733, Heraklion, Greece, 2010.","title":"Extracting structures in image collections for object recognition","authors":"Sandra Ebert, Diane Larlus, Bernt Schiele","year":"2010","venue":"Proceedings of European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fan et al., 2008] Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. Liblinear: A library for large linear classification. Journal of Machine Learning Research, 9:1871-1874, 2008.","title":"Liblinear: A library for large linear classification","authors":"Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, Chih-Jen Lin","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Himanshu et al., 2016] Jain Himanshu, Prabhu Yashoteja, and Manik Varma. Extreme multi-label loss functions for recommendation, tagging, ranking & other missing label applications. In Proceedings of the 22nd ACM SIGKDD In- ternational Conference on Knowledge Discovery and Data Mining, pages 935-944, San Francisco, CA, 2016.","title":"Extreme multi-label loss functions for recommendation, tagging, ranking & other missing label applications","authors":"Jain Himanshu, Prabhu Yashoteja, Manik Varma","year":"2016","venue":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hsiang-Fu et al., 2014] Yu Hsiang-Fu, Jain Prateek, Kar Pu- rushottam, and Dhillon Inderjit S. Large-scale multi-label learning with missing labels. In Proceedings of the 31st International Conference on Machine Learning, pages 593- 601, Beijing, China, 2014.","title":"Large-scale multi-label learning with missing labels","authors":"Yu Hsiang-Fu, Jain Prateek, Kar Purushottam, Dhillon Inderjit, S","year":"2014","venue":"Proceedings of the 31st International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ioannis et al., 2015] Partalas Ioannis, Kosmopoulos Aris, Baskiotis Nicolas, Artieres Thierry, Paliouras George, Gaussier Eric, Androutsopoulos Ion, Amini Massih-Reza, and Galinari Patrick. Lshtc: A benchmark for large-scale text classification. arXiv preprint arXiv:1503.08581, 2015.","title":"Lshtc: A benchmark for large-scale text classification","authors":"Ioannis","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jia and Zhang, 2019] Bin-Bin Jia and Min-Ling Zhang. Multi-dimensional classification via knn feature augmen- tation. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence, Honolulu, HI, 2019.","title":"Multi-dimensional classification via knn feature augmentation","authors":"Zhang ; Bin-Bin Jia, Min-Ling Jia, Zhang","year":"2019","venue":"Proceedings of the 33rd AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jian et al., 2016] Ling Jian, Jundong Li, Kai Shu, and Huan Liu. Multi-label informed feature selection. In Proceedings of the 25th International Joint Conference on Artificial Intelligence, pages 1627-1633, New York City, NY, 2016.","title":"Multi-label informed feature selection","authors":"Ling Jian, Jundong Li, Kai Shu, Huan Liu","year":"2016","venue":"Proceedings of the 25th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li and Liang, 2019] Yu-Feng Li and De-Ming Liang. Safe semi-supervised learning: a brief introduction. Frontiers of Computer Science, 13(4):669-676, 2019.","title":"Safe semi-supervised learning: a brief introduction","authors":"Yu-Feng Li, De-Ming Liang","year":"2019","venue":"Frontiers of Computer Science","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2016] Yu-Feng Li, Shao-Bo Wang, and Zhi-Hua Zhou. Graph quality judgement: A large margin expedition. In Proceedings of the 25th International Joint Conference on Artificial Intelligence, pages 1725-1731, New York City, NY, 2016.","title":"Graph quality judgement: A large margin expedition","authors":"Yu-Feng Li, Shao-Bo Wang, Zhi-Hua Zhou","year":"2016","venue":"Proceedings of the 25th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ma et al., 2012] Zhigang Ma, Feiping Nie, Yi Yang, Jasper R. Uijlings, and Nicu Sebe. Web image annota- tion via subspace-sparsity collaborated feature selection. IEEE Transactions on Multimedia, 14(4):1021-1030, 2012.","title":"Web image annotation via subspace-sparsity collaborated feature selection","authors":"Zhigang Ma, Feiping Nie, Yi Yang, Jasper R Uijlings, Nicu Sebe","year":"2012","venue":"IEEE Transactions on Multimedia","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Maier et al., 2009] Markus Maier, Ulrike von Luxburg, and Matthias Hein. Influence of graph construction on graph- based clustering measures. In Advances in Neural Informa- tion Processing Systems 22, pages 1025-1032, Vancouver, Canada, 2009.","title":"Influence of graph construction on graphbased clustering measures","authors":"Maier","year":"2009","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[McAuley et al., 2015] Julian McAuley, Rahul Pandey, and Jure Leskovec. Inferring networks of substitutable and complementary products. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discov- ery and Data Mining, pages 785-794, Sydney, Australia, 2015.","title":"Inferring networks of substitutable and complementary products","authors":"Julian Mcauley, Rahul Pandey, Jure Leskovec","year":"2015","venue":"Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Prabhu and Varma, 2014] Yashoteja Prabhu and Manik Varma. Fastxml: A fast, accurate and stable tree-classifier for extreme multi-label learning. In Proceedings of the 20th ACM SIGKDD International Conference on Knowl- edge Discovery and Data Mining, pages 263-272, New York City, NY, 2014.","title":"Yashoteja Prabhu and Manik Varma. Fastxml: A fast, accurate and stable tree-classifier for extreme multi-label learning","authors":"Varma Prabhu","year":"2014","venue":"Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sami et al., 2016] Abu-El-Haija Sami, Kothari Nisarg, Lee Joonseok, Natsev Paul, Toderici George, Varadarajan Bal- akrishnan, and Vijayanarasimhan Sudheendra. Youtube- 8m: A large-scale video classification benchmark. arXiv preprint arXiv:1609.08675, 2016.","title":"Abu-El-Haija Sami","authors":"Sami","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shen et al., 2018] Xiaobo Shen, Weiwei Liu, Ivor W. Tsang, Quan-Sen Sun, and Yew-Soon Ong. Compact multi-label learning. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, pages 4066-4073, New Orleans, LA, 2018.","title":"Compact multi-label learning","authors":"Xiaobo Shen, Weiwei Liu, Ivor W Tsang, Quan-Sen, Yew-Soon Sun, Ong","year":"2018","venue":"Proceedings of the 32nd AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2017] Yu-Xiong Wang, Deva Ramanan, and Martial Hebert. Learning to model the tail. In Advances in Neural Information Processing Systems 30, pages 7032- 7042, Long Beach, CA, 2017.","title":"Learning to model the tail","authors":"Yu-Xiong Wang, Deva Ramanan, Martial Hebert","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wei and Li, 2018] Tong Wei and Yu-Feng Li. Does tail label help for large-scale multi-label learning. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, pages 2847-2853, Stockholm, Sweden, 2018.","title":"Does tail label help for large-scale multi-label learning","authors":"Tong Wei, Yu-Feng Li","year":"2018","venue":"Proceedings of the 27th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2018/395"},{"position":22,"raw":"[Wei and Li, 2019] Tong Wei and Yu-Feng Li. Learning com- pact model for large-scale multi-label data. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence, Hon- olulu, HI, 2019.","title":"Learning compact model for large-scale multi-label data","authors":"Tong Wei, Yu-Feng Li","year":"2019","venue":"Proceedings of the 33rd AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xu et al., 2016] Chang Xu, Da-Cheng Tao, and Chao Xu. Robust extreme multi-label learning. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowl- edge Discovery and Data Mining, pages 1275-1284, San Francisco, CA, 2016.","title":"Robust extreme multi-label learning","authors":"Chang Xu, Da-Cheng Tao, Chao Xu","year":"2016","venue":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yen et al., 2016] Ian E.-H. Yen, Xiangru Huang, Kai Zhong, Pradeep Ravikumar, and Inderjit S. Dhillon. Pd-sparse: A primal and dual sparse approach to extreme multiclass and multilabel classification. In Proceedings of the 33rd Interna- tional Conference on Machine Learning, pages 3069-3077, New York City, NY, 2016.","title":"Pd-sparse: A primal and dual sparse approach to extreme multiclass and multilabel classification","authors":"Ian E.-H Yen, Xiangru Huang, Kai Zhong, Pradeep Ravikumar, Inderjit S Dhillon","year":"2016","venue":"Proceedings of the 33rd International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang and Wu, 2015] Min-Ling Zhang and Lei Wu. Lift: Multi-label learning with label-specific features. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(1):107-120, 2015.","title":"Lift: Multi-label learning with label-specific features","authors":"Wu ; ] Min-Ling Zhang, Lei Zhang, Wu","year":"2015","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang and Zhou, 2014] Min-Ling Zhang and Zhi-Hua Zhou. A review on multi-label learning algorithms. IEEE Trans- actions on Knowledge and Data Engineering, 26(8):1819- 1837, 2014.","title":"A review on multi-label learning algorithms","authors":"Zhou ; ] Min-Ling Zhang, Zhi-Hua Zhang, Zhou","year":"2014","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2009] Min-Ling Zhang, José M. Peña, and Vic- tor Robles. Feature selection for multi-label naive bayes classification. Information Sciences, 179(19):3218-3229, 2009.","title":"and Victor Robles. Feature selection for multi-label naive bayes classification","authors":"Zhang","year":"2009","venue":"Information Sciences","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}