{"type":"ScholarlyArticle","title":"Binary Linear Compression for Multi-label Classification","authors":[{"name":"Wen-Ji Zhou","slug":"wen-ji-zhou","given":"Wen-Ji","family":"Zhou","affiliation":"♣ Collaborative Innovation Center of Novel Software Technology and Industrialization , Nanjing , China"},{"name":"Yang Yu","slug":"yang-yu","given":"Yang","family":"Yu","affiliation":"♣ Collaborative Innovation Center of Novel Software Technology and Industrialization , Nanjing , China; ♣ National Key Laboratory for Novel Software Technology , Nanjing University , Nanjing , China"},{"name":"Min-Ling Zhang","slug":"min-ling-zhang","given":"Min-Ling","family":"Zhang","affiliation":"♠ School of Computer Science and Engineering , Southeast University , Nanjing , China; ♠ Key Laboratory of Computer Network and Information Integration (SEU) , Ministry of Education , China"}],"abstract":"In multi-label classification tasks, labels are commonly related with each other. It has been well recognized that utilizing label relationship is essential to multi-label learning. One way to utilizing label relationship is to map labels to a lower-dimensional space of uncorrelated labels, where the relationship could be encoded in the mapping. Previous linear mapping methods commonly result in regression subproblems in the lower-dimensional label space. In this paper, we disclose that mappings to a low-dimensional multi-label regression problem can be worse than mapping to a classification problem, since regression requires more complex model than classification. We then propose the binary linear compression (BILC) method that results in a binary\r\nlabel space, leading to classification subproblems. Experiments on several multi-label datasets show that, employing classification in the embedded space results in much simpler models than regression, leading to smaller structure risk. The proposed methods are also shown to be superior to some state-of-the-art approaches.","doi":"10.24963/ijcai.2017/496","url":"https://ijcai.org/proceedings/2017/496","pdf":"https://ijcai.org/proceedings/2017/0496.pdf","markdown":"https://ijcai.org/proceedings/2017/496.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/496","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":496,"pages":{"first":3546,"last":3552},"keywords":["Machine Learning: Multi-instance/Multi-label/Multi-view learning","Machine Learning: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Agrawal et al., 2013] Rahul Agrawal, Archit Gupta, Yashoteja Prabhu, and Manik Varma. Multi-label learning with millions of labels: recommending advertiser bid phrases for web pages. In WWW, pages 13-24, 2013.","title":"Multi-label learning with millions of labels: recommending advertiser bid phrases for web pages","authors":"Rahul Agrawal, Archit Gupta, Yashoteja Prabhu, Manik Varma","year":"2013","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"Krishnakumar Bala- subramanian and Guy Lebanon. The landmark selection method for multiple output prediction. In ICML, 2012.","title":"The landmark selection method for multiple output prediction","authors":"Krishnakumar Balasubramanian, Guy Lebanon","year":"2012","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bhatia et al., 2015] Kush Bhatia, Himanshu Jain, Purushot- tam Kar, Manik Varma, and Prateek Jain. Sparse local em- beddings for extreme multi-label classification. In NIPS, pages 730-738, 2015.","title":"Sparse local embeddings for extreme multi-label classification","authors":"Bhatia","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Boufounos and Baraniuk, 2008] Petros Boufounos and Richard G. Baraniuk. 1-bit compressive sensing. In Pro- ceedings of the 42nd Annual Conference on Information Sciences and Systems, Princeton, NJ, pages 16-21, 2008.","title":"Petros Boufounos and Richard G. Baraniuk. 1-bit compressive sensing","authors":"Baraniuk Boufounos","year":"2008","venue":"Proceedings of the 42nd Annual Conference on Information Sciences and Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Boutell et al., 2004] Matthew R. Boutell, Jiebo Luo, Xipeng Shen, and Christopher M. Brown. Learning multi-label scene classification. Pattern Recognition, 37(9):1757-1771, 2004.","title":"Learning multi-label scene classification","authors":"Matthew R Boutell, Jiebo Luo, Xipeng Shen, Christopher M Brown","year":"2004","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Brochu et al., 2010] Eric Brochu, Vlad M. Cora, and Nando de Freitas. A tutorial on bayesian optimization of expen- sive cost functions, with application to active user mod- eling and hierarchical reinforcement learning. Computing Research Repository, abs/1012.2599, 2010.","title":"A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning","authors":"Eric Brochu, Vlad M Cora, Nando De Freitas","year":"2010","venue":"Computing Research Repository","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Carneiro et al., 2007] Gustavo Carneiro, Antoni B. Chan, Pedro J. Moreno, and Nuno Vasconcelos. Supervised learning of semantic classes for image annotation and re- trieval. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 29(3):394-410, 2007.","title":"Supervised learning of semantic classes for image annotation and retrieval","authors":"Gustavo Carneiro, Antoni B Chan, Pedro J Moreno, Nuno Vasconcelos","year":"2007","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chua et al., 2009] Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. NUS- WIDE: a real-world web image database from national university of singapore. In Proceedings of the 8th ACM International Conference on Image and Video Retrieval, Santorini Island, Greece, 2009.","title":"NUS-WIDE: a real-world web image database from national university of singapore","authors":"Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, Yantao Zheng","year":"2009","venue":"Proceedings of the 8th ACM International Conference on Image and Video Retrieval","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Duygulu et al., 2002] Pinar Duygulu, Kobus Barnard, João F. G. de Freitas, and David A. Forsyth. Object recogni- tion as machine translation: Learning a lexicon for a fixed image vocabulary. In ECCV, pages 97-112, 2002.","title":"Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary","authors":"Pinar Duygulu, Kobus Barnard, F G João, David A De Freitas, Forsyth","year":"2002","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Freund and Schapire, 1997] Yoav Freund and Robert E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Com- puter and System Sciences, 55(1):119-139, 1997.","title":"A decision-theoretic generalization of on-line learning and an application to boosting","authors":"Yoav Freund, Robert E Schapire","year":"1997","venue":"Journal of Computer and System Sciences","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fürnkranz et al., 2008] Johannes Fürnkranz, Eyke Hüllermeier, Eneldo Loza Mencía, and Klaus Brinker. Multilabel classification via calibrated label ranking. Machine Learning, 73(2):133-153, 2008.","title":"Multilabel classification via calibrated label ranking","authors":"Johannes Fürnkranz, Eyke Hüllermeier, Eneldo Loza Mencía, Klaus Brinker","year":"2008","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Goldberg, 1989] David E. Goldberg. Genetic Algorithms in Search Optimization and Machine Learning. Addison- Wesley, 1989.","title":"Genetic Algorithms in Search Optimization and Machine Learning","authors":"David E Goldberg","year":"1989","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hsu et al., 2009] Daniel J. Hsu, Sham Kakade, John Lang- ford, and Tong Zhang. Multi-label prediction via com- pressed sensing. In NIPS, pages 772-780, 2009.","title":"Multi-label prediction via compressed sensing","authors":"J Daniel, Sham Hsu, John Kakade, Tong Langford, Zhang","year":"2009","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jerome et al., 2001] Friedman Jerome, Hastie Trevor, and Tibshirani Robert. The Elements of Statistical Learning. Springer, Berlin, 2001.","title":"The Elements of Statistical Learning","authors":"Friedman Jerome, Trevor Hastie, Tibshirani Robert","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Katakis et al., 2008] Ioannis Katakis, Tsoumakas Grigo- rios, and Vlahavas Ioannis. Multilabel text classification for automated tag suggestion. ECML/PKDD, 75, 2008.","title":"Multilabel text classification for automated tag suggestion","authors":"Ioannis Katakis, Tsoumakas Grigorios, Vlahavas Ioannis","year":"2008","venue":"ECML/PKDD","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2010] Hong Li, Yuejian Guo, Min Wu, Ping Li, and Yao Xiang. Combine multi-valued attribute decom- position with multi-label learning. Expert Systems with Applications, 37(12):8721-8728, 2010.","title":"Combine multi-valued attribute decomposition with multi-label learning","authors":"Li","year":"2010","venue":"Expert Systems with Applications","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Munos, 2014] Rémi Munos. From bandits to monte-carlo tree search: The optimistic principle applied to optimiza- tion and planning. Foundations and Trends in Machine Learning, 7(1):1-129, 2014.","title":"From bandits to monte-carlo tree search: The optimistic principle applied to optimization and planning","authors":"Rémi Munos, Munos","year":"2014","venue":"Foundations and Trends in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pati et al., 1993] Yagyensh Chandra Pati, Ramin Rezaii- far, and PS Krishnaprasad. Orthogonal matching pur- suit: Recursive function approximation with applications to wavelet decomposition. In Proceedings of the 27th Asilomar Conference on Signals, Systems and Computers, pages 40-44, 1993.","title":"Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition","authors":"Pati","year":"1993","venue":"Proceedings of the 27th Asilomar Conference on Signals, Systems and Computers","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Prabhu and Varma, 2014] Yashoteja Prabhu and Manik Varma. Fastxml: a fast, accurate and stable tree-classifier for extreme multi-label learning. In SIGKDD, pages 263- 272, 2014.","title":"Yashoteja Prabhu and Manik Varma. Fastxml: a fast, accurate and stable tree-classifier for extreme multi-label learning","authors":"Varma Prabhu","year":"2014","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tai and Lin, 2012] Farbound Tai and Hsuan-Tien Lin. Mul- tilabel classification with principal label space transforma- tion. Neural Computation, 24(9):2508-2542, 2012.","title":"Farbound Tai and Hsuan-Tien Lin. Multilabel classification with principal label space transformation","authors":"Lin Tai","year":"2012","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tsoumakas and Katakis, 2007] Grigorios Tsoumakas and Ioannis Katakis. Multi-label classification: An overview. International Journal of Data Warehousing and Mining, 3:1-13, 2007.","title":"Grigorios Tsoumakas and Ioannis Katakis. Multi-label classification: An overview","authors":"Katakis Tsoumakas","year":"2007","venue":"International Journal of Data Warehousing and Mining","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tsoumakas and Vlahavas, 2007] Grigorios Tsoumakas and Ioannis P. Vlahavas. Random k -labelsets: An ensem- ble method for multilabel classification. In ECML/PKDD, pages 406-417, 2007.","title":"Random k -labelsets: An ensemble method for multilabel classification","authors":"Grigorios Tsoumakas, Ioannis P Vlahavas","year":"2007","venue":"ECML/PKDD","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tsoumakas et al., 2008] Grigorios Tsoumakas, Ioannis Katakis, and Ioannis Vlahavas. Effective and efficient multilabel classification in domains with large number of labels. In ECML/PKDD, pages 30-44, 2008.","title":"Effective and efficient multilabel classification in domains with large number of labels","authors":"Grigorios Tsoumakas, Ioannis Katakis, Ioannis Vlahavas","year":"2008","venue":"ECML/PKDD","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wicker et al., 2012] Jörg Wicker, Bernhard Pfahringer, and Stefan Kramer. Multi-label classification using boolean matrix decomposition. In Proceedings of the 27th Annual ACM Symposium on Applied Computing, pages 179-186. ACM, 2012.","title":"Multi-label classification using boolean matrix decomposition","authors":"Wicker","year":"2012","venue":"Proceedings of the 27th Annual ACM Symposium on Applied Computing","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yu et al., 2016] Yang Yu, Hong Qian, and Yi-Qi Hu. Derivative-free optimization via classification. In AAAI, pages 2286-2292, 2016.","title":"Derivative-free optimization via classification","authors":"Yang Yu, Hong Qian, Yi-Qi Hu","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang and Schneider, 2011] Yi Zhang and Jeff G. Schnei- der. Multi-label output codes using canonical correlation analysis. In AISTAT, pages 873-882, 2011.","title":"Schneider. Multi-label output codes using canonical correlation analysis","authors":"Yi Zhang, Jeff G","year":"2011","venue":"AISTAT","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang and Zhou, 2006] Min-Ling Zhang and Zhi-Hua Zhou. Multi-label neural networks with applications to functional genomics and text categorization. IEEE Transactions on Knowledge and Data Engineering, 18(10):1338-1351, 2006.","title":"Multi-label neural networks with applications to functional genomics and text categorization","authors":"Zhou ; ] Min-Ling Zhang, Zhi-Hua Zhang, Zhou","year":"2006","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang and Zhou, 2007] Min-Ling Zhang and Zhi-Hua Zhou. ML-KNN: A lazy learning approach to multi-label learning. Pattern Recognition, 40(7):2038-2048, 2007.","title":"ML-KNN: A lazy learning approach to multi-label learning","authors":"Zhou ; ] Min-Ling Zhang, Zhi-Hua Zhang, Zhou","year":"2007","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}