{"type":"ScholarlyArticle","title":"Deep Discrete Prototype Multilabel Learning","authors":[{"name":"Xiaobo Shen","slug":"xiaobo-shen","given":"Xiaobo","family":"Shen","affiliation":"NTU Corporate Lab , Nanyang Technological University School of Computer Science and Engineering , The University of New South Wales"},{"name":"Weiwei Liu","slug":"weiwei-liu","given":"Weiwei","family":"Liu","affiliation":"NTU Corporate Lab , Nanyang Technological University School of Computer Science and Engineering , The University of New South Wales"},{"name":"Yong Luo","slug":"yong-luo","given":"Yong","family":"Luo","affiliation":"NTU Corporate Lab , Nanyang Technological University School of Computer Science and Engineering , The University of New South Wales"},{"name":"Yew-Soon Ong","slug":"yew-soon-ong","given":"Yew-Soon","family":"Ong","affiliation":"FEIT , University of Technology Sydney"},{"name":"Ivor W. Tsang","slug":"ivor-w-tsang","given":"Ivor W.","family":"Tsang","affiliation":"§ School of Computer Science and Engineering , Nanyang Technological University Centre for Artificial Intelligence ,"}],"abstract":"kNN embedding methods, such as the state-of-the-art LM-kNN, have shown impressive results in multi-label learning. Unfortunately, these approaches suffer expensive computation and memory costs in large-scale settings. To fill this gap, this paper proposes a novel deep prototype compression, i.e., DBPC for fast multi-label prediction. DBPC compresses the database into a small set of short discrete prototypes, and uses the prototypes for prediction. The benefit of DBPC comes from two aspects: 1) The number of distance comparisons are reduced in the prototype; 2) The distance computation cost is significantly decreased in the reduced space. We propose to jointly learn the deep latent subspace and discrete prototypes within one framework. The encoding and decoding neural networks are employed to make deep discrete prototypes well represent the instances and labels. Extensive experiments on several large-scale datasets demonstrate that DBPC achieves several orders of magnitude lower storage and prediction complexity than state-of-the-art multi-label methods, while achieving competitive accuracy.","doi":"10.24963/ijcai.2018/371","url":"https://ijcai.org/proceedings/2018/371","pdf":"https://ijcai.org/proceedings/2018/0371.pdf","markdown":"https://ijcai.org/proceedings/2018/371.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/371","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":371,"pages":{"first":2675,"last":2681},"keywords":["Machine Learning: Classification","Machine Learning: Multi-instance;Multi-label;Multi-view learning"],"videos":[],"references":[{"position":1,"raw":"[Bhatia et al., 2015] Kush Bhatia, Himanshu Jain, Purushot- tam Kar, Manik Varma, and Prateek Jain. Sparse local em- beddings for extreme multi-label classification. In NIPS, pages 730-738, 2015.","title":"Sparse local embeddings for extreme multi-label classification","authors":"Bhatia","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chatfield et al., 2014] Ken Chatfield, Karen Simonyan, An- drea Vedaldi, and Andrew Zisserman. Return of the devil in the details: Delving deep into convolutional nets. In BMVC, 2014.","title":"Return of the devil in the details: Delving deep into convolutional nets","authors":"Ken Chatfield, Karen Simonyan, Andrea Vedaldi, Andrew Zisserman","year":"2014","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen and Lin, 2012] Yao-Nan Chen and Hsuan-Tien Lin. Feature-aware label space dimension reduction for multi- label classification. In NIPS, pages 1529-1537, 2012.","title":"Feature-aware label space dimension reduction for multilabel classification","authors":"Chen, Lin ; ] Yao-Nan Chen, Hsuan-Tien Lin","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chua et al., 2009] Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. Nus- wide: a real-world web image database from national uni- versity of singapore. In CIVR, 2009.","title":"Nuswide: a real-world web image database from national university of singapore","authors":"Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, Yantao Zheng","year":"2009","venue":"CIVR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cissé et al., 2016] Moustapha Cissé, Maruan Al-Shedivat, and Samy Bengio. ADIOS: architectures deep in output space. In ICML, pages 2770-2779, 2016.","title":"ADIOS: architectures deep in output space","authors":"Moustapha Cissé, Maruan Al-Shedivat, Samy Bengio","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fan et al., 2008] Rong-En Fan, Kai-Wei Chang, Cho-Jui H- sieh, Xiang-Rui Wang, and Chih-Jen Lin. Liblinear: A library for large linear classification. JMLR, 9:1871-1874, 2008.","title":"Liblinear: A library for large linear classification","authors":"Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, Chih-Jen Lin","year":"2008","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[García et al., 2012] Salvador García, Joaquín Derrac, José Ramón Cano, and Francisco Herrera. Prototype selection for nearest neighbor classification: Taxonomy and empirical study. 34(3):417-435, 2012.","title":"Salvador García, Joaquín Derrac, José Ramón Cano, and Francisco Herrera. Prototype selection for nearest neighbor classification","authors":"García","year":"2012","venue":"Taxonomy and empirical study","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hsu et al., 2009] Daniel Hsu, Sham Kakade, John Lang- ford, and Tong Zhang. Multi-label prediction via com- pressed sensing. In NIPS, pages 772-780, 2009.","title":"Multi-label prediction via compressed sensing","authors":"Daniel Hsu, Sham Kakade, John Langford, Tong Zhang","year":"2009","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kusner et al., 2014] Matt J. Kusner, Stephen Tyree, Kil- ian Q. Weinberger, and Kunal Agrawal. Stochastic neigh- bor compression. In ICML, pages 622-630, 2014.","title":"Stochastic neighbor compression","authors":"Matt J Kusner, Stephen Tyree, Kilian Q Weinberger, Kunal Agrawal","year":"2014","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kwok and Tsang, 2003] James T. Kwok and Ivor W. Tsang. Learning with idealized kernels. In ICML, pages 400-407, 2003.","title":"Learning with idealized kernels","authors":"James T Kwok, Ivor W Tsang","year":"2003","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu and Tsang, 2015] Weiwei Liu and Ivor W. Tsang. Large margin metric learning for multi-label prediction. In AAAI, pages 2800-2806, 2015.","title":"Large margin metric learning for multi-label prediction","authors":"Tsang Liu, Weiwei Liu, Ivor W Tsang","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu and Tsang, 2017] Weiwei Liu and Ivor W. Tsang. Mak- ing decision trees feasible in ultrahigh feature and label dimensions. JMLR, 18:81:1-81:36, 2017.","title":"Making decision trees feasible in ultrahigh feature and label dimensions","authors":"Tsang Liu, Weiwei Liu, Ivor W Tsang","year":"2017","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2017] Weiwei Liu, Ivor W. Tsang, and Klaus- Robert Müller. An easy-to-hard learning paradigm for multiple classes and multiple labels. JMLR, 18:94:1- 94:38, 2017.","title":"An easy-to-hard learning paradigm for multiple classes and multiple labels","authors":"Weiwei Liu, Ivor W Tsang, Klaus-Robert Müller","year":"2017","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Nam et al., 2017] Jinseok Nam, Eneldo Loza Mencía, Hyunwoo J. Kim, and Johannes Fürnkranz. Maximizing subset accuracy with recurrent neural networks in multi- label classification. In NIPS, pages 5419-5429, 2017.","title":"Maximizing subset accuracy with recurrent neural networks in multilabel classification","authors":"Nam","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Norouzi et al., 2012] Mohammad Norouzi, David J. Fleet, and Ruslan Salakhutdinov. Hamming distance metric learning. In NIPS, pages 1070-1078, 2012.","title":"Hamming distance metric learning","authors":"Mohammad Norouzi, David J Fleet, Ruslan Salakhutdinov","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Schönemann, 1966] Peter H Schönemann. A generalized solution of the orthogonal procrustes problem. Psychome- trika, 31(1):1-10, 1966.","title":"A generalized solution of the orthogonal procrustes problem","authors":"Peter H Schönemann","year":"1966","venue":"Psychometrika","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shen et al., 2017] Xiaobo Shen, Weiwei Liu, Ivor W. Tsang, Fumin Shen, and Quan-Sen Sun. Compressed k-means for large-scale clustering. In AAAI, pages 2527-2533, 2017.","title":"Compressed k-means for large-scale clustering","authors":"Xiaobo Shen, Weiwei Liu, Ivor W Tsang, Fumin Shen, Quan-Sen Sun","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shen et al., 2018] Xiaobo Shen, Weiwei Liu, Ivor W. Tsang, Quan-Sen Sun, and Yew-Soon Ong. Multilabel prediction via cross-view search. TNNLS, PP(P):1-15, 2018.","title":"Multilabel prediction via cross-view search","authors":"Xiaobo Shen, Weiwei Liu, Ivor W Tsang, Quan-Sen, Yew-Soon Sun, Ong","year":"2018","venue":"TNNLS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Tai and Lin, 2012] Farbound Tai and Hsuan-Tien Lin. Mul- tilabel classification with principal label space transforma- tion. Neural Computation, 24(9):2508-2542, 2012.","title":"Farbound Tai and Hsuan-Tien Lin. Multilabel classification with principal label space transformation","authors":"Lin Tai","year":"2012","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tsoumakas et al., 2009] Grigorios Tsoumakas, Ioannis Katakis, and Ioannis Vlahavas. Mining multi-label data. In Data mining and knowledge discovery handbook, pages 667-685. 2009.","title":"Mining multi-label data","authors":"Grigorios Tsoumakas, Ioannis Katakis, Ioannis Vlahavas","year":"2009","venue":"Data mining and knowledge discovery handbook","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2018] Jingdong Wang, Ting Zhang, Jingkuan Song, Nicu Sebe, and Heng Tao Shen. A survey on learn- ing to hash. TPAMI, 40(4):769-790, 2018.","title":"A survey on learning to hash","authors":"Jingdong Wang, Ting Zhang, Jingkuan Song, Nicu Sebe, Heng Tao Shen","year":"2018","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xu et al., 2015] Zhixiang Eddie Xu, Jacob R. Gardner, Stephen Tyree, and Kilian Q. Weinberger. Compressed support vector machines. CoRR, abs/1501.06478, 2015.","title":"Compressed support vector machines","authors":"Eddie Zhixiang, Jacob R Xu, Stephen Gardner, Kilian Q Tyree, Weinberger","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang and Schneider, 2011] Yi Zhang and Jeff Schneider. Multi-label output codes using canonical correlation anal- ysis. In AISTATS, pages 873-882, 2011.","title":"Multi-label output codes using canonical correlation analysis","authors":"Yi Zhang, Jeff Schneider","year":"2011","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang and Schneider, 2012] Yi Zhang and Jeff Schneider. Maximum margin output coding. In ICML, pages 1575- 1582, 2012.","title":"Maximum margin output coding","authors":"Yi Zhang, Jeff Schneider","year":"2012","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang and Zhou, 2007] Min-Ling Zhang and Zhi-Hua Zhou. ML-KNN: A lazy learning approach to multi-label learning. PR, 40(7):2038-2048, 2007.","title":"ML-KNN: A lazy learning approach to multi-label learning","authors":"Zhou ; ] Min-Ling Zhang, Zhi-Hua Zhang, Zhou","year":"2007","venue":"PR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang and Zhou, 2014] Min-Ling Zhang and Zhi-Hua Zhou. A review on multi-label learning algorithms. TKDE, 26(8):1819-1837, 2014.","title":"A review on multi-label learning algorithms","authors":"Zhou ; ] Min-Ling Zhang, Zhi-Hua Zhang, Zhou","year":"2014","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2009] Kai Zhang, James T. Kwok, and Bahram Parvin. Prototype vector machine for large scale semi-supervised learning. In ICML, pages 1233-1240, 2009.","title":"Prototype vector machine for large scale semi-supervised learning","authors":"Kai Zhang, James T Kwok, Bahram Parvin","year":"2009","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhong et al., 2017] Kai Zhong, Ruiqi Guo, Sanjiv Kumar, Bowei Yan, David Simcha, and Inderjit S. Dhillon. Fast classification with binary prototypes. In AISTATS, pages 1255-1263, 2017.","title":"Fast classification with binary prototypes","authors":"Kai Zhong, Ruiqi Guo, Sanjiv Kumar, Bowei Yan, David Simcha, Inderjit S Dhillon","year":"2017","venue":"AISTATS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}