{"type":"ScholarlyArticle","title":"Disentangled Variational Autoencoder based Multi-Label Classification with Covariance-Aware Multivariate Probit Model","authors":[{"name":"Junwen Bai","slug":"junwen-bai","given":"Junwen","family":"Bai","affiliation":"Department of Computer Science , Cornell University"},{"name":"Shufeng Kong","slug":"shufeng-kong","given":"Shufeng","family":"Kong","affiliation":"Department of Computer Science , Cornell University"},{"name":"Carla Gomes","slug":"carla-gomes","given":"Carla","family":"Gomes","affiliation":"Department of Computer Science , Cornell University"}],"abstract":"Multi-label classification is the challenging task of predicting the presence and absence of multiple targets, involving representation learning and label correlation modeling. We propose a novel framework for multi-label classification, Multivariate Probit Variational AutoEncoder (MPVAE), that effectively learns latent embedding spaces as well as label correlations. MPVAE learns and aligns two probabilistic embedding spaces for labels and features respectively. The decoder of MPVAE takes in the samples from the embedding spaces and models the joint distribution of output targets under a Multivariate Probit model by learning a shared covariance matrix. We show that MPVAE outperforms the existing state-of-the-art methods on important computational sustainability applications as well as on other application domains, using public real-world datasets. MPVAE is further shown to remain robust under noisy settings. Lastly, we demonstrate the interpretability of the learned covariance by a case study on a bird observation dataset.","doi":"10.24963/ijcai.2020/595","url":"https://ijcai.org/proceedings/2020/595","pdf":"https://ijcai.org/proceedings/2020/0595.pdf","markdown":"https://ijcai.org/proceedings/2020/595.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/595","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Special track on AI for CompSust and Human well-being","track":"Special track on AI for CompSust and Human well-being","number":595,"pages":{"first":4313,"last":4321},"keywords":["Machine Learning: Multi-instance;Multi-label;Multi-view learning","Uncertainty in AI: Approximate Probabilistic Inference","Machine Learning: Deep Generative Models","Machine Learning: Interpretability"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24920/file_24920.m3u8","length_sec":281,"provider":"hls"}],"references":[{"position":1,"raw":"[Alazaidah et al., 2015] Raed Alazaidah, Fadi Thabtah, and Qasem Al-Radaideh. A multi-label classification approach based on correlations among labels. International Journal of Advanced Computer Science and Applications, 2015.","title":"A multi-label classification approach based on correlations among labels","authors":"Alazaidah","year":"2015","venue":"International Journal of Advanced Computer Science and Applications","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Benites and Sapozhnikova, 2015] Fernando Benites and Elena Sapozhnikova. Haram: a hierarchical aram neural network for large-scale text classification. In 2015 IEEE International Conference on Data Mining Workshop (ICDMW), pages 847-854. IEEE, 2015.","title":"Haram: a hierarchical aram neural network for large-scale text classification","authors":"Sapozhnikova Benites, Fernando Benites, Elena Sapozhnikova","year":"2015","venue":"IEEE International Conference on Data Mining Workshop (ICDMW)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bhatia et al., 2015] Kush Bhatia, Himanshu Jain, Purushot- tam Kar, Manik Varma, and Prateek Jain. Sparse local embeddings for extreme multi-label classification. In Ad- vances in neural information processing systems, 2015.","title":"Sparse local embeddings for extreme multi-label classification","authors":"Bhatia","year":"2015","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bi and Kwok, 2014] Wei Bi and James T Kwok. Multilabel classification with label correlations and missing labels. In Proceedings of the Twenty-Eighth AAAI Conference on Ar- tificial Intelligence, pages 1680-1686, 2014.","title":"Multilabel classification with label correlations and missing labels","authors":"Wei Bi, James T Kwok","year":"2014","venue":"Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Boutell et al., 2004] Matthew R Boutell, Jiebo Luo, Xipeng Shen, and Christopher M Brown. Learning multi-label scene classification. Pattern recognition, 37(9):1757- 1771, 2004.","title":"Learning multi-label scene classification","authors":"Jiebo Matthew R Boutell, Xipeng Luo, Christopher M Shen, Brown","year":"2004","venue":"Pattern recognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Carton et al., 2018] James A Carton, Gennady A Chepurin, and Ligang Chen. Soda3: A new ocean climate reanalysis. Journal of Climate, 31(17):6967-6983, 2018.","title":"Soda3: A new ocean climate reanalysis","authors":"Gennady A James A Carton, Ligang Chepurin, Chen","year":"2018","venue":"Journal of Climate","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen and Lin, 2012] Yao-Nan Chen and Hsuan-Tien Lin. Feature-aware label space dimension reduction for multi- label classification. In Advances in Neural Information Processing Systems, pages 1529-1537, 2012.","title":"Feature-aware label space dimension reduction for multilabel classification","authors":"Chen, Lin ; ] Yao-Nan Chen, Hsuan-Tien Lin","year":"2012","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen et al., 2017] Di Chen, Yexiang Xue, Daniel Fink, Shuo Chen, and Carla P Gomes. Deep multi-species em- bedding. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, pages 3639-3646, 2017.","title":"Deep multi-species embedding","authors":"Di Chen, Yexiang Xue, Daniel Fink, Shuo Chen, Carla P Gomes","year":"2017","venue":"Proceedings of the 26th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/509"},{"position":9,"raw":"[Chen et al., 2018] Di Chen, Yexiang Xue, and Carla Gomes. End-to-end learning for the deep multivariate pro- bit model. In International Conference on Machine Learn- ing, pages 932-941, 2018.","title":"End-to-end learning for the deep multivariate probit model","authors":"Di Chen, Yexiang Xue, Carla Gomes","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Chen et al., 2019a] Chen Chen, Haobo Wang, Weiwei Liu, Xingyuan Zhao, Tianlei Hu, and Gang Chen. Two- stage label embedding via neural factorization machine for multi-label classification. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 33, pages 3304- 3311, 2019.","title":"Twostage label embedding via neural factorization machine for multi-label classification","authors":"Chen","year":"2019","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Chen et al., 2019b] Zhao-Min Chen, Xiu-Shen Wei, Peng Wang, and Yanwen Guo. Multi-label image recognition with graph convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition, pages 5177-5186, 2019.","title":"Multi-label image recognition with graph convolutional networks","authors":"Chen","year":"2019","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Chiang et al., 2012] Tsung-Hsien Chiang, Hung-Yi Lo, and Shou-De Lin. A ranking-based knn approach for multi- label classification. In Asian Conference on Machine Learning, pages 81-96, 2012.","title":"A ranking-based knn approach for multilabel classification","authors":"Tsung-Hsien Chiang, Hung-Yi Lo, Shou-De Lin","year":"2012","venue":"Asian Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Chu et al., 2018] Hong-Min Chu, Chih-Kuan Yeh, and Yu- Chiang Frank Wang. Deep generative models for weakly- supervised multi-label classification. In Proceedings of the European Conference on Computer Vision, 2018.","title":"Deep generative models for weaklysupervised multi-label classification","authors":"Chu","year":"2018","venue":"Proceedings of the European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Chua et al., 2009] Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. Nus- wide: a real-world web image database from national uni- versity of singapore. In Proceedings of the ACM interna- tional conference on image and video retrieval, pages 1-9, 2009.","title":"Nuswide: a real-world web image database from national university of singapore","authors":"Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, Yantao Zheng","year":"2009","venue":"Proceedings of the ACM international conference on image and video retrieval","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Chung et al., 2015] Junyoung Chung, Kyle Kastner, Lau- rent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Bengio. A recurrent latent variable model for sequential data. In Advances in neural information processing sys- tems, pages 2980-2988, 2015.","title":"A recurrent latent variable model for sequential data","authors":"Chung","year":"2015","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Evans et al., 2017] Daniel M Evans, Judy P Che-Castaldo, Deborah Crouse, Frank W Davis, Rebecca Epanchin- Niell, Curtis H Flather, R Kipp Frohlich, Dale D Goble, Ya-Wei Li, and Timothy D Male. Species recovery in the united states: increasing the effectiveness of the endan- gered species act. Issues in Ecology, 2017.","title":"Species recovery in the united states: increasing the effectiveness of the endangered species act","authors":"Judy P Daniel M Evans, Deborah Che-Castaldo, Frank W Crouse, Rebecca Davis, Curtis H Epanchin-Niell, Kipp Flather, Dale D Frohlich, Ya-Wei Goble, Timothy D Li, Male","year":"2017","venue":"Issues in Ecology","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Gomes et al., 2019] Carla Gomes, Thomas Dietterich, Christopher Barrett, Jon Conrad, Bistra Dilkina, Stefano Ermon, Fei Fang, Andrew Farnsworth, Alan Fern, Xiaoli Fern, et al. Computational sustainability: Computing for a better world and a sustainable future. Communications of the ACM, 62(9):56-65, 2019.","title":"Computational sustainability: Computing for a better world and a sustainable future","authors":"Carla Gomes, Thomas Dietterich, Christopher Barrett, Jon Conrad, Bistra Dilkina, Stefano Ermon, Fei Fang, Andrew Farnsworth, Alan Fern, Xiaoli Fern","year":"2019","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Higgins et al., 2017] Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. β-vae: Learn- ing basic visual concepts with a constrained variational framework. The Eighth International Conference on Learning Representations, 2(5):6, 2017.","title":"β-vae: Learning basic visual concepts with a constrained variational framework","authors":"Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, Alexander Lerchner","year":"2017","venue":"The Eighth International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Huiskes and Lew, 2008] Mark J. Huiskes and Michael S. Lew. The mir flickr retrieval evaluation. In MIR '08: Pro- ceedings of the 2008 ACM International Conference on Multimedia Information Retrieval, New York, NY, USA, 2008. ACM.","title":"The mir flickr retrieval evaluation","authors":"Mark J Huiskes, Michael S Lew","year":"2008","venue":"MIR '08: Proceedings of the 2008 ACM International Conference on Multimedia Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Katakis et al., 2008] Ioannis Katakis, Grigorios Tsoumakas, and Ioannis Vlahavas. Multilabel text classification for automated tag suggestion. Discovery Challenge in Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECML-PKDD), page 75, 2008.","title":"Multilabel text classification for automated tag suggestion","authors":"Ioannis Katakis, Grigorios Tsoumakas, Ioannis Vlahavas","year":"2008","venue":"Discovery Challenge in Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECML-PKDD)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kingma and Ba, 2015] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. Interna- tional conference on learning representation, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"International conference on learning representation","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Kuhn et al., 2015] Michael Kuhn, Ivica Letunic, Lars Juhl Jensen, and Peer Bork. The sider database of drugs and side effects. Nucleic acids research, 44(D1):D1075- D1079, 2015.","title":"The sider database of drugs and side effects","authors":"Michael Kuhn, Ivica Letunic, Lars Juhl Jensen, Peer Bork","year":"2015","venue":"Nucleic acids research","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Kuhn et al., 2016] Michael Kuhn, Ivica Letunic, Lars Juhl Jensen, and Peer Bork. The sider database of drugs and side effects. Nucleic acids research, 44(D1):D1075- D1079, 2016.","title":"The sider database of drugs and side effects","authors":"Michael Kuhn, Ivica Letunic, Lars Juhl Jensen, Peer Bork","year":"2016","venue":"Nucleic acids research","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Lanchantin et al., 2019] Jack Lanchantin, Arshdeep Sekhon, and Yanjun Qi. Neural message passing for multi-label classification. Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 2019.","title":"Neural message passing for multi-label classification","authors":"Lanchantin","year":"2019","venue":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Morley et al., 2018] James W. Morley, Rebecca L. Selden, Robert J. Latour, Thomas L. Frölicher, Richard J. Sea- graves, and Malin L. Pinsky. Projecting shifts in thermal habitat for 686 species on the north american continental shelf. PLOS ONE, 13(5):1-28, 05 2018.","title":"Projecting shifts in thermal habitat for 686 species on the north american continental shelf","authors":"James W Morley, Rebecca L Selden, Robert J Latour, Thomas L Frölicher, Richard J Seagraves, Malin L Pinsky","year":"2018","venue":"PLOS ONE","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Munson et al., 2011] M Arthur Munson, Kevin Webb, Daniel Sheldon, Daniel Fink, Wesley M Hochachka, Mar- shall Iliff, Mirek Riedewald, Daria Sorokina, Brian Sulli- van, Christopher Wood, et al. The ebird reference dataset. Cornell Lab of Ornithology and National Audubon Soci- ety, 2011.","title":"The ebird reference dataset","authors":"Kevin Arthur Munson, Daniel Webb, Daniel Sheldon, Wesley M Fink, Marshall Hochachka, Mirek Iliff, Daria Riedewald, Brian Sorokina, Christopher Sullivan, Wood","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Nakai and Kanehisa, 1992] Kenta Nakai and Minoru Kane- hisa. A knowledge base for predicting protein localization sites in eukaryotic cells. Genomics, 14(4):897-911, 1992.","title":"A knowledge base for predicting protein localization sites in eukaryotic cells","authors":"Nakai, Kanehisa ;, Minoru Kanehisa","year":"1992","venue":"Genomics","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Read et al., 2009] Jesse Read, Bernhard Pfahringer, Geoff Holmes, and Eibe Frank. Classifier chains for multi-label classification. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 254-269. Springer, 2009.","title":"Classifier chains for multi-label classification","authors":"Read","year":"2009","venue":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Tsoumakas et al., 2008] Grigorios Tsoumakas, Ioannis Katakis, and Ioannis Vlahavas. Effective and efficient multilabel classification in domains with large number of labels. In Proceedings of ECML-PKDD 2008 Workshop on Mining Multidimensional Data, pages 53-59, 2008.","title":"Effective and efficient multilabel classification in domains with large number of labels","authors":"Grigorios Tsoumakas, Ioannis Katakis, Ioannis Vlahavas","year":"2008","venue":"Proceedings of ECML-PKDD 2008 Workshop on Mining Multidimensional Data","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[van Steenkiste et al., 2019] Sjoerd van Steenkiste, Francesco Locatello, Jürgen Schmidhuber, and Olivier Bachem. Are disentangled representations helpful for abstract visual reasoning? In Advances in Neural Information Processing Systems, pages 14222-14235, 2019.","title":"Are disentangled representations helpful for abstract visual reasoning?","authors":"Francesco Sjoerd Van Steenkiste, Jürgen Locatello, Olivier Schmidhuber, Bachem","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2016] Jiang Wang, Yi Yang, Junhua Mao, Zhi- heng Huang, Chang Huang, and Wei Xu. Cnn-rnn: A unified framework for multi-label image classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2285-2294, 2016.","title":"Cnn-rnn: A unified framework for multi-label image classification","authors":"Jiang Wang, Yi Yang, Junhua Mao, Zhiheng Huang, Chang Huang, Wei Xu","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wu et al., 2018] Baoyuan Wu, Fan Jia, Wei Liu, Bernard Ghanem, and Siwei Lyu. Multi-label learning with miss- ing labels using mixed dependency graphs. International Journal of Computer Vision, 126(8):875-896, 2018.","title":"Multi-label learning with missing labels using mixed dependency graphs","authors":"Baoyuan Wu, Fan Jia, Wei Liu, Bernard Ghanem, Siwei Lyu","year":"2018","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Yeh et al., 2017] Chih-Kuan Yeh, Wei-Chieh Wu, Wei-Jen Ko, and Yu-Chiang Frank Wang. Learning deep latent space for multi-label classification. In Thirty-First AAAI Conference on Artificial Intelligence, 2017.","title":"Learning deep latent space for multi-label classification","authors":"Chih-Kuan Yeh, Wei-Chieh Wu, Wei-Jen Ko, Yu-Chiang Frank, Wang","year":"2017","venue":"Thirty-First AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yu et al., 2014] Hsiang-Fu Yu, Prateek Jain, Purushottam Kar, and Inderjit Dhillon. Large-scale multi-label learn- ing with missing labels. In International conference on machine learning, pages 593-601, 2014.","title":"Large-scale multi-label learning with missing labels","authors":"Hsiang-Fu Yu, Prateek Jain, Purushottam Kar, Inderjit Dhillon","year":"2014","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhang and Yeung, 2013] Yu Zhang and Dit-Yan Yeung. Multilabel relationship learning. ACM Transactions on Knowledge Discovery from Data (TKDD), 7(2):7, 2013.","title":"Multilabel relationship learning","authors":"Yeung ; ] Yu Zhang, Dit-Yan Zhang, Yeung","year":"2013","venue":"ACM Transactions on Knowledge Discovery from Data (TKDD)","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhang and Zhou, 2007] Min-Ling Zhang and Zhi-Hua Zhou. Ml-knn: A lazy learning approach to multi-label learning. Pattern recognition, 40(7):2038-2048, 2007.","title":"Ml-knn: A lazy learning approach to multi-label learning","authors":"Zhou ; ] Min-Ling Zhang, Zhi-Hua Zhang, Zhou","year":"2007","venue":"Pattern recognition","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhang and Zhou, 2013] Min-Ling Zhang and Zhi-Hua Zhou. A review on multi-label learning algorithms. IEEE transactions on knowledge and data engineering, 26(8):1819-1837, 2013.","title":"A review on multi-label learning algorithms","authors":"Zhou ; ] Min-Ling Zhang, Zhi-Hua Zhang, Zhou","year":"2013","venue":"IEEE transactions on knowledge and data engineering","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zhang et al., 2018] Min-Ling Zhang, Yu-Kun Li, Xu-Ying Liu, and Xin Geng. Binary relevance for multi-label learning: an overview. Frontiers of Computer Science, 12(2):191-202, 2018.","title":"Binary relevance for multi-label learning: an overview","authors":"Zhang","year":"2018","venue":"Frontiers of Computer Science","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}