{"type":"ScholarlyArticle","title":"Zero-shot Learning with Many Classes by High-rank Deep Embedding Networks","authors":[{"name":"Yuchen Guo","slug":"yuchen-guo","given":"Yuchen","family":"Guo","affiliation":"Department of Automation , Tsinghua University , Beijing , China"},{"name":"Guiguang Ding","slug":"guiguang-ding","given":"Guiguang","family":"Ding","affiliation":"School of Software , Tsinghua University , Beijing , China"},{"name":"Jungong Han","slug":"jungong-han","given":"Jungong","family":"Han","affiliation":"WMG Data Science , University of Warwick , Coventry , UK"},{"name":"Hang Shao","slug":"hang-shao","given":"Hang","family":"Shao","affiliation":"Zhejiang Future Technology Institute (Jiaxing) , Zhejiang , China"},{"name":"Xin Lou","slug":"xin-lou","given":"Xin","family":"Lou","affiliation":"Chinese PLA General Hospital , Beijing , China"},{"name":"Qionghai Dai","slug":"qionghai-dai","given":"Qionghai","family":"Dai","affiliation":"Department of Automation , Tsinghua University , Beijing , China"}],"abstract":"Zero-shot learning (ZSL) is a recently emerging research topic which aims to build classification models for unseen classes with knowledge from auxiliary seen classes. Though many ZSL works have shown promising results on small-scale datasets by utilizing a bilinear compatibility function, the ZSL performance on large-scale datasets with many classes (say, ImageNet) is still unsatisfactory. We argue that the bilinear compatibility function is a low-rank approximation of the true compatibility function such that it is not expressive enough especially when there are a large number of classes because of the rank limitation. To address this issue, we propose a novel approach, termed as High-rank Deep Embedding Networks (GREEN), for ZSL with many classes. In particular, we propose a feature-dependent mixture of softmaxes as the image-class compatibility function, which is a simple extension of the bilinear compatibility function, but yields much better results. It utilizes a mixture of non-linear transformations with feature-dependent latent variables to approximate the true function in a high-rank way, which makes GREEN more expressive. Experiments on several datasets including ImageNet demonstrate GREEN significantly outperforms the state-of-the-art approaches.","doi":"10.24963/ijcai.2019/337","url":"https://ijcai.org/proceedings/2019/337","pdf":"https://ijcai.org/proceedings/2019/0337.pdf","markdown":"https://ijcai.org/proceedings/2019/337.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/337","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":337,"pages":{"first":2428,"last":2434},"keywords":["Machine Learning: Transfer, Adaptation, Multi-task Learning","Computer Vision: Language and Vision","Machine Learning: Classification"],"videos":[],"references":[{"position":1,"raw":"[Akata et al., 2015] Zeynep Akata, Scott E. Reed, Daniel Walter, Honglak Lee, and Bernt Schiele. Evaluation of output embed- dings for fine-grained image classification. In CVPR, 2015.","title":"Evaluation of output embeddings for fine-grained image classification","authors":"Zeynep Akata, Scott E Reed, Daniel Walter, Honglak Lee, Bernt Schiele","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Akata et al., 2016] Zeynep Akata, Florent Perronnin, Zaïd Har- chaoui, and Cordelia Schmid. Label-embedding for image clas- sification. IEEE Trans. Pattern Anal. Mach. Intell., 2016.","title":"Label-embedding for image classification","authors":"Zeynep Akata, Florent Perronnin, Zaïd Harchaoui, Cordelia Schmid","year":"2016","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bishop and others, 2006] Christopher M Bishop et al. Pattern recognition and machine learning, volume 1. Springer, New Y- ork, 2006.","title":"Pattern recognition and machine learning","authors":"M Christopher, Bishop","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bishop, 1998] Christopher M Bishop. Latent variable models. In Learning in graphical models, pages 371-403. Springer, 1998.","title":"Latent variable models","authors":"M Christopher, Bishop","year":"1998","venue":"Learning in graphical models","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Changpinyo et al., 2016] Soravit Changpinyo, Wei-Lun Chao, Bo- qing Gong, and Fei Sha. Synthesized classifiers for zero-shot learning. In 2016 IEEE Conference on Computer Vision and Pat- tern Recognition, pages 5327-5336, 2016.","title":"Synthesized classifiers for zero-shot learning","authors":"Changpinyo","year":"2016","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Farhadi et al., 2009] Ali Farhadi, Ian Endres, Derek Hoiem, and David A. Forsyth. Describing objects by their attributes. In 2009 IEEE Computer Society Conference on Computer Vision and Pat- tern Recognition, pages 1778-1785, 2009.","title":"Describing objects by their attributes","authors":"Ali Farhadi, Ian Endres, Derek Hoiem, David A Forsyth","year":"2009","venue":"IEEE Computer Society Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Frome et al., 2013] Andrea Frome, Gregory S. Corrado, Jonathon Shlens, Samy Bengio, Jeffrey Dean, Marc'Aurelio Ranzato, and Tomas Mikolov. Devise: A deep visual-semantic embedding model. In NIPS, 2013.","title":"Devise: A deep visual-semantic embedding model","authors":"Andrea Frome, Gregory S Corrado, Jonathon Shlens, Samy Bengio, Jeffrey Dean, Marc'aurelio Ranzato, Tomas Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fu et al., 2015a] Yanwei Fu, Timothy M. Hospedales, Tao Xiang, and Shaogang Gong. Transductive multi-view zero-shot learning. IEEE Trans. Pattern Anal. Mach. Intell., 2015.","title":"Transductive multi-view zero-shot learning","authors":"Fu","year":"2015","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fu et al., 2015b] Zhenyong Fu, Tao Xiang, Elyor Kodirov, and Shaogang Gong. Zero-shot object recognition by semantic man- ifold distance. In CVPR, 2015.","title":"Zero-shot object recognition by semantic manifold distance","authors":"Fu","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Guo et al., 2016] Yuchen Guo, Guiguang Ding, Xiaoming Jin, and Jianmin Wang. Transductive zero-shot recognition via shared model space learning. In AAAI, 2016.","title":"Transductive zero-shot recognition via shared model space learning","authors":"Yuchen Guo, Guiguang Ding, Xiaoming Jin, Jianmin Wang","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Guo et al., 2017a] Yuchen Guo, Guiguang Ding, Jungong Han, and Yue Gao. Synthesizing samples for zero-shot learning. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, pages 1774-1780, 2017.","title":"Synthesizing samples for zero-shot learning","authors":"Guo","year":"2017","venue":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/246"},{"position":12,"raw":"[Guo et al., 2017b] Yuchen Guo, Guiguang Ding, Jungong Han, and Yue Gao. Zero-shot learning with transferred samples. IEEE TIP, 26(7):3277-3290, 2017.","title":"Zero-shot learning with transferred samples","authors":"Guo","year":"2017","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kodirov et al., 2017] Elyor Kodirov, Tao Xiang, and Shaogang Gong. Semantic autoencoder for zero-shot learning. In CVPR, 2017.","title":"Semantic autoencoder for zero-shot learning","authors":"Kodirov","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lampert et al., 2014] Christoph H. Lampert, Hannes Nickisch, and Stefan Harmeling. Attribute-based classification for zero- shot visual object categorization. IEEE Trans. Pattern Anal. Mach. Intell., 36(3):453-465, 2014.","title":"Attribute-based classification for zeroshot visual object categorization","authors":"Christoph H Lampert, Hannes Nickisch, Stefan Harmeling","year":"2014","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mikolov et al., 2013a] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. ICLR Workshops, 2013.","title":"Efficient estimation of word representations in vector space","authors":"Mikolov","year":"2013","venue":"ICLR Workshops","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mikolov et al., 2013b] Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. Distributed represen- tations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems 26, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Norouzi et al., 2013] Mohammad Norouzi, Tomas Mikolov, Samy Bengio, Yoram Singer, Jonathon Shlens, Andrea Frome, Greg Corrado, and Jeffrey Dean. Zero-shot learning by convex combi- nation of semantic embeddings. CoRR, abs/1312.5650, 2013.","title":"Zero-shot learning by convex combination of semantic embeddings","authors":"Norouzi","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Patterson and Hays, 2012] Genevieve Patterson and James Hays. SUN attribute database: Discovering, annotating, and recogniz- ing scene attributes. In CVPR, 2012.","title":"SUN attribute database: Discovering, annotating, and recognizing scene attributes","authors":"Genevieve Patterson, James Hays","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Romera-Paredes and Torr, 2015] Bernardino Romera-Paredes and Philip H. S. Torr. An embarrassingly simple approach to zero- shot learning. In ICML, 2015.","title":"An embarrassingly simple approach to zeroshot learning","authors":"Romera-Paredes, Torr ; Bernardino Romera-Paredes, Philip H S Torr","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Russakovsky et al., 2015] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z.g Huang, A. Karpathy, A. Khosla, M. S. Bernstein, A. C. Berg, and Fei-Fei Li. Ima- genet large scale visual recognition challenge. IJCV, 2015.","title":"Imagenet large scale visual recognition challenge","authors":"O Russakovsky, J Deng, H Su, J Krause, S Satheesh, S Ma, Z Huang, A Karpathy, A Khosla, M S Bernstein, A C Berg, Fei-Fei Li","year":"2015","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Socher et al., 2013] Richard Socher, Milind Ganjoo, Christo- pher D. Manning, and Andrew Y. Ng. Zero-shot learning through cross-modal transfer. In NIPS, 2013.","title":"Zero-shot learning through cross-modal transfer","authors":"Richard Socher, Milind Ganjoo, Christopher D Manning, Andrew Y Ng","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wah et al., 2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The Caltech-UCSD Birds- 200-2011 Dataset. Technical Report CNS-TR-2011-001, Cali- fornia Institute of Technology, 2011.","title":"The Caltech-UCSD Birds-200-2011 Dataset","authors":"Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, Serge Belongie","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wu et al., 2018] Fan Wu, Kai Tian, Jihong Guan, and Shuigeng Zhou. Global semantic consistency for zero-shot learning. In ECCV, 2018.","title":"Global semantic consistency for zero-shot learning","authors":"Fan Wu, Kai Tian, Jihong Guan, Shuigeng Zhou","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xian et al., 2016] Yongqin Xian, Zeynep Akata, Gaurav Sharma, Quynh N. Nguyen, Matthias Hein, and Bernt Schiele. Latent embeddings for zero-shot classification. In CVPR, 2016.","title":"Latent embeddings for zero-shot classification","authors":"Xian","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xian et al., 2017] Yongqin Xian, Christoph H. Lampert, Bern- t Schiele, and Zeynep Akata. Zero-shot learning -A compre- hensive evaluation of the good, the bad and the ugly. CoRR, ab- s/1707.00600, 2017.","title":"Zero-shot learning -A comprehensive evaluation of the good, the bad and the ugly","authors":"Xian","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yang et al., 2017] Zhilin Yang, Zihang Dai, Ruslan Salakhutdinov, and William W. Cohen. Breaking the softmax bottleneck: A high- rank RNN language model. In ICLR, 2017.","title":"Breaking the softmax bottleneck: A highrank RNN language model","authors":"Zhilin Yang, Zihang Dai, Ruslan Salakhutdinov, William W Cohen","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yang et al., 2018] Zhilin Yang, Zihang Dai, Ruslan Salakhutdinov, and William W. Cohen. Breaking the softmax bottleneck: A high- rank RNN language model. In ICLR, 2018.","title":"Breaking the softmax bottleneck: A highrank RNN language model","authors":"Zhilin Yang, Zihang Dai, Ruslan Salakhutdinov, William W Cohen","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang and Saligrama, 2015] Ziming Zhang and Venkatesh Saligrama. Zero-shot learning via semantic similarity embed- ding. In ICCV, 2015.","title":"Zero-shot learning via semantic similarity embedding","authors":"Ziming Zhang, Venkatesh Saligrama","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang and Saligrama, 2016] Ziming Zhang and Venkatesh Saligrama. Zero-shot learning via joint latent similarity embedding. In CVPR, 2016.","title":"Zero-shot learning via joint latent similarity embedding","authors":"Ziming Zhang, Venkatesh Saligrama","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}