{"type":"ScholarlyArticle","title":"Semantic Compression Embedding for Generative Zero-Shot Learning","authors":[{"name":"Ziming Hong","slug":"ziming-hong","given":"Ziming","family":"Hong","affiliation":"Huazhong University of Science and Technology"},{"name":"Shiming Chen","slug":"shiming-chen","given":"Shiming","family":"Chen","affiliation":"Huazhong University of Science and Technology"},{"name":"Guo-Sen Xie","slug":"guo-sen-xie","given":"Guo-Sen","family":"Xie","affiliation":"Nanjing University of Science and Technology"},{"name":"Wenhan Yang","slug":"wenhan-yang","given":"Wenhan","family":"Yang","affiliation":"Nanyang Technological University"},{"name":"Jian Zhao","slug":"jian-zhao","given":"Jian","family":"Zhao","affiliation":"Institute of North Electronic Equipment"},{"name":"Yuanjie Shao","slug":"yuanjie-shao","given":"Yuanjie","family":"Shao","affiliation":"Huazhong University of Science and Technology"},{"name":"Qinmu Peng","slug":"qinmu-peng","given":"Qinmu","family":"Peng","affiliation":"Huazhong University of Science and Technology"},{"name":"Xinge You","slug":"xinge-you","given":"Xinge","family":"You","affiliation":"Huazhong University of Science and Technology"}],"abstract":"Generative methods have been successfully applied in zero-shot learning (ZSL) by learning an implicit mapping to alleviate the visual-semantic domain gaps and synthesizing unseen samples to handle the data imbalance between seen and unseen classes. However, existing generative methods simply use visual features extracted by the pre-trained CNN backbone. These visual features lack attribute-level semantic information. Consequently, seen classes are indistinguishable, and the knowledge transfer from seen to unseen classes is limited. To tackle this issue, we propose a novel Semantic Compression Embedding Guided Generation (SC-EGG) model, which cascades a semantic compression embedding network (SCEN) and an embedding guided generative network (EGGN). The SCEN extracts a group of attribute-level local features for each sample and further compresses them into the new low-dimension visual feature. Thus, a dense-semantic visual space is obtained. The EGGN learns a mapping from the class-level semantic space to the dense-semantic visual space, thus improving the discriminability of the synthesized dense-semantic unseen visual features. Extensive experiments on three benchmark datasets, i.e., CUB, SUN and AWA2, demonstrate the significant performance gains of SC-EGG over current state-of-the-art methods and its baselines.","doi":"10.24963/ijcai.2022/134","url":"https://ijcai.org/proceedings/2022/134","pdf":"https://ijcai.org/proceedings/2022/0134.pdf","markdown":"https://ijcai.org/proceedings/2022/134.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/134","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":134,"pages":{"first":956,"last":963},"keywords":["Computer Vision: Recognition (object detection, categorization)","Computer Vision: Transfer, low-shot, semi- and un- supervised learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986550","length_sec":79,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984693","length_sec":319,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Akata et al., 2016] Zeynep Akata, F. Perronnin, Z. Har- chaoui, and C. Schmid. Label-embedding for image clas- sification. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 38:1425-1438, 2016.","title":"Label-embedding for image classification","authors":"F Zeynep Akata, Z Perronnin, C Harchaoui, Schmid","year":"2016","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2021a] Shiming Chen, Wenjie Wang, Beihao Xia, Qinmu Peng, Xinge You, Feng Zheng, and Ling Shao. Free: Feature refinement for generalized zero-shot learn- ing. In ICCV, pages 122-131, 2021.","title":"Free: Feature refinement for generalized zero-shot learning","authors":"Chen","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2021b] Shiming Chen, Guo-Sen Xie, Qinmu Peng, Yang Liu, Baigui Sun, Hao Li, Xinge You, and Ling Shao. Hsva: Hierarchical semantic-visual adaptation for zero-shot learning. In NeurIPS, 2021.","title":"Hsva: Hierarchical semantic-visual adaptation for zero-shot learning","authors":"Chen","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2021c] Zhi Chen, Yadan Luo, Ruihong Qiu, Sen Wang, Zi-Yu Huang, Jingjing Li, and Zheng Zhang. Semantics disentangling for generalized zero-shot learn- ing. In ICCV, 2021.","title":"Semantics disentangling for generalized zero-shot learning","authors":"Chen","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2022a] Shiming Chen, Ziming Hong, Guo-Sen Xie, Qinmu Peng, Xinge You, Weiping Ding, and Ling Shao. Gndan: Graph navigated dual attention network for zero-shot learning. IEEE Transactions on Neural Net- works and Learning Systems, 2022.","title":"Gndan: Graph navigated dual attention network for zero-shot learning","authors":"Chen","year":"2022","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2022b] Shiming Chen, Ziming Hong, Guo- Sen Xie, Wenhan Wang, Qinmu Peng, Kai Wang, Jian Zhao, and Xinge You. Msdn: Mutually semantic dis- tillation network for zero-shot learning. arXiv preprint arXiv:2203.03137, 2022.","title":"Msdn: Mutually semantic distillation network for zero-shot learning","authors":"Chen","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cheng et al., 2017] Yu Cheng, Jian Zhao, Zhecan Wang, Yan Xu, Karlekar Jayashree, Shengmei Shen, and Jiashi Feng. Know you at one glance: A compact vector repre- sentation for low-shot learning. In ICCV Workshops, pages 1924-1932, 2017.","title":"Know you at one glance: A compact vector representation for low-shot learning","authors":"Yu Cheng, Jian Zhao, Zhecan Wang, Yan Xu, Karlekar Jayashree, Shengmei Shen, Jiashi Feng","year":"2017","venue":"ICCV Workshops","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Felix et al., 2018] Rafael Felix, Ian Reid, Gustavo Carneiro, et al. Multi-modal cycle-consistent generalized zero-shot learning. In ECCV, pages 21-37, 2018.","title":"Multi-modal cycle-consistent generalized zero-shot learning","authors":"Rafael Felix, Ian Reid, Gustavo Carneiro","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Han et al., 2021] Zongyan Han, Zhenyong Fu, Shuo Chen, and Jian Yang. Contrastive embedding for generalized zero-shot learning. In CVPR, pages 2371-2381, 2021.","title":"Contrastive embedding for generalized zero-shot learning","authors":"Han","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2016] Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"X Kaiming He, Shaoqing Zhang, Jian Ren, Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Huynh and Elhamifar, 2020a] Dat Huynh and Ehsan El- hamifar. Compositional zero-shot learning via fine- grained dense feature composition. NeurIPS, 2020.","title":"Dat Huynh and Ehsan Elhamifar. Compositional zero-shot learning via finegrained dense feature composition","authors":"Elhamifar Huynh","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huynh and Elhamifar, 2020b] Dat Huynh and Ehsan El- hamifar. Fine-grained generalized zero-shot learning via dense attribute-based attention. In CVPR, pages 4483- 4493, 2020.","title":"Dat Huynh and Ehsan Elhamifar. Fine-grained generalized zero-shot learning via dense attribute-based attention","authors":"Elhamifar Huynh","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jiang et al., 2019] Huajie Jiang, Ruiping Wang, Shiguang Shan, and Xilin Chen. Transferable contrastive network for generalized zero-shot learning. In ICCV, pages 9765- 9774, 2019.","title":"Transferable contrastive network for generalized zero-shot learning","authors":"Huajie Jiang, Ruiping Wang, Shiguang Shan, Xilin Chen","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Keshari et al., 2020] Rohit Keshari, Richa Singh, and Mayank Vatsa. Generalized zero-shot learning via over- complete distribution. In CVPR, pages 13300-13308, 2020.","title":"Generalized zero-shot learning via overcomplete distribution","authors":"Rohit Keshari, Richa Singh, Mayank Vatsa","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lampert et al., 2014] Christoph H. Lampert, H. Nickisch, and S. Harmeling. Attribute-based classification for zero- shot visual object categorization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36:453-465, 2014.","title":"Attribute-based classification for zeroshot visual object categorization","authors":"Christoph H Lampert, H Nickisch, S Harmeling","year":"2014","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2019] Jingjing Li, Mengmeng Jing, Ke Lu, Zheng- ming Ding, Lei Zhu, and Zi Huang. Leveraging the invari- ant side of generative zero-shot learning. In CVPR, pages 7402-7411, 2019.","title":"Leveraging the invariant side of generative zero-shot learning","authors":"Jingjing Li, Mengmeng Jing, Ke Lu, Zhengming Ding, Lei Zhu, Zi Huang","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Narayan et al., 2020] Sanath Narayan, Akshita Gupta, Fa- had Shahbaz Khan, Cees GM Snoek, and Ling Shao. La- tent embedding feedback and discriminative features for zero-shot classification. In ECCV, pages 479-495, 2020.","title":"Latent embedding feedback and discriminative features for zero-shot classification","authors":"Sanath Narayan, Akshita Gupta, Fahad Shahbaz Khan, G M Cees, Ling Snoek, Shao","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Palatucci et al., 2009] Mark Palatucci, D. Pomerleau, Ge- offrey E. Hinton, and Tom Michael Mitchell. Zero-shot learning with semantic output codes. In NeurIPS, pages 1410-1418, 2009.","title":"Zero-shot learning with semantic output codes","authors":"D Mark Palatucci, Geoffrey E Pomerleau, Tom Hinton, Mitchell Michael","year":"2009","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Patterson and Hays, 2012] Genevieve Patterson and James Hays. Sun attribute database: Discovering, annotating, and recognizing scene attributes. In CVPR, pages 2751-2758, 2012.","title":"Sun attribute database: Discovering, annotating, and recognizing scene attributes","authors":"Genevieve Patterson, James Hays","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In EMNLP, pages 1532-1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher D Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Scholkopf and Smola, 2002] B. Scholkopf and A. Smola. Learning with kernels. MIT Press, 2002.","title":"Learning with kernels","authors":"B Scholkopf, A Smola","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sylvain et al., 2020] Tristan Sylvain, Linda Petrini, and R. Devon Hjelm. Locality and compositionality in zero- shot learning. In ICLR, 2020.","title":"Locality and compositionality in zeroshot learning","authors":"Tristan Sylvain, Linda Petrini, R, Devon Hjelm","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Van der Maaten and Hinton, 2008] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.","title":"Visualizing data using t-sne","authors":"Van Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Welinder et al., 2010] Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, Florian Schroff, Serge Belongie, and Pietro Perona. Caltech-ucsd birds 200. California Institute of Technology, 2010.","title":"Caltech-ucsd birds 200","authors":"Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, Florian Schroff, Serge Belongie, Pietro Perona","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xian et al., 2017] Yongqin Xian, Bernt Schiele, and Zeynep Akata. Zero-shot learning-the good, the bad and the ugly. In CVPR, pages 4582-4591, 2017.","title":"Zero-shot learning-the good, the bad and the ugly","authors":"Xian","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xian et al., 2018] Yongqin Xian, Tobias Lorenz, Bernt Schiele, and Zeynep Akata. Feature generating networks for zero-shot learning. In CVPR, pages 5542-5551, 2018.","title":"Feature generating networks for zero-shot learning","authors":"Xian","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xian et al., 2019] Yongqin Xian, Saurabh Sharma, Bernt Schiele, and Zeynep Akata. f-vaegan-d2: A feature gen- erating framework for any-shot learning. In CVPR, pages 10275-10284, 2019.","title":"f-vaegan-d2: A feature generating framework for any-shot learning","authors":"Xian","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xie et al., 2019] Guo-Sen Xie, L. Liu, Xiaobo Jin, F. Zhu, Zheng Zhang, J. Qin, Yazhou Yao, and L. Shao. Atten- tive region embedding network for zero-shot learning. In CVPR, pages 9376-9385, 2019.","title":"Attentive region embedding network for zero-shot learning","authors":"Guo-Sen Xie, L Liu, Xiaobo Jin, F Zhu, Zheng Zhang, J Qin, Yazhou Yao, L Shao","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Xie et al., 2020] Guo-Sen Xie, Li Liu, Fan Zhu, Fang Zhao, Zheng Zhang, Yazhou Yao, Jie Qin, and Ling Shao. Re- gion graph embedding network for zero-shot learning. In ECCV, pages 562-580, 2020.","title":"Region graph embedding network for zero-shot learning","authors":"Guo-Sen Xie, Li Liu, Fan Zhu, Fang Zhao, Zheng Zhang, Yazhou Yao, Jie Qin, Ling Shao","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xu et al., 2020] Wenjia Xu, Yongqin Xian, Jiuniu Wang, Bernt Schiele, and Zeynep Akata. Attribute prototype net- work for zero-shot learning. In NeurIPS, 2020.","title":"Attribute prototype network for zero-shot learning","authors":"Xu","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yan et al., 2021] Caixia Yan, Xiaojun Chang, Zhihui Li, ZongYuan Ge, Weili Guan, Lei Zhu, and Qinghua Zheng. Zeronas: Differentiable generative adversarial networks search for zero-shot learning. IEEE transactions on pat- tern analysis and machine intelligence, 2021.","title":"Zeronas: Differentiable generative adversarial networks search for zero-shot learning","authors":"Yan","year":"2021","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yue et al., 2021] Zhongqi Yue, Tan Wang, Qianru Sun, Xian-Sheng Hua, and Hanwang Zhang. Counterfactual zero-shot and open-set visual recognition. In CVPR, pages 15404-15414, 2021.","title":"Counterfactual zero-shot and open-set visual recognition","authors":"Yue","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}