{"type":"ScholarlyArticle","title":"Zero-Shot Logit Adjustment","authors":[{"name":"Dubing Chen","slug":"dubing-chen","given":"Dubing","family":"Chen","affiliation":"Nanjing University of Science and Technology"},{"name":"Yuming Shen","slug":"yuming-shen","given":"Yuming","family":"Shen","affiliation":"University of Oxford"},{"name":"Haofeng Zhang","slug":"haofeng-zhang","given":"Haofeng","family":"Zhang","affiliation":"Nanjing University of Science and Technology"},{"name":"Philip H.S. Torr","slug":"philip-h-s-torr","given":"Philip H.S.","family":"Torr","affiliation":"University of Oxford"}],"abstract":"Semantic-descriptor-based Generalized Zero-Shot Learning (GZSL) poses challenges in recognizing novel classes in the test phase. The development of generative models enables current GZSL techniques to probe further into the semantic-visual link, culminating in a two-stage form that includes a generator and a classifier. However, existing generation-based methods focus on enhancing the generator's effect while neglecting the improvement of the classifier. In this paper, we first analyze of two properties of the generated pseudo unseen samples: bias and homogeneity. Then, we perform variational Bayesian inference to back-derive the evaluation metrics, which reflects the balance of the seen and unseen classes. As a consequence of our derivation, the aforementioned two properties are incorporated into the classifier training as seen-unseen priors via logit adjustment. The Zero-Shot Logit Adjustment further puts semantic-based classifiers into effect in generation-based GZSL. Our experiments demonstrate that the proposed technique achieves state-of-the-art when combined with the basic generator, and it can improve various generative Zero-Shot Learning frameworks. Our codes are available on https://github.com/cdb342/IJCAI-2022-ZLA.","doi":"10.24963/ijcai.2022/114","url":"https://ijcai.org/proceedings/2022/114","pdf":"https://ijcai.org/proceedings/2022/0114.pdf","markdown":"https://ijcai.org/proceedings/2022/114.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/114","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":114,"pages":{"first":813,"last":819},"keywords":["Computer Vision: Recognition (object detection, categorization)","Computer Vision: Vision and language","Computer Vision: Transfer, low-shot, semi- and un- supervised learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986510","length_sec":59,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984653","length_sec":314,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Buda et al., 2018] Mateusz Buda, Atsuto Maki, and Maciej A Mazurowski. A systematic study of the class imbalance problem in convolutional neural networks. Neural Networks, 106:249- 259, 2018.","title":"A systematic study of the class imbalance problem in convolutional neural networks","authors":"Mateusz Buda, Atsuto Maki, Maciej A Mazurowski","year":"2018","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2019] Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with label-distribution-aware margin loss. NeurIPS, 2019.","title":"Learning imbalanced datasets with label-distribution-aware margin loss","authors":"Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, Tengyu Ma","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chao et al., 2016] Wei-Lun Chao, Soravit Changpinyo, Boqing Gong, and Fei Sha. An empirical study and analysis of gener- alized zero-shot learning for object recognition in the wild. In ECCV, pages 52-68, 2016.","title":"An empirical study and analysis of generalized zero-shot learning for object recognition in the wild","authors":"Wei-Lun Chao, Soravit Changpinyo, Boqing Gong, Fei Sha","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chawla et al., 2002] Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. JAIR, 16:321-357, 2002.","title":"Smote: synthetic minority over-sampling technique","authors":"Kevin W Nitesh V Chawla, Lawrence O Bowyer, Philip Hall, Kegelmeyer","year":"2002","venue":"JAIR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2021a] Shiming Chen, Wenjie Wang, Beihao Xia, Qinmu Peng, Xinge You, Feng Zheng, and Ling Shao. Free: Fea- ture refinement for generalized zero-shot learning. ICCV, 2021.","title":"Free: Feature refinement for generalized zero-shot learning","authors":"Chen","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2021b] Zhi Chen, Yadan Luo, Ruihong Qiu, Zi Huang, Jingjing Li, and Zheng Zhang. Semantics disentangling for gen- eralized zero-shot learning. In ICCV, 2021.","title":"Semantics disentangling for generalized zero-shot learning","authors":"Chen","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chou et al., 2021] Yu-Ying Chou, Hsuan-Tien Lin, and Tyng-Luh Liu. Adaptive and generative zero-shot learning. In ICLR, 2021.","title":"Adaptive and generative zero-shot learning","authors":"Yu-Ying Chou, Hsuan-Tien Lin, Tyng-Luh Liu","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Collell et al., 2016] Guillem Collell, Drazen Prelec, and Kaustubh Patil. Reviving threshold-moving: a simple plug-in bagging en- semble for binary and multiclass imbalanced data. CoRR, 2016.","title":"Reviving threshold-moving: a simple plug-in bagging ensemble for binary and multiclass imbalanced data","authors":"Guillem Collell, Drazen Prelec, Kaustubh Patil","year":"2016","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Farhadi et al., 2009] Ali Farhadi, Ian Endres, Derek Hoiem, and David Forsyth. Describing objects by their attributes. In CVPR, pages 1778-1785, 2009.","title":"Describing objects by their attributes","authors":"Ali Farhadi, Ian Endres, Derek Hoiem, David Forsyth","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fu et al., 2014] Yanwei Fu, Timothy M Hospedales, Tao Xiang, Zhenyong Fu, and Shaogang Gong. Transductive multi-view embedding for zero-shot recognition and annotation. In ECCV, pages 584-599. Springer, 2014.","title":"Transductive multi-view embedding for zero-shot recognition and annotation","authors":"Yanwei Fu, Timothy M Hospedales, Tao Xiang, Zhenyong Fu, Shaogang Gong","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. NeurIPS, 27, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gulrajani et al., 2017] Ishaan Gulrajani, Faruk Ahmed, Martin Ar- jovsky, Vincent Dumoulin, and Aaron Courville. Improved train- ing of wasserstein gans. arXiv preprint arXiv:1704.00028, 2017.","title":"Improved training of wasserstein gans","authors":"Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Aaron Vincent Dumoulin, Courville","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Han et al., 2021] Zongyan Han, Zhenyong Fu, Shuo Chen, and Jian Yang. Contrastive embedding for generalized zero-shot learning. In CVPR, pages 2371-2381, 2021.","title":"Contrastive embedding for generalized zero-shot learning","authors":"Han","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hinton et al., 2015] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. NeurIPS, 2015.","title":"Distilling the knowledge in a neural network","authors":"Geoffrey Hinton, Oriol Vinyals, Jeff Dean","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kingma and Welling, 2013] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. ICLR, 2013.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2013","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lampert et al., 2009] Christoph H Lampert, Hannes Nickisch, and Stefan Harmeling. Learning to detect unseen object classes by between-class attribute transfer. In CVPR, pages 951-958, 2009.","title":"Learning to detect unseen object classes by between-class attribute transfer","authors":"Hannes Christoph H Lampert, Stefan Nickisch, Harmeling","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lampert et al., 2013] Christoph H Lampert, Hannes Nickisch, and Stefan Harmeling. Attribute-based classification for zero-shot vi- sual object categorization. IEEE TPAMI, 36(3):453-465, 2013.","title":"Attribute-based classification for zero-shot visual object categorization","authors":"Hannes Christoph H Lampert, Stefan Nickisch, Harmeling","year":"2013","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2019] Kai Li, Martin Renqiang Min, and Yun Fu. Re- thinking zero-shot learning: A conditional visual classification perspective. In ICCV, pages 3583-3592, 2019.","title":"Rethinking zero-shot learning: A conditional visual classification perspective","authors":"Li","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Li et al., 2021] Xiangyu Li, Zhe Xu, Kun Wei, and Cheng Deng. Generalized zero-shot learning via disentangled representation. In AAAI, volume 35, pages 1966-1974, 2021.","title":"Generalized zero-shot learning via disentangled representation","authors":"Xiangyu Li, Zhe Xu, Kun Wei, Cheng Deng","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lin et al., 2017] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaim- ing He, and Piotr Dollár. Focal loss for dense object detection. In ICCV, pages 2980-2988, 2017.","title":"Focal loss for dense object detection","authors":"Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu et al., 2018] Shichen Liu, Mingsheng Long, Jianmin Wang, and Michael I Jordan. Generalized zero-shot learning with deep calibration network. In NeurIPS, pages 2005-2015, 2018.","title":"Generalized zero-shot learning with deep calibration network","authors":"Shichen Liu, Mingsheng Long, Jianmin Wang, Michael I Jordan","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Menon et al., 2013] Aditya Menon, Harikrishna Narasimhan, Shivani Agarwal, and Sanjay Chawla. On the statistical consis- tency of algorithms for binary classification under class imbal- ance. In ICML, pages 603-611. PMLR, 2013.","title":"On the statistical consistency of algorithms for binary classification under class imbalance","authors":"Aditya Menon, Harikrishna Narasimhan, Shivani Agarwal, Sanjay Chawla","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Menon et al., 2020] Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar. Long-tail learning via logit adjustment. arXiv preprint arXiv:2007.07314, 2020.","title":"Long-tail learning via logit adjustment","authors":"Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, Sanjiv Kumar","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Min et al., 2020] Shaobo Min, Hantao Yao, Hongtao Xie, Chao- qun Wang, Zheng-Jun Zha, and Yongdong Zhang. Domain-aware visual bias eliminating for generalized zero-shot learning. In CVPR, pages 12664-12673, 2020.","title":"Domain-aware visual bias eliminating for generalized zero-shot learning","authors":"Hantao Shaobo Min, Hongtao Yao, Chaoqun Xie, Zheng-Jun Wang, Yongdong Zha, Zhang","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Patterson and Hays, 2012] Genevieve Patterson and James Hays. Sun attribute database: Discovering, annotating, and recognizing scene attributes. In CVPR, pages 2751-2758, 2012.","title":"Sun attribute database: Discovering, annotating, and recognizing scene attributes","authors":"Genevieve Patterson, James Hays","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Reed et al., 2016] Scott Reed, Zeynep Akata, Honglak Lee, and Bernt Schiele. Learning deep representations of fine-grained vi- sual descriptions. In CVPR, pages 49-58, 2016.","title":"Learning deep representations of fine-grained visual descriptions","authors":"Scott Reed, Zeynep Akata, Honglak Lee, Bernt Schiele","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Shen et al., 2020] Yuming Shen, Jie Qin, Lei Huang, Li Liu, Fan Zhu, and Ling Shao. Invertible zero-shot recognition flows. In ECCV, pages 614-631, 2020.","title":"Invertible zero-shot recognition flows","authors":"Yuming Shen, Jie Qin, Lei Huang, Li Liu, Fan Zhu, Ling Shao","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Skorokhodov and Elhoseiny, 2021] Ivan Skorokhodov and Mo- hamed Elhoseiny. Class normalization for (continual)? gener- alized zero-shot learning. ICLR, 2021.","title":"Ivan Skorokhodov and Mohamed Elhoseiny. Class normalization for (continual)? generalized zero-shot learning","authors":"Elhoseiny Skorokhodov","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Tan et al., 2020] Jingru Tan, Changbao Wang, Buyu Li, Quanquan Li, Wanli Ouyang, Changqing Yin, and Junjie Yan. Equalization loss for long-tailed object recognition. In CVPR, pages 11662- 11671, 2020.","title":"Equalization loss for long-tailed object recognition","authors":"Tan","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wah et al., 2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200- 2011 dataset. california institute of technology, 2011.","title":"The caltech-ucsd birds-200-2011 dataset","authors":"Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, Serge Belongie","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Xian et al., 2017] Yongqin Xian, Bernt Schiele, and Zeynep Akata. Zero-shot learning-the good, the bad and the ugly. In CVPR, pages 4582-4591, 2017.","title":"Zero-shot learning-the good, the bad and the ugly","authors":"Xian","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Xian et al., 2018] Yongqin Xian, Tobias Lorenz, Bernt Schiele, and Zeynep Akata. Feature generating networks for zero-shot learning. In CVPR, pages 5542-5551, 2018.","title":"Feature generating networks for zero-shot learning","authors":"Xian","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Xie et al., 2020] Guo-Sen Xie, Li Liu, Fan Zhu, Fang Zhao, Zheng Zhang, Yazhou Yao, Jie Qin, and Ling Shao. Region graph em- bedding network for zero-shot learning. In ECCV, pages 562- 580. Springer, 2020.","title":"Region graph embedding network for zero-shot learning","authors":"Guo-Sen Xie, Li Liu, Fan Zhu, Fang Zhao, Zheng Zhang, Yazhou Yao, Jie Qin, Ling Shao","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Xu et al., 2020] Wenjia Xu, Yongqin Xian, Jiuniu Wang, Bernt Schiele, and Zeynep Akata. Attribute prototype network for zero- shot learning. NeurIPS, 2020.","title":"Attribute prototype network for zeroshot learning","authors":"Xu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Yue et al., 2021] Zhongqi Yue, Tan Wang, Qianru Sun, Xian- Sheng Hua, and Hanwang Zhang. Counterfactual zero-shot and open-set visual recognition. In CVPR, pages 15404-15414, 2021.","title":"Counterfactual zero-shot and open-set visual recognition","authors":"Yue","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}