{"type":"ScholarlyArticle","title":"Boosting Multi-Label Image Classification with Complementary Parallel Self-Distillation","authors":[{"name":"Jiazhi Xu","slug":"jiazhi-xu","given":"Jiazhi","family":"Xu","affiliation":"School of Big Data and Software Engineering , Chongqing University"},{"name":"Sheng Huang","slug":"sheng-huang","given":"Sheng","family":"Huang","affiliation":"School of Big Data and Software Engineering , Chongqing University"},{"name":"Fengtao Zhou","slug":"fengtao-zhou","given":"Fengtao","family":"Zhou","affiliation":"School of Big Data and Software Engineering , Chongqing University"},{"name":"Luwen Huangfu","slug":"luwen-huangfu","given":"Luwen","family":"Huangfu","affiliation":"Fowler College of Business & CHDMA , San Diego State University ,"},{"name":"Daniel Zeng","slug":"daniel-zeng","given":"Daniel","family":"Zeng","affiliation":"Institute of Automation , Chinese Academy of Sciences ,"},{"name":"Bo Liu","slug":"bo-liu","given":"Bo","family":"Liu","affiliation":"JD Finance America Corporation"}],"abstract":"Multi-Label Image Classification (MLIC) appro-aches usually exploit label correlations to achieve good performance. However, emphasizing correlation like co-occurrence may overlook discriminative features and lead to model overfitting. In this study, we propose a generic framework named Parallel Self-Distillation (PSD) for boosting MLIC models. PSD decomposes the original MLIC task into several simpler MLIC sub-tasks via two elaborated complementary task decomposition strategies named Co-occurrence Graph Partition (CGP) and Dis-occurrence Graph Partition (DGP). Then, the MLIC models of fewer categories are trained with these sub-tasks in parallel for respectively learning the joint patterns and the category-specific patterns of labels. Finally, knowledge distillation is leveraged to learn a compact global ensemble of full categories with these learned patterns for reconciling the label correlation exploitation and model overfitting. Extensive results on MS-COCO and NUS-WIDE datasets demonstrate that our framework can be easily plugged into many MLIC approaches and improve performances of recent state-of-the-art approaches. The source code is released at https://github.com/Robbie-Xu/CPSD.","doi":"10.24963/ijcai.2022/208","url":"https://ijcai.org/proceedings/2022/208","pdf":"https://ijcai.org/proceedings/2022/0208.pdf","markdown":"https://ijcai.org/proceedings/2022/208.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/208","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":208,"pages":{"first":1495,"last":1501},"keywords":["Machine Learning: Multi-label","Computer Vision: Recognition (object detection, categorization)","Computer Vision: Machine Learning for Vision"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986546","length_sec":86,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984689","length_sec":339,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Chen et al., 2019a] Tianshui Chen, Muxin Xu, Xiaolu Hui, Hefeng Wu, and Liang Lin. Learning semantic-specific graph representa- tion for multi-label image recognition. In ICCV, pages 522-531, 2019.","title":"Learning semantic-specific graph representation for multi-label image recognition","authors":"Chen","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2019b] Zhao-Min Chen, Xiu-Shen Wei, Xin Jin, and Yanwen Guo. Multi-label image recognition with joint class- aware map disentangling and label correlation embedding. In IEEE ICME, pages 622-627. IEEE, 2019.","title":"Multi-label image recognition with joint classaware map disentangling and label correlation embedding","authors":"Chen","year":"2019","venue":"IEEE ICME","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2019c] Zhao-Min Chen, Xiu-Shen Wei, Peng Wang, and Yanwen Guo. Multi-label image recognition with graph con- volutional networks. In CVPR, pages 5177-5186, 2019.","title":"Multi-label image recognition with graph convolutional networks","authors":"Chen","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2021] Yawen Chen, Zeyi Wen, Bingsheng He, and Jian Chen. Efficient decomposition selection for multi-class clas- sification. IEEE TKDE, 2021.","title":"Efficient decomposition selection for multi-class classification","authors":"Yawen Chen, Zeyi Wen, Bingsheng He, Jian Chen","year":"2021","venue":"IEEE TKDE","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dosovitskiy et al., 2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Galar et al., 2011] Mikel Galar, Alberto Fernández, Edurne Bar- renechea, Humberto Bustince, and Francisco Herrera. An overview of ensemble methods for binary classifiers in multi- class problems: Experimental study on one-vs-one and one-vs-all schemes. Pattern Recognition, 44(8):1761-1776, 2011.","title":"An overview of ensemble methods for binary classifiers in multiclass problems: Experimental study on one-vs-one and one-vs-all schemes","authors":"Mikel Galar, Alberto Fernández, Edurne Barrenechea, Humberto Bustince, Francisco Herrera","year":"2011","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gao and Zhou, 2021] Bin-Bin Gao and Hong-Yu Zhou. Learning to discover multi-class attentional regions for multi-label image recognition. IEEE TIP, 30:5920-5932, 2021.","title":"Learning to discover multi-class attentional regions for multi-label image recognition","authors":"Zhou ; Bin-Bin Gao, Hong-Yu Gao, Zhou","year":"2021","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gou et al., 2021] Jianping Gou, Baosheng Yu, Stephen J May- bank, and Dacheng Tao. Knowledge distillation: A survey. IJCV, 129(6):1789-1819, 2021.","title":"Knowledge distillation: A survey","authors":"Jianping Gou, Baosheng Yu, Stephen J Maybank, Dacheng Tao","year":"2021","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hinton et al., 2015] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.","title":"Distilling the knowledge in a neural network","authors":"Geoffrey Hinton, Oriol Vinyals, Jeff Dean","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lanchantin et al., 2021] Jack Lanchantin, Tianlu Wang, Vicente Ordonez, and Yanjun Qi. General multi-label image classifica- tion with transformers. In CVPR, pages 16478-16488, 2021.","title":"General multi-label image classification with transformers","authors":"Lanchantin","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2018] Yongcheng Liu, Lu Sheng, Jing Shao, Junjie Yan, Shiming Xiang, and Chunhong Pan. Multi-label image classifi- cation via knowledge distillation from weakly-supervised detec- tion. In ACM MM, pages 700-708, 2018.","title":"Multi-label image classification via knowledge distillation from weakly-supervised detection","authors":"Yongcheng Liu, Lu Sheng, Jing Shao, Junjie Yan, Shiming Xiang, Chunhong Pan","year":"2018","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2021] Shilong Liu, Lei Zhang, Xiao Yang, Hang Su, and Jun Zhu. Query2label: A simple transformer way to multi-label classification. arXiv preprint arXiv:2107.10834, 2021.","title":"Query2label: A simple transformer way to multi-label classification","authors":"Shilong Liu, Lei Zhang, Xiao Yang, Hang Su, Jun Zhu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Nguyen et al., 2021] Hoang D Nguyen, Xuan-Son Vu, and Duc- Trong Le. Modular graph transformer networks for multi-label image classification. In AAAI, volume 35, pages 9092-9100, 2021.","title":"Modular graph transformer networks for multi-label image classification","authors":"Xuan-Son Hoang D Nguyen, Duc-Trong Vu, Le","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word repre- sentation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532- 1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Pennington","year":"2014","venue":"Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ridnik et al., 2021] Tal Ridnik, Emanuel Ben-Baruch, Nadav Za- mir, Asaf Noy, Itamar Friedman, Matan Protter, and Lihi Zelnik- Manor. Asymmetric loss for multi-label classification. In ICCV, pages 82-91, 2021.","title":"Asymmetric loss for multi-label classification","authors":"Tal Ridnik, Emanuel Ben-Baruch, Nadav Zamir, Asaf Noy, Itamar Friedman, Matan Protter, Lihi Zelnik-Manor","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Song et al., 2021] Liangchen Song, Jialian Wu, Ming Yang, Qian Zhang, Yuan Li, and Junsong Yuan. Handling difficult labels for multi-label image classification via uncertainty distillation. In ACM MM, pages 2410-2419, 2021.","title":"Handling difficult labels for multi-label image classification via uncertainty distillation","authors":"Song","year":"2021","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2016] Jiang Wang, Yi Yang, Junhua Mao, Zhiheng Huang, Chang Huang, and Wei Xu. Cnn-rnn: A unified frame- work for multi-label image classification. In CVPR, pages 2285- 2294, 2016.","title":"Cnn-rnn: A unified framework for multi-label image classification","authors":"Jiang Wang, Yi Yang, Junhua Mao, Zhiheng Huang, Chang Huang, Wei Xu","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wu et al., 2020] Tong Wu, Qingqiu Huang, Ziwei Liu, Yu Wang, and Dahua Lin. Distribution-balanced loss for multi-label clas- sification in long-tailed datasets. In ECCV, pages 162-178. Springer, 2020.","title":"Distribution-balanced loss for multi-label classification in long-tailed datasets","authors":"Tong Wu, Qingqiu Huang, Ziwei Liu, Yu Wang, Dahua Lin","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xiang et al., 2020] Liuyu Xiang, Guiguang Ding, and Jungong Han. Learning from multiple experts: Self-paced knowledge dis- tillation for long-tailed classification. In ECCV, pages 247-263. Springer, 2020.","title":"Learning from multiple experts: Self-paced knowledge distillation for long-tailed classification","authors":"Xiang","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yalniz et al., 2019] I Zeki Yalniz, Hervé Jégou, Kan Chen, Manohar Paluri, and Dhruv Mahajan. Billion-scale semi- supervised learning for image classification. arXiv preprint arXiv:1905.00546, 2019.","title":"Billion-scale semisupervised learning for image classification","authors":"Yalniz","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ye et al., 2020] Jin Ye, Junjun He, Xiaojiang Peng, Wenhao Wu, and Yu Qiao. Attention-driven dynamic graph convolutional net- work for multi-label image recognition. In ECCV, pages 649- 665. Springer, 2020.","title":"Attention-driven dynamic graph convolutional network for multi-label image recognition","authors":"Jin Ye, Junjun He, Xiaojiang Peng, Wenhao Wu, Yu Qiao","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[You et al., 2020] Renchun You, Zhiyao Guo, Lei Cui, Xiang Long, Yingze Bao, and Shilei Wen. Cross-modality attention with se- mantic graph embedding for multi-label classification. In AAAI, volume 34, pages 12709-12716, 2020.","title":"Cross-modality attention with semantic graph embedding for multi-label classification","authors":"You","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yuan et al., 2020] Li Yuan, Francis EH Tay, Guilin Li, Tao Wang, and Jiashi Feng. Revisiting knowledge distillation via label smoothing regularization. In CVPR, 2020.","title":"Revisiting knowledge distillation via label smoothing regularization","authors":"Li Yuan, Francis Eh Tay, Guilin Li, Tao Wang, Jiashi Feng","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2019] Linfeng Zhang, Jiebo Song, Anni Gao, Jing- wei Chen, Chenglong Bao, and Kaisheng Ma. Be your own teacher: Improve the performance of convolutional neural net- works via self distillation. In ICCV, pages 3713-3722, 2019.","title":"Be your own teacher: Improve the performance of convolutional neural networks via self distillation","authors":"Zhang","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhou et al., 2021a] Fengtao Zhou, Sheng Huang, Bo Liu, and Dan Yang. Multi-label image classification via category prototype compositional learning. IEEE TCSVT, 2021.","title":"Multi-label image classification via category prototype compositional learning","authors":"Zhou","year":"2021","venue":"IEEE TCSVT","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhou et al., 2021b] Fengtao Zhou, Sheng Huang, and Yun Xing. Deep semantic dictionary learning for multi-label image classifi- cation. In AAAI, volume 35, pages 3572-3580, 2021.","title":"Deep semantic dictionary learning for multi-label image classification","authors":"Zhou","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhou et al., 2021c] Helong Zhou, Liangchen Song, Jiajie Chen, Ye Zhou, Guoli Wang, Junsong Yuan, and Qian Zhang. Re- thinking soft labels for knowledge distillation: A bias-variance tradeoff perspective. arXiv preprint arXiv:2102.00650, 2021.","title":"Rethinking soft labels for knowledge distillation: A bias-variance tradeoff perspective","authors":"Zhou","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhu et al., 2017] Feng Zhu, Hongsheng Li, Wanli Ouyang, Neng- hai Yu, and Xiaogang Wang. Learning spatial regularization with image-level supervisions for multi-label image classification. In CVPR, pages 5513-5522, 2017.","title":"Learning spatial regularization with image-level supervisions for multi-label image classification","authors":"Feng Zhu, Hongsheng Li, Wanli Ouyang, Nenghai Yu, Xiaogang Wang","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}