{"type":"ScholarlyArticle","title":"Fusion Label Enhancement for Multi-Label Learning","authors":[{"name":"Xingyu Zhao","slug":"xingyu-zhao","given":"Xingyu","family":"Zhao","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing 211189 , China; Key Laboratory of Computer Network and Information Integration (Ministry of Education) , Southeast University , Nanjing 211189 , China"},{"name":"Yuexuan An","slug":"yuexuan-an","given":"Yuexuan","family":"An","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing 211189 , China; Key Laboratory of Computer Network and Information Integration (Ministry of Education) , Southeast University , Nanjing 211189 , China"},{"name":"Ning Xu","slug":"ning-xu","given":"Ning","family":"Xu","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing 211189 , China; Key Laboratory of Computer Network and Information Integration (Ministry of Education) , Southeast University , Nanjing 211189 , China"},{"name":"Xin Geng","slug":"xin-geng","given":"Xin","family":"Geng","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing 211189 , China; Key Laboratory of Computer Network and Information Integration (Ministry of Education) , Southeast University , Nanjing 211189 , China"}],"abstract":"Multi-label learning (MLL) refers to the problem of tagging a given instance with a set of relevant labels. In MLL, the implicit relative importance of different labels representing a single instance is generally different, which recently gained considerable attention and should be fully leveraged. Therefore, label enhancement (LE) has been widely applied in various MLL tasks as the ability to effectively mine the implicit relative importance information of different labels. However, due to the fact that the label enhancement process in previous LE-based MLL methods is decoupled from the training process on the predictive models, the objective of LE does not match the training process and finally affects the whole learning system. In this paper, we propose a novel approach named Fusion Label Enhancement for Multi-label learning (FLEM) to effectively integrate the LE process and the training process. Specifically, we design a matching and interaction mechanism which leverages a novel interaction label enhancement loss to avoid that the recovered label distribution does not match the need of the predictive model. In the meantime, we present a unified label distribution loss for establishing the corresponding relationship between the recovered label distribution and the training of the predictive model. With the proposed loss, the label distributions recovered from the LE process can be efficiently utilized for training the predictive model. Experimental results on multiple benchmark datasets validate the effectiveness of the proposed approach.","doi":"10.24963/ijcai.2022/524","url":"https://ijcai.org/proceedings/2022/524","pdf":"https://ijcai.org/proceedings/2022/0524.pdf","markdown":"https://ijcai.org/proceedings/2022/524.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/524","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":524,"pages":{"first":3773,"last":3779},"keywords":["Machine Learning: Multi-label"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986968","length_sec":88,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985111","length_sec":623,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Boutell et al., 2004] Matthew R. Boutell, Jiebo Luo, Xipeng Shen, and Christopher M. Brown. Learning multi-label scene classification. Pattern Recognition, 2004.","title":"Learning multi-label scene classification","authors":"Matthew R Boutell, Jiebo Luo, Xipeng Shen, Christopher M Brown","year":"2004","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chua et al., 2009] Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. NUS- WIDE: a real-world web image database from national university of singapore. In CIVR, 2009.","title":"NUS-WIDE: a real-world web image database from national university of singapore","authors":"Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, Yantao Zheng","year":"2009","venue":"CIVR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cui et al., 2019] Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge J. Belongie. Class-balanced loss based on effective number of samples. In CVPR, 2019.","title":"Class-balanced loss based on effective number of samples","authors":"Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, Serge J Belongie","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Debole and Sebastiani, 2005] Franca Debole and Fabrizio Sebastiani. An analysis of the relative hardness of reuters- 21578 subsets. JASIST, 2005.","title":"Franca Debole and Fabrizio Sebastiani. An analysis of the relative hardness of reuters-21578 subsets","authors":"Sebastiani Debole","year":"2005","venue":"JASIST","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Everingham et al., 2015] Mark Everingham, Seyed Mo- hammadali Eslami, Luc Van Gool, Christopher K. I. Williams, John M. Winn, and Andrew Zisserman. The pas- cal visual object classes challenge: A retrospective. IJCV, 2015.","title":"The pascal visual object classes challenge: A retrospective","authors":"Mark Everingham, Seyed Mohammadali Eslami, Luc Van Gool, K I Christopher, John M Williams, Andrew Winn, Zisserman","year":"2015","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fürnkranz et al., 2008] Johannes Fürnkranz, Eyke Hüllermeier, Eneldo Loza Mencía, and Klaus Brinker. Multilabel classification via calibrated label ranking. Machine Learning, 2008.","title":"Multilabel classification via calibrated label ranking","authors":"Johannes Fürnkranz, Eyke Hüllermeier, Eneldo Loza Mencía, Klaus Brinker","year":"2008","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gao et al., 2017] Bin-Bin Gao, Chao Xing, Chen-Wei Xie, Jianxin Wu, and Xin Geng. Deep label distribution learn- ing with label ambiguity. TIP, 2017.","title":"Deep label distribution learning with label ambiguity","authors":"Gao","year":"2017","venue":"TIP","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Geng, 2016] Xin Geng. Label distribution learning. TKDE, 2016.","title":"Label distribution learning","authors":"Xin Geng","year":"2016","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2021] Yi Huang, Buse Giledereli, Abdullatif Köksal, Arzucan Özgür, and Elif Ozkirimli. Balancing methods for multi-label text classification with long-tailed class distribution. In EMNLP, 2021.","title":"Balancing methods for multi-label text classification with long-tailed class distribution","authors":"Yi Huang, Buse Giledereli, Abdullatif Köksal, Arzucan Özgür, Elif Ozkirimli","year":"2021","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lanchantin et al., 2021] Jack Lanchantin, Tianlu Wang, Vi- cente Ordonez, and Yanjun Qi. General multi-label image classification with transformers. In CVPR, 2021.","title":"General multi-label image classification with transformers","authors":"Lanchantin","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge J. Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and Charles Lawrence Zitnick. Microsoft COCO: common objects in context. In ECCV, 2014.","title":"Microsoft COCO: common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge J Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, Charles Lawrence Zitnick","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lin et al., 2020] Tsung-Yi Lin, Priya Goyal, Ross B. Gir- shick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. TPAMI, 2020.","title":"Focal loss for dense object detection","authors":"Tsung-Yi Lin, Priya Goyal, Ross B Girshick, Kaiming He, Piotr Dollár","year":"2020","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2017] Jingzhou Liu, Wei-Cheng Chang, Yuexin Wu, and Yiming Yang. Deep learning for extreme multi- label text classification. In SIGIR, 2017.","title":"Deep learning for extreme multilabel text classification","authors":"Jingzhou Liu, Wei-Cheng Chang, Yuexin Wu, Yiming Yang","year":"2017","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2021] Weiwei Liu, Haobo Wang, Xiaobo Shen, and Ivor W. Tsang. The emerging trends of multi-label learning. TPAMI, 2021.","title":"The emerging trends of multi-label learning","authors":"Weiwei Liu, Haobo Wang, Xiaobo Shen, Ivor W Tsang","year":"2021","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lo et al., 2011] Hung-Yi Lo, Ju-Chiang Wang, Hsin-Min Wang, and Shou-De Lin. Cost-sensitive multi-label learn- ing for audio tag annotation and retrieval. TMM, 2011.","title":"Cost-sensitive multi-label learning for audio tag annotation and retrieval","authors":"Hung-Yi Lo, Ju-Chiang Wang, Hsin-Min Wang, Shou-De Lin","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Loshchilov and Hutter, 2017] Ilya Loshchilov and Frank Hutter. Fixing weight decay regularization in adam. CoRR, 2017.","title":"Ilya Loshchilov and Frank Hutter. Fixing weight decay regularization in adam","authors":"Hutter Loshchilov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Reddi et al., 2018] Sashank J. Reddi, Satyen Kale, and San- jiv Kumar. On the convergence of adam and beyond. In ICLR, 2018.","title":"On the convergence of adam and beyond","authors":"J Sashank, Satyen Reddi, Sanjiv Kale, Kumar","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ridnik et al., 2021] Tal Ridnik, Emanuel Ben Baruch, Na- dav Zamir, Asaf Noy, Itamar Friedman, Matan Protter, and Lihi Zelnik-Manor. Asymmetric loss for multi-label clas- sification. In ICCV, 2021.","title":"Asymmetric loss for multi-label classification","authors":"Tal Ridnik, Emanuel Ben Baruch, Nadav Zamir, Asaf Noy, Itamar Friedman, Matan Protter, Lihi Zelnik-Manor","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shao et al., 2018] Ruifeng Shao, Ning Xu, and Xin Geng. Multi-label learning with label enhancement. In ICDM, 2018.","title":"Multi-label learning with label enhancement","authors":"Ruifeng Shao, Ning Xu, Xin Geng","year":"2018","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tsoumakas et al., 2011] Grigorios Tsoumakas, Ioannis Katakis, and Ioannis P. Vlahavas. Random k-labelsets for multilabel classification. TKDE, 2011.","title":"Random k-labelsets for multilabel classification","authors":"Grigorios Tsoumakas, Ioannis Katakis, Ioannis P Vlahavas","year":"2011","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wah et al., 2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech- ucsd birds-200-2011 dataset. California Institute of Tech- nology, 2011.","title":"The caltechucsd birds-200-2011 dataset","authors":"Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, Serge Belongie","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2019] Haobo Wang, Weiwei Liu, Yang Zhao, Chen Zhang, Tianlei Hu, and Gang Chen. Discrimina- tive and correlative partial multi-label learning. In IJCAI, 2019.","title":"Discriminative and correlative partial multi-label learning","authors":"Wang","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/512"},{"position":23,"raw":"[Wang et al., 2020] Haobo Wang, Zhao Li, Jiaming Huang, Pengrui Hui, Weiwei Liu, Tianlei Hu, and Gang Chen. Collaboration based multi-label propagation for fraud de- tection. In IJCAI, 2020.","title":"Collaboration based multi-label propagation for fraud detection","authors":"Haobo Wang, Zhao Li, Jiaming Huang, Pengrui Hui, Weiwei Liu, Tianlei Hu, Gang Chen","year":"2020","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/343"},{"position":24,"raw":"[Wu et al., 2020] Tong Wu, Qingqiu Huang, Ziwei Liu, Yu Wang, and Dahua Lin. Distribution-balanced loss for multi-label classification in long-tailed datasets. In ECCV, 2020.","title":"Distribution-balanced loss for multi-label classification in long-tailed datasets","authors":"Tong Wu, Qingqiu Huang, Ziwei Liu, Yu Wang, Dahua Lin","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xu et al., 2020] Ning Xu, Jun Shu, Yun-Peng Liu, and Xin Geng. Variational label enhancement. In ICML, 2020.","title":"Variational label enhancement","authors":"Ning Xu, Jun Shu, Yun-Peng Liu, Xin Geng","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xu et al., 2021a] Ning Xu, Yun-Peng Liu, and Xin Geng. Label enhancement for label distribution learning. TKDE, 2021.","title":"Label enhancement for label distribution learning","authors":"Xu","year":"2021","venue":"TKDE","doi":null,"url":null,"internal_url":"/proceedings/2018/406"},{"position":27,"raw":"[Xu et al., 2021b] Ning Xu, Congyu Qiao, Xin Geng, and Min-Ling Zhang. Instance-dependent partial label learn- ing. In NeurIPS, 2021.","title":"Instance-dependent partial label learning","authors":"Xu","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yang et al., 2018] Pengcheng Yang, Xu Sun, Wei Li, Shum- ing Ma, Wei Wu, and Houfeng Wang. SGM: sequence generation model for multi-label classification. In COL- ING, 2018.","title":"SGM: sequence generation model for multi-label classification","authors":"Pengcheng Yang, Xu Sun, Wei Li, Shuming Ma, Wei Wu, Houfeng Wang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang and Zhou, 2007] Min-Ling Zhang and Zhi-Hua Zhou. ML-KNN: A lazy learning approach to multi-label learning. Pattern Recognition, 2007.","title":"ML-KNN: A lazy learning approach to multi-label learning","authors":"Zhou Zhang, Min-Ling Zhang, Zhi-Hua Zhou","year":"2007","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang and Zhou, 2014] Min-Ling Zhang and Zhi-Hua Zhou. A review on multi-label learning algorithms. TKDE, 2014.","title":"A review on multi-label learning algorithms","authors":"Zhou Zhang, Min-Ling Zhang, Zhi-Hua Zhou","year":"2014","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhang et al., 2021] Ningyu Zhang, Xiang Chen, Xin Xie, Shumin Deng, Chuanqi Tan, Mosha Chen, Fei Huang, Luo Si, and Huajun Chen. Document-level relation extraction as semantic segmentation. In IJCAI, 2021.","title":"Document-level relation extraction as semantic segmentation","authors":"Zhang","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/551"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}