{"type":"ScholarlyArticle","title":"Webly-Supervised Fine-Grained Recognition with Partial Label Learning","authors":[{"name":"Yu-Yan Xu","slug":"yu-yan-xu","given":"Yu-Yan","family":"Xu","affiliation":"Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education , and Jiangsu Key Lab of Image and Video Understanding for Social Security , School of Computer Science and Engineering , Nanjing University of Science and Technology; State Key Laboratory of Integrated Services Networks , Xidian University"},{"name":"Yang Shen","slug":"yang-shen","given":"Yang","family":"Shen","affiliation":"Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education , and Jiangsu Key Lab of Image and Video Understanding for Social Security , School of Computer Science and Engineering , Nanjing University of Science and Technology"},{"name":"Xiu-Shen Wei","slug":"xiu-shen-wei","given":"Xiu-Shen","family":"Wei","affiliation":"Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education , and Jiangsu Key Lab of Image and Video Understanding for Social Security , School of Computer Science and Engineering , Nanjing University of Science and Technology; State Key Laboratory of Integrated Services Networks , Xidian University"},{"name":"Jian Yang","slug":"jian-yang","given":"Jian","family":"Yang","affiliation":"Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education , and Jiangsu Key Lab of Image and Video Understanding for Social Security , School of Computer Science and Engineering , Nanjing University of Science and Technology"}],"abstract":"The task of webly-supervised fine-grained recognition is to boost recognition accuracy of classifying subordinate categories (e.g., different bird species) by utilizing freely available but noisy web data. As the label noises significantly hurt the network training, it is desirable to distinguish and eliminate noisy images. In this paper, we propose two strategies, i.e., open-set noise removal and closed-set noise correction, to both remove such two kinds of web noises w.r.t. fine-grained recognition. Specifically, for open-set noise removal, we utilize a pre-trained deep model to perform deep descriptor transformation to estimate the positive correlation between these web images, and detect the open-set noises based on the correlation values. Regarding closed-set noise correction, we develop a top-k recall optimization loss for firstly assigning a label set towards each web image to reduce the impact of hard label assignment for closed-set noises. Then, we further propose to correct the sample with its label set as the true single label from a partial label learning perspective. Experiments on several webly-supervised fine-grained benchmark datasets show that our method obviously outperforms other existing state-of-the-art methods.","doi":"10.24963/ijcai.2022/209","url":"https://ijcai.org/proceedings/2022/209","pdf":"https://ijcai.org/proceedings/2022/0209.pdf","markdown":"https://ijcai.org/proceedings/2022/209.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/209","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":209,"pages":{"first":1502,"last":1508},"keywords":["Computer Vision: Recognition (object detection, categorization)","Machine Learning: Multi-label","Computer Vision: Machine Learning for Vision","Machine Learning: Weakly Supervised Learning","Computer Vision: Representation Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986803","length_sec":80,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984946","length_sec":347,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bouveyron and Girard, 2009] Charles Bouveyron and Stéphane Girard. Robust supervised classification with mixture models: Learning from data with uncertain labels. PR, 42(11):2649-2658, 2009.","title":"Robust supervised classification with mixture models: Learning from data with uncertain labels","authors":"Girard Bouveyron, Charles Bouveyron, Stéphane Girard","year":"2009","venue":"PR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cour et al., 2011] Timothee Cour, Ben Sapp, and Ben Taskar. Learning from partial labels. JMLR, 12(42):1501-1536, 2011.","title":"Learning from partial labels","authors":"Timothee Cour, Ben Sapp, Ben Taskar","year":"2011","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Eran and Shai, 2017] Malach Eran and Shalev-Shwartz Shai. De- coupling \"when to update\" from \"how to update\". In NeurIPS, pages 960-970, 2017.","title":"Decoupling \"when to update\" from \"how to update","authors":"Malach Eran, Shalev-Shwartz Shai","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Han et al., 2018] Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor W. Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with ex- tremely noisy labels. In NeurIPS, pages 8527-8537, 2018.","title":"Co-teaching: Robust training of deep neural networks with extremely noisy labels","authors":"Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor W Tsang, Masashi Sugiyama","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hendrycks et al., 2018] Dan Hendrycks, Mantas Mazeika, Duncan Wilson, and Kevin Gimpel. Using trusted data to train deep net- works on labels corrupted by severe noise. In NeurIPS, pages 10456-10465, 2018.","title":"Using trusted data to train deep networks on labels corrupted by severe noise","authors":"Dan Hendrycks, Mantas Mazeika, Duncan Wilson, Kevin Gimpel","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Horn et al., 2018] Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The iNaturalist species classification and detection dataset. In CVPR, pages 8769-8778, 2018.","title":"The iNaturalist species classification and detection dataset","authors":"Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, Serge Belongie","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2020] Lang Huang, Chao Zhang, and Hongyang Zhang. Self-adaptive training: Beyond empirical risk minimiza- tion. In NeurIPS, pages 960-970, 2020.","title":"Self-adaptive training: Beyond empirical risk minimization","authors":"Lang Huang, Chao Zhang, Hongyang Zhang","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lv et al., 2020] Jiaqi Lv, Miao Xu, Lei Feng, Gang Niu, Xin Geng, and Masashi Sugiyama. Progressive identification of true labels for partial-label learning. In ICML, pages 6500-6510, 2020.","title":"Progressive identification of true labels for partial-label learning","authors":"Jiaqi Lv, Miao Xu, Lei Feng, Gang Niu, Xin Geng, Masashi Sugiyama","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Maji et al., 2013] Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151, 2013.","title":"Fine-grained visual classification of aircraft","authors":"Maji","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Niu et al., 2018] Li Niu, Ashok Veeraraghavan, and Ashu Sabhar- wal. Webly supervised learning meets zero-shot learning: A hy- brid approach for fine-grained classification. In CVPR, pages 7171-7180, 2018.","title":"Webly supervised learning meets zero-shot learning: A hybrid approach for fine-grained classification","authors":"Li Niu, Ashok Veeraraghavan, Ashu Sabharwal","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Patrini et al., 2017] Giorgio Patrini, Alessandro Rozza, Aditya Kr- ishna Menon, Richard Nock, and Lizhen Qu. Making deep neu- ral networks robust to label noise: A loss correction approach. In CVPR, pages 1944-1952, 2017.","title":"Making deep neural networks robust to label noise: A loss correction approach","authors":"Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, Lizhen Qu","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Peng et al., 2020] Xiaojiang Peng, Kai Wang, Zhaoyang Zeng, Qing Li, Jianfei Yang, and Yu Qiao. Suppressing mislabeled data via grouping and self-attention. In ECCV, pages 786-802, 2020.","title":"Suppressing mislabeled data via grouping and self-attention","authors":"Xiaojiang Peng, Kai Wang, Zhaoyang Zeng, Qing Li, Jianfei Yang, Yu Qiao","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale im- age recognition. arXiv preprint arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Song et al., 2020] Kaitao Song, Xiu-Shen Wei, Xiangbo Shu, Ren- Jie Song, and Jianfeng Lu. Bi-modal progressive mask attention for fine-grained recognition. IEEE TIP, 29:7006-7018, 2020.","title":"Bi-modal progressive mask attention for fine-grained recognition","authors":"Song","year":"2020","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sun et al., 2020a] Guolei Sun, Hisham Cholakkal, Salman Khan, Fahad Khan, and Ling Shao. Fine-grained recognition: Account- ing for subtle differences between similar classes. In AAAI, pages 12047-12054, 2020.","title":"Fine-grained recognition: Accounting for subtle differences between similar classes","authors":"Sun","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sun et al., 2020b] Zeren Sun, Xian-Sheng Hua, Yazhou Yao, Xiu- Shen Wei, Guosheng Hu, and Jian Zhang. CRSSC: Salvage reusable samples from noisy data for robust learning. In ACM MM, pages 92-101, 2020.","title":"CRSSC: Salvage reusable samples from noisy data for robust learning","authors":"Sun","year":"2020","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sun et al., 2021a] Zeren Sun, Huafeng Liu, Qiong Wang, Tianfei Zhou, Qi Wu, and Zhenmin Tang. Co-LDL: A co-training-based label distribution learning method for tackling label noise. IEEE TMM, 2021. DOI: 10.1109/TMM.2021.3116430.","title":"Co-LDL: A co-training-based label distribution learning method for tackling label noise","authors":"Sun","year":"2021","venue":"IEEE TMM","doi":"10.1109/tmm.2021.3116430","url":null,"internal_url":null},{"position":19,"raw":"[Sun et al., 2021b] Zeren Sun, Yazhou Yao, Xiu-Shen Wei, Yong- shun Zhang, Fumin Shen, Jianxin Wu, Jian Zhang, and Heng Tao Shen. Webly supervised fine-grained recognition: Benchmark datasets and an approach. In ICCV, pages 10602-10611, 2021.","title":"Webly supervised fine-grained recognition: Benchmark datasets and an approach","authors":"Sun","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Szegedy et al., 2015] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Er- han, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In CVPR, pages 1-9, 2015.","title":"Going deeper with convolutions","authors":"Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2019] Qian-Wei Wang, Yu-Feng Li, and Zhi-Hua Zhou. Partial label learning with unlabeled data. In IJCAI, pages 3755-3761, 2019.","title":"Partial label learning with unlabeled data","authors":"Qian-Wei Wang, Yu-Feng Li, Zhi-Hua Zhou","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/521"},{"position":22,"raw":"[Wei et al., 2017] Xiu-Shen Wei, Chen-Lin Zhang, Yao Li, Chen- Wei Xie, Jianxin Wu, Chunhua Shen, and Zhi-Hua Zhou. Deep descriptor transforming for image Co-localization. In IJCAI, pages 3048-3054, 2017.","title":"Deep descriptor transforming for image Co-localization","authors":"Xiu-Shen Wei, Chen-Lin Zhang, Yao Li, Chen-Wei Xie, Jianxin Wu, Chunhua Shen, Zhi-Hua Zhou","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/425"},{"position":23,"raw":"[Wei et al., 2020] Hongxin Wei, Lei Feng, Xiangyu Chen, and Bo An. Combating noisy labels by agreement: A joint training method with co-regularization. In CVPR, pages 13726-13735, 2020.","title":"Combating noisy labels by agreement: A joint training method with co-regularization","authors":"Hongxin Wei, Lei Feng, Xiangyu Chen, Bo An","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wei et al., 2021a] Xiu-Shen Wei, Yang Shen, Xuhao Sun, Han-Jia Ye, and Jian Yang. A 2 -Net: Learning attribute-aware hash codes for large-scale fine-grained image retrieval. In NeurIPS, pages 5720-5730, 2021.","title":"A 2 -Net: Learning attribute-aware hash codes for large-scale fine-grained image retrieval","authors":"Wei","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wei et al., 2021b] Xiu-Shen Wei, Yi-Zhe Song, Oisin Mac Aodha, Jianxin Wu, Yuxin Peng, Jinhui Tang, Jian Yang, and Serge Be- longie. Fine-grained image analysis with deep learning: A sur- vey. IEEE TPAMI, 2021. DOI: 10.1109/TPAMI.2021.3126648.","title":"Fine-grained image analysis with deep learning: A survey","authors":"Wei","year":"2021","venue":"IEEE TPAMI","doi":"10.1109/tpami.2021.3126648","url":null,"internal_url":null},{"position":26,"raw":"[Wold et al., 1987] Svante Wold, Kim Esbensen, and Paul Geladi. Principal component analysis. Chemometrics and Intelligent Laboratory Systems, 2(1-3):37-52, 1987.","title":"Principal component analysis","authors":"Svante Wold, Kim Esbensen, Paul Geladi","year":"1987","venue":"Chemometrics and Intelligent Laboratory Systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xu et al., 2021] Ning Xu, Congyu Qiao, Xin Geng, and Min-Ling Zhang. Instance-dependent partial label learning. In NeurIPS, pages 27119-27130, 2021.","title":"Instance-dependent partial label learning","authors":"Ning Xu, Congyu Qiao, Xin Geng, Min-Ling Zhang","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yao et al., 2021] Yazhou Yao, Zeren Sun, Chuanyi Zhang, Fumin Shen, Qi Wu, Jian Zhang, and Zhenmin Tang. Jo-SRC: A con- trastive approach for combating noisy labels. In CVPR, pages 5192-5201, 2021.","title":"Jo-SRC: A contrastive approach for combating noisy labels","authors":"Yazhou Yao, Zeren Sun, Chuanyi Zhang, Fumin Shen, Qi Wu, Jian Zhang, Zhenmin Tang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yi and Wu, 2019] Kun Yi and Jianxin Wu. Probabilistic end-to- end noise correction for learning with noisy labels. In CVPR, pages 7017-7025, 2019.","title":"Probabilistic end-toend noise correction for learning with noisy labels","authors":"Yi, Wu ; ] Kun Yi, Jianxin Wu","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yu et al., 2019] Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor Tsang, and Masashi Sugiyama. How does disagreement help generalization against label corruption? In ICML, pages 7164- 7173, 2019.","title":"How does disagreement help generalization against label corruption?","authors":"Yu","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhang and Yu, 2015] Min-Ling Zhang and Fei Yu. Solving the partial label learning problem: An instance-based approach. In IJCAI, pages 4048-4054, 2015.","title":"Solving the partial label learning problem: An instance-based approach","authors":"Yu ; ] Min-Ling Zhang, Fei Zhang, Yu","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/568"},{"position":32,"raw":"[Zhang et al., 2020] Chuanyi Zhang, Yazhou Yao, Xiangbo Shu, Zechao Li, Zhenmin Tang, and Qi Wu. Data-driven meta-set based fine-grained visual recognition. In ACM MM, pages 2372- 2381, 2020.","title":"Data-driven meta-set based fine-grained visual recognition","authors":"Zhang","year":"2020","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhu et al., 2020] Yaohui Zhu, Chenlong Liu, and Shuqiang Jiang. Multi-attention meta learning for few-shot fine-grained image recognition. In IJCAI, pages 1090-1096, 2020.","title":"Multi-attention meta learning for few-shot fine-grained image recognition","authors":"Zhu","year":"2020","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/152"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}