{"type":"ScholarlyArticle","title":"Ambiguity-Induced Contrastive Learning for Instance-Dependent Partial Label Learning","authors":[{"name":"Shi-Yu Xia","slug":"shi-yu-xia","given":"Shi-Yu","family":"Xia","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing 211189 , China; Key Laboratory of Computer Network and Information Integration (Ministry of Education) , Southeast University , Nanjing 211189 , China"},{"name":"Jiaqi Lv","slug":"jiaqi-lv","given":"JiaQi","family":"Lv","affiliation":"RIKEN Center for Advanced Intelligence Project shiyu"},{"name":"Ning Xu","slug":"ning-xu","given":"Ning","family":"Xu","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing 211189 , China; Key Laboratory of Computer Network and Information Integration (Ministry of Education) , Southeast University , Nanjing 211189 , China"},{"name":"Xin Geng","slug":"xin-geng","given":"Xin","family":"Geng","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing 211189 , China; Key Laboratory of Computer Network and Information Integration (Ministry of Education) , Southeast University , Nanjing 211189 , China"}],"abstract":"Partial label learning (PLL) learns from a typical weak supervision, where each training instance is labeled with a set of ambiguous candidate labels (CLs) instead of its exact ground-truth label. Most existing PLL works directly eliminate, rather than exploiting the label ambiguity, since they explicitly or implicitly assume that incorrect CLs are noise independent of the instance. While a more practical setting in the wild should be instance-dependent, namely, the CLs depend on both the true label and the instance itself, such that each CL may describe the instance from some sensory channel, thereby providing some noisy but really valid information about the instance. In this paper, we leverage such additional information acquired from the ambiguity and propose AmBiguity-induced contrastive LEarning (ABLE) under the framework of contrastive learning. Specifically, for each CL of an anchor, we select a group of samples currently predicted as that class as ambiguity-induced positives, based on which we synchronously learn a representor (RP) that minimizes the weighted sum of contrastive losses of all groups and a classifier (CS) that minimizes a classification loss. Although they are circularly dependent: RP requires the ambiguity-induced positives on-the-fly induced by CS, and CS needs the first half of RP as the representation extractor, ABLE still enables RP and CS to be trained simultaneously within a coherent framework. Experiments on benchmark datasets demonstrate its substantial improvements over state-of-the-art methods for learning from the instance-dependent partially labeled data.","doi":"10.24963/ijcai.2022/502","url":"https://ijcai.org/proceedings/2022/502","pdf":"https://ijcai.org/proceedings/2022/0502.pdf","markdown":"https://ijcai.org/proceedings/2022/502.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/502","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":502,"pages":{"first":3615,"last":3621},"keywords":["Machine Learning: Weakly Supervised Learning","Machine Learning: Self-supervised Learning","Machine Learning: Classification"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987075","length_sec":84,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985218","length_sec":642,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Arpit et al., 2017] D. Arpit, S. Jastrzebski, N. Ballas, D. Krueger, E. Bengio, M. S. Kanwal, T. Maharaj, A. Fis- cher, A. C. Courville, Y. Bengio, and S. Lacoste-Julien. A closer look at memorization in deep networks. In Interna- tional Conference on Machine Learning, pages 233-242, Portland, OR, USA, August 2017. ACM.","title":"A closer look at memorization in deep networks","authors":"D Arpit, S Jastrzebski, N Ballas, D Krueger, E Bengio, M S Kanwal, T Maharaj, A Fischer, A C Courville, Y Bengio, S Lacoste-Julien","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2017] Ching-Hui Chen, Vishal M Patel, and Rama Chellappa. Learning from ambiguously labeled face images. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 40(7):1653-1667, 2017.","title":"Learning from ambiguously labeled face images","authors":"Ching-Hui Chen, M Vishal, Rama Patel, Chellappa","year":"2017","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2020] Ting Chen, Simon Kornblith, Moham- mad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Inter- national Conference on Machine Learning, pages 1597- 1607, Virtual Event, July 2020. ACM.","title":"A simple framework for contrastive learning of visual representations","authors":"Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Clanuwat et al., 2018] Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical japanese literature. CoRR, 2018.","title":"Deep learning for classical japanese literature","authors":"Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, David Ha","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cour et al., 2011] T. Cour, B. Sapp, and B. Taskar. Learn- ing from partial labels. Journal of Machine Learning Re- search, 12(5):1501-1536, 2011.","title":"Learning from partial labels","authors":"T Cour, B Sapp, B Taskar","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Feldman and Zhang, 2020] Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discov- ering the long tail via influence estimation. In Advances in Neural Information Processing Systems, pages 2881- 2891, Virtual Event, December 2020. MIT Press.","title":"What neural networks memorize and why: Discovering the long tail via influence estimation","authors":"Zhang Feldman, Vitaly Feldman, Chiyuan Zhang","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Feng et al., 2020] L. Feng, J. Lv, B. Han, M. Xu, G. Niu, X. Geng, B. An, and M. Sugiyama. Provably consistent partial-label learning. In Advances in Neural Information Processing Systems, pages 10948-10960, Virtual Event, December 2020. MIT Press.","title":"Provably consistent partial-label learning","authors":"L Feng, J Lv, B Han, M Xu, G Niu, X Geng, B An, M Sugiyama","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In IEEE Conference on Computer Vision and Pat- tern Recognition, pages 770-778, Las Vegas, NV, USA, June 2016. IEEE.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2020] Kaiming He, Haoqi Fan, Yuxin Wu, Sain- ing Xie, and Ross Girshick. Momentum contrast for unsu- pervised visual representation learning. In IEEE Confer- ence on Computer Vision and Pattern Recognition, pages 9726-9735, Seattle, WA, USA, June 2020. IEEE.","title":"Momentum contrast for unsupervised visual representation learning","authors":"Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, Ross Girshick","year":"2020","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Khosla et al., 2020] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised con- trastive learning. In Advances in Neural Information Pro- cessing Systems, pages 18661-18673, Virtual Event, De- cember 2020. MIT Press.","title":"Supervised contrastive learning","authors":"Khosla","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[LeCun et al., 1998] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2021] Junnan Li, Caiming Xiong, and Steven C. H. Hoi. Mopro: Webly supervised learning with momentum prototypes. In International Conference on Learning Rep- resentations, Virtual Event, May 2021. OpenReview.net.","title":"Mopro: Webly supervised learning with momentum prototypes","authors":"Li","year":"2021","venue":"International Conference on Learning Representations, Virtual Event","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lv et al., 2020] J. Lv, M. Xu, L. Feng, G. Niu, X. Geng, and M. Sugiyama. Progressive identification of true labels for partial-label learning. In International Conference on Machine Learning, pages 6500-6510, Virtual Event, July 2020. ACM.","title":"Progressive identification of true labels for partial-label learning","authors":"J Lv, M Xu, L Feng, G Niu, X Geng, M Sugiyama","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Wang et al., 2020a] Haobo Wang, Weiwei Liu, Yang Zhao, Tianlei Hu, Ke Chen, and Gang Chen. Learning from multi-dimensional partial labels. In International Joint Conference on Artificial Intelligence, pages 2943-2949, Virtual Event, Japan, January 2020. ijcai.org.","title":"Learning from multi-dimensional partial labels","authors":"Wang","year":"2020","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2020/407"},{"position":15,"raw":"[Wang et al., 2020b] Haobo Wang, Yuzhou Qiang, Chen Chen, Weiwei Liu, Tianlei Hu, Zhao Li, and Gang Chen. Online partial label learning. In Joint European Confer- ence on Machine Learning and Knowledge Discovery in Databases, pages 455-470, Ghent, Belgium, September 2020. Springer.","title":"Online partial label learning","authors":"Wang","year":"2020","venue":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wang et al., 2022] Haobo Wang, Ruixuan Xiao, Yixuan Li, Lei Feng, Gang Niu, Gang Chen, and Junbo Zhao. PiCO: Contrastive label disambiguation for partial label learning. In International Conference on Learning Representations, 2022.","title":"PiCO: Contrastive label disambiguation for partial label learning","authors":"Wang","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Xiao et al., 2017] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for bench- marking machine learning algorithms. CoRR, 2017.","title":"Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms","authors":"Han Xiao, Kashif Rasul, Roland Vollgraf","year":"2017","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Xu et al., 2019] N. Xu, J. Lv, and X. Geng. Partial label learning via label enhancement. In Association for the Advance of Artificial Intelligence, pages 5557-5564, Hon- olulu, Hawaii, January 2019. AAAI Press.","title":"Partial label learning via label enhancement","authors":"N Xu, J Lv, X Geng","year":"2019","venue":"Association for the Advance of Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xu et al., 2021a] Ning Xu, Yun-Peng Liu, and Xin Geng. Label enhancement for label distribution learning. IEEE Trans. Knowl. Data Eng., 33(4):1632-1643, 2021.","title":"Label enhancement for label distribution learning","authors":"Xu","year":"2021","venue":"IEEE Trans. Knowl. Data Eng","doi":null,"url":null,"internal_url":"/proceedings/2018/406"},{"position":20,"raw":"[Xu et al., 2021b] Ning Xu, Congyu Qiao, Xin Geng, and Min-Ling Zhang. Instance-dependent partial label learn- ing. In Advances in Neural Information Processing Sys- tems, Virtual Event, December 2021. MIT Press.","title":"Instance-dependent partial label learning","authors":"Xu","year":"2021","venue":"Advances in Neural Information Processing Systems, Virtual Event","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yao et al., 2020] Y. Yao, C. Gong, J. Deng, X. Chen, J. Wu, and J. Yang. Deep discriminative CNN with temporal ensembling for ambiguously-labeled image classification. In Association for the Advance of Artificial Intelligence, pages 12669-12676, New York, NY, USA, February 2020. AAAI Press.","title":"Deep discriminative CNN with temporal ensembling for ambiguously-labeled image classification","authors":"Y Yao, C Gong, J Deng, X Chen, J Wu, J Yang","year":"2020","venue":"Association for the Advance of Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yu and Zhang, 2017] F. Yu and M. Zhang. Maximum mar- gin partial label learning. Machine Learning, 106(4):573- 593, 2017.","title":"Maximum margin partial label learning","authors":"F Yu, M Zhang","year":"2017","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang and Yu, 2015] M. Zhang and F. Yu. Solving the partial label learning problem: An instance-based ap- proach. In International Joint Conference on Artificial Intelligence, pages 4048-4054, Buenos Aires, Argentina, July 2015. AAAI Press.","title":"Solving the partial label learning problem: An instance-based approach","authors":"M Zhang, F Yu","year":"2015","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2015/568"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}