{"type":"ScholarlyArticle","title":"Unreliable Partial Label Learning with Recursive Separation","authors":[{"name":"Yu Shi","slug":"yu-shi","given":"Yu","family":"Shi","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing 211189 , China"},{"name":"Ning Xu","slug":"ning-xu","given":"Ning","family":"Xu","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing 211189 , China"},{"name":"Hua Yuan","slug":"hua-yuan","given":"Hua","family":"Yuan","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing 211189 , China"},{"name":"Xin Geng","slug":"xin-geng","given":"Xin","family":"Geng","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing 211189 , China"}],"abstract":"Partial label learning (PLL) is a typical weakly supervised learning problem in which each instance is associated with a candidate label set, and among which only one is true. However, the assumption that the ground-truth label is always among the candidate label set would be unrealistic, as the reliability of the candidate label sets in real-world applications cannot be guaranteed by annotators. Therefore, a generalized PLL named Unreliable Partial Label Learning (UPLL) is proposed, in which the true label may not be in the candidate label set. Due to the challenges posed by unreliable labeling, previous PLL methods will experience a marked decline in performance when applied to UPLL. To address the issue, we propose a two-stage framework named Unreliable Partial Label Learning with Recursive Separation (UPLLRS). In the first stage, the self-adaptive recursive separation strategy is proposed to separate the training set into a reliable subset and an unreliable subset. In the second stage, a disambiguation strategy is employed to progressively identify the ground-truth labels in the reliable subset. Simultaneously, semi-supervised learning methods are adopted to extract valuable information from the unreliable subset. Our method demonstrates state-of-the-art performance as evidenced by experimental results, particularly in situations of high unreliability. Code and supplementary materials are available at https://github.com/dhiyu/UPLLRS.","doi":"10.24963/ijcai.2023/468","url":"https://ijcai.org/proceedings/2023/468","pdf":"https://ijcai.org/proceedings/2023/0468.pdf","markdown":"https://ijcai.org/proceedings/2023/468.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/468","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":468,"pages":{"first":4208,"last":4216},"keywords":["Machine Learning: ML: Weakly supervised learning"],"videos":[],"references":[{"position":1,"raw":"[Bai et al., 2021] Yingbin Bai, Erkun Yang, Bo Han, Yanhua Yang, Jiatong Li, Yinian Mao, Gang Niu, and Tongliang Liu. Understanding and improving early stopping for learning with noisy labels. Advances in Neural Informa- tion Processing Systems, 34, 2021.","title":"Understanding and improving early stopping for learning with noisy labels","authors":"Yingbin Bai, Erkun Yang, Bo Han, Yanhua Yang, Jiatong Li, Yinian Mao, Gang Niu, Tongliang Liu","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2013] Yi-Chen Chen, Vishal M Patel, Jais- hanker K Pillai, Rama Chellappa, and P Jonathon Phillips. Dictionary learning from ambiguously labeled data. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 353-360, 2013.","title":"Dictionary learning from ambiguously labeled data","authors":"Yi-Chen Chen, M Vishal, Patel, K Jaishanker, Rama Pillai, Jonathon Chellappa, Phillips","year":"2013","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018] Ching-Hui Chen, Vishal M Patel, and Rama Chellappa. Learning from ambiguously labeled face images. IEEE Transactions on Pattern Analysis & Ma- chine Intelligence, 40(07):1653-1667, 2018.","title":"Learning from ambiguously labeled face images","authors":"Ching-Hui Chen, M Vishal, Rama Patel, Chellappa","year":"2018","venue":"IEEE Transactions on Pattern Analysis & Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cour et al., 2011] Timothee Cour, Ben Sapp, and Ben Taskar. Learning from partial labels. The Journal of Ma- chine Learning Research, 12:1501-1536, 2011.","title":"Learning from partial labels","authors":"Timothee Cour, Ben Sapp, Ben Taskar","year":"2011","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dua and Graff, 2017] Dheeru Dua and Casey Graff. UCI machine learning repository. Available at http://archive. ics.uci.edu/ml, 2017.","title":"Dheeru Dua and Casey Graff. UCI machine learning repository","authors":"Graff Dua","year":"2017","venue":null,"doi":null,"url":"http://archive.ics.uci.edu/ml,2017","internal_url":null},{"position":6,"raw":"[Feng and An, 2019] Lei Feng and Bo An. Partial label learning with self-guided retraining. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 3542-3549, 2019.","title":"Lei Feng and Bo An. Partial label learning with self-guided retraining","authors":"An Feng","year":"2019","venue":"Proceedings of the AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Feng et al., 2020] Lei Feng, Jiaqi Lv, Bo Han, Miao Xu, Gang Niu, Xin Geng, Bo An, and Masashi Sugiyama. Provably consistent partial-label learning. Advances in Neural Information Processing Systems, 33:10948-10960, 2020.","title":"Provably consistent partial-label learning","authors":"Lei Feng, Jiaqi Lv, Bo Han, Miao Xu, Gang Niu, Xin Geng, Bo An, Masashi Sugiyama","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gong et al., 2017] Chen Gong, Tongliang Liu, Yuanyan Tang, Jian Yang, Jie Yang, and Dacheng Tao. A regulariza- tion approach for instance-based superset label learning. IEEE transactions on cybernetics, 48(3):967-978, 2017.","title":"A regularization approach for instance-based superset label learning","authors":"Chen Gong, Tongliang Liu, Yuanyan Tang, Jian Yang, Jie Yang, Dacheng Tao","year":"2017","venue":"IEEE transactions on cybernetics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hüllermeier and Beringer, 2006] Eyke Hüllermeier and Jürgen Beringer. Learning from ambiguously labeled examples. Intelligent Data Analysis, 10(5):419-439, 2006.","title":"Eyke Hüllermeier and Jürgen Beringer. Learning from ambiguously labeled examples","authors":"Beringer Hüllermeier","year":"2006","venue":"Intelligent Data Analysis","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Krizhevsky et al., 2009] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny im- ages. Master's thesis, Dept. of Comp. Sci., University of Toronto, 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky, Geoffrey Hinton","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu and Dietterich, 2012] Liping Liu and Thomas Diet- terich. A conditional multinomial mixture model for su- perset label learning. Advances in neural information pro- cessing systems, 25, 2012.","title":"A conditional multinomial mixture model for su-perset label learning","authors":"Dietterich Liu, Liping Liu, Thomas Dietterich","year":"2012","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2020] Sheng Liu, Jonathan Niles-Weed, Narges Razavian, and Carlos Fernandez-Granda. Early-learning regularization prevents memorization of noisy labels. Advances in neural information processing systems, 33:20331-20342, 2020.","title":"Early-learning regularization prevents memorization of noisy labels","authors":"Sheng Liu, Jonathan Niles-Weed, Narges Razavian, Carlos Fernandez-Granda","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Loshchilov and Hutter, 2016] Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016.","title":"Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts","authors":"Hutter Loshchilov","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Luo and Orabona, 2010] Jie Luo and Francesco Orabona. Learning from candidate labeling sets. Advances in neural information processing systems, 23, 2010.","title":"Learning from candidate labeling sets","authors":"Orabona Luo, Jie Luo, Francesco Orabona","year":"2010","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lv et al., 2020] Jiaqi Lv, Miao Xu, Lei Feng, Gang Niu, Xin Geng, and Masashi Sugiyama. Progressive identifi- cation of true labels for partial-label learning. In Inter- national Conference on Machine Learning, pages 6500- 6510. PMLR, 2020.","title":"Progressive identification of true labels for partial-label learning","authors":"Jiaqi Lv, Miao Xu, Lei Feng, Gang Niu, Xin Geng, Masashi Sugiyama","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lv et al., 2023] Jiaqi Lv, Biao Liu, Lei Feng, Ning Xu, Miao Xu, Bo An, Gang Niu, and Xin Geng. On the ro- bustness of average losses for partial-label learning. IEEE Transactions on Pattern Analysis & Machine Intelligence, in press, 2023.","title":"On the robustness of average losses for partial-label learning","authors":"Jiaqi Lv, Biao Liu, Lei Feng, Ning Xu, Miao Xu, Bo An, Gang Niu, Xin Geng","year":"2023","venue":"IEEE Transactions on Pattern Analysis & Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lyu et al., 2019] Gengyu Lyu, Songhe Feng, Tao Wang, Congyan Lang, and Yidong Li. Gm-pll: graph matching based partial label learning. IEEE Transactions on Knowl- edge and Data Engineering, 33(2):521-535, 2019.","title":"Gm-pll: graph matching based partial label learning","authors":"Lyu","year":"2019","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Paszke et al., 2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural in- formation processing systems, 32, 2019.","title":"Pytorch: An imperative style, highperformance deep learning library","authors":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga","year":"2019","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Robbins and Monro, 1951] Herbert Robbins and Sutton Monro. A stochastic approximation method. The annals of mathematical statistics, pages 400-407, 1951.","title":"The annals of mathematical statistics","authors":"Monro Robbins, Herbert Robbins, Sutton Monro","year":"1951","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shorten and Khoshgoftaar, 2019] Connor Shorten and Taghi M. Khoshgoftaar. A survey on image data augmentation for deep learning. J. Big Data, 6:60, 2019.","title":"A survey on image data augmentation for deep learning","authors":"Connor Shorten, M Taghi, Khoshgoftaar","year":"2019","venue":"J. Big Data","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sohn et al., 2020] Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi-supervised learning with con- sistency and confidence. Advances in neural information processing systems, 33:596-608, 2020.","title":"Fixmatch: Simplifying semi-supervised learning with consistency and confidence","authors":"Sohn","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tang and Zhang, 2017] Cai-Zhi Tang and Min-Ling Zhang. Confidence-rated discriminative partial label learning. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 31, 2017.","title":"Confidence-rated discriminative partial label learning","authors":"Cai-Zhi Tang, Min-Ling Zhang","year":"2017","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2022] Haobo Wang, Ruixuan Xiao, Yixuan Li, Lei Feng, Gang Niu, Gang Chen, and Junbo Zhao. Pico: Contrastive label disambiguation for partial label learning. arXiv preprint arXiv:2201.08984, 2022.","title":"Pico: Contrastive label disambiguation for partial label learning","authors":"Wang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wen et al., 2021] Hongwei Wen, Jingyi Cui, Hanyuan Hang, Jiabin Liu, Yisen Wang, and Zhouchen Lin. Lever- aged weighted loss for partial label learning. In Inter- national Conference on Machine Learning, pages 11091- 11100. PMLR, 2021.","title":"Leveraged weighted loss for partial label learning","authors":"Hongwei Wen, Jingyi Cui, Hanyuan Hang, Jiabin Liu, Yisen Wang, Zhouchen Lin","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wu et al., 2022] Dong-Dong Wu, Deng-Bao Wang, and Min-Ling Zhang. Revisiting consistency regularization for deep partial label learning. In International Conference on Machine Learning, pages 24212-24225. PMLR, 2022.","title":"Revisiting consistency regularization for deep partial label learning","authors":"Dong-Dong Wu, Deng-Bao Wang, Min-Ling Zhang","year":"2022","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xu et al., 2019] Ning Xu, Jiaqi Lv, and Xin Geng. Partial label learning via label enhancement. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 5557-5564, 2019.","title":"Partial label learning via label enhancement","authors":"Ning Xu, Jiaqi Lv, Xin Geng","year":"2019","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xu et al., 2021a] Ning Xu, Yun-Peng Liu, and Xin Geng. Label enhancement for label distribution learning. IEEE Transactions on Knowledge and Data Engineering, 33(4):1632-1643, 2021.","title":"Label enhancement for label distribution learning","authors":"Xu","year":"2021","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":"/proceedings/2018/406"},{"position":28,"raw":"[Xu et al., 2021b] Ning Xu, Congyu Qiao, Xin Geng, and Min-Ling Zhang. Instance-dependent partial label learn- ing. Advances in Neural Information Processing Systems, 34:27119-27130, 2021.","title":"Instance-dependent partial label learning","authors":"Xu","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Xu et al., 2023] Ning Xu, Jun Shu, Renyi Zheng, Xin Geng, Deyu Meng, and Min-Ling Zhang. Variational label en- hancement. IEEE Transactions on Pattern Analysis & Ma- chine Intelligence, (01):1-15, 2023.","title":"Variational label enhancement","authors":"Ning Xu, Jun Shu, Renyi Zheng, Xin Geng, Deyu Meng, Min-Ling Zhang","year":"2023","venue":"IEEE Transactions on Pattern Analysis & Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yao et al., 2020a] Yao Yao, Jiehui Deng, Xiuhua Chen, Chen Gong, Jianxin Wu, and Jian Yang. Deep discrim- inative cnn with temporal ensembling for ambiguously- labeled image classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 12669-12676, 2020.","title":"Deep discriminative cnn with temporal ensembling for ambiguouslylabeled image classification","authors":"Yao","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yao et al., 2020b] Yao Yao, Chen Gong, Jiehui Deng, and Jian Yang. Network cooperation with progressive disam- biguation for partial label learning. In Joint European Conference on Machine Learning and Knowledge Discov- ery in Databases, pages 471-488. Springer, 2020.","title":"Network cooperation with progressive disambiguation for partial label learning","authors":"Yao","year":"2020","venue":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yu and Zhang, 2016] Fei Yu and Min-Ling Zhang. Maxi- mum margin partial label learning. In Asian conference on machine learning, pages 96-111. PMLR, 2016.","title":"Maximum margin partial label learning","authors":"Yu, Zhang ; ] Fei Yu, Min-Ling Zhang","year":"2016","venue":"Asian conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zagoruyko and Komodakis, 2016] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC, 2016.","title":"Wide residual networks","authors":"Sergey Zagoruyko, Nikos Komodakis","year":"2016","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zeng et al., 2013] Zinan Zeng, Shijie Xiao, Kui Jia, Tsung- Han Chan, Shenghua Gao, Dong Xu, and Yi Ma. Learning by associating ambiguously labeled images. In Proceed- ings of the IEEE Conference on Computer Vision and Pat- tern Recognition, pages 708-715, 2013.","title":"Learning by associating ambiguously labeled images","authors":"Zinan Zeng, Shijie Xiao, Kui Jia, Tsung-Han Chan, Shenghua Gao, Dong Xu, Yi Ma","year":"2013","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhang and Yu, 2015] Min-Ling Zhang and Fei Yu. Solving the partial label learning problem: An instance-based ap- proach. In Twenty-fourth international joint conference on artificial intelligence, 2015.","title":"Solving the partial label learning problem: An instance-based approach","authors":"Yu ; ] Min-Ling Zhang, Fei Zhang, Yu","year":"2015","venue":"Twenty-fourth international joint conference on artificial intelligence","doi":null,"url":null,"internal_url":"/proceedings/2015/568"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}