{"type":"ScholarlyArticle","title":"No Regularization Is Needed: Efficient and Effective Incomplete Label Distribution Learning","authors":[{"name":"Xiang Li","slug":"xiang-li","given":"Xiang","family":"Li","affiliation":"College of Computer Science and Technology , Nanjing University of Aeronautics and Astronautics; MIIT Key Laboratory of Pattern Analysis and Machine Intelligence"},{"name":"Songcan Chen","slug":"songcan-chen","given":"Songcan","family":"Chen","affiliation":"College of Computer Science and Technology , Nanjing University of Aeronautics and Astronautics; MIIT Key Laboratory of Pattern Analysis and Machine Intelligence"}],"abstract":"In reality, it is laborious to obtain complete label degrees, giving birth to Incomplete Label Distribution Learning (InLDL), where some degrees are missing. Existing InLDL methods often assume that degrees are uniformly random missing. However, it is often not the case in practice, which arises the first issue. Besides, they often adopt explicit regularization to compensate the incompleteness, leading to burdensome parameter tuning and extra computation, causing the second issue. To address the first issue, we adopt a more practical setting, i.e., small degrees are more prone to be missing, since large degrees are likely to catch more attention. To tackle the second issue, we argue that label distribution itself already contains abundant knowledge, such as label correlation and ranking order, thus it may have provided enough prior for learning. It is precisely because existing methods overlook such a prior that leads to the forced adoption of explicit regularization. By directly utilizing the label degrees prior, we design a properly weighted objective function, exempting the need from explicit regularization. Moreover, we provide rigorous theoretical analysis, revealing in principle that the weighting plays an implicit regularization role. To sum up, our method has four advantages, it is 1) model selection free; 2) with closed-form solution (sub-problem) and easy-to-implement (a few lines of codes); 3) with linear computational complexity in the number of samples, thus scalable to large datasets; 4) competitive with state-of-the-arts in both random and non-random missing scenarios.","doi":"10.24963/ijcai.2024/494","url":"https://ijcai.org/proceedings/2024/494","pdf":"https://ijcai.org/proceedings/2024/0494.pdf","markdown":"https://ijcai.org/proceedings/2024/494.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/494","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":494,"pages":{"first":4470,"last":4478},"keywords":["Machine Learning: ML: Weakly supervised learning","Machine Learning: ML: Multi-label learning"],"videos":[],"references":[{"position":1,"raw":"[Bartlett and Mendelson, 2002] Peter L Bartlett and Shahar Mendelson. Rademacher and gaussian complexities: Risk bounds and structural results. Journal of Machine Learn- ing Research, 3(Nov):463-482, 2002.","title":"Rademacher and gaussian complexities: Risk bounds and structural results","authors":"Mendelson ; Bartlett, L Peter, Shahar Bartlett, Mendelson","year":"2002","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Boyd et al., 2011] Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. Distributed op- timization and statistical learning via the alternating direc- tion method of multipliers. Foundations and Trends® in Machine learning, 3(1):1-122, 2011.","title":"Distributed optimization and statistical learning via the alternating direction method of multipliers","authors":"Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein","year":"2011","venue":"Foundations and Trends® in Machine learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2020] Shikai Chen, Jianfeng Wang, Yuedong Chen, Zhongchao Shi, Xin Geng, and Yong Rui. Label dis- tribution learning on auxiliary label space graphs for facial expression recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13984-13993, 2020.","title":"Label distribution learning on auxiliary label space graphs for facial expression recognition","authors":"Shikai Chen, Jianfeng Wang, Yuedong Chen, Zhongchao Shi, Xin Geng, Yong Rui","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Condat, 2016] Laurent Condat. Fast projection onto the simplex and the l 1 ball. Mathematical Programming, 158(1-2):575-585, 2016.","title":"Fast projection onto the simplex and the l 1 ball","authors":"Laurent Condat","year":"2016","venue":"Mathematical Programming","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Della Pietra et al., 1997] Stephen Della Pietra, Vincent Della Pietra, and John Lafferty. Inducing features of ran- dom fields. IEEE transactions on pattern analysis and ma- chine intelligence, 19(4):380-393, 1997.","title":"Inducing features of random fields","authors":"Stephen Della, Pietra, Vincent Della, Pietra, John Lafferty","year":"1997","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Demšar, 2006] Janez Demšar. Statistical comparisons of classifiers over multiple data sets. The Journal of Machine learning research, 7:1-30, 2006.","title":"Janez Demšar. Statistical comparisons of classifiers over multiple data sets","authors":"Demšar","year":"2006","venue":"The Journal of Machine learning research","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fletcher, 2013] Roger Fletcher. Practical methods of opti- mization. John Wiley & Sons, 2013.","title":"Practical methods of optimization","authors":"Roger Fletcher","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Foster and Rakhlin, 2019] Dylan J Foster and Alexander Rakhlin. ℓ ∞ vector contraction for rademacher complex- ity. arXiv preprint arXiv:1911.06468, 6, 2019.","title":"ℓ ∞ vector contraction for rademacher complexity","authors":"J Dylan, Alexander Foster, Rakhlin","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gao et al., 2018] Bin-Bin Gao, Hong-Yu Zhou, Jianxin Wu, and Xin Geng. Age estimation using expectation of la- bel distribution learning. In Proceedings of the Interna- tional Joint Conference on Artificial Intelligence, pages 712-718, 2018.","title":"Age estimation using expectation of label distribution learning","authors":"Gao","year":"2018","venue":"Proceedings of the International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2018/99"},{"position":10,"raw":"[Geng et al., 2021] Xin Geng, Renyi Zheng, Jiaqi Lv, and Yu Zhang. Multilabel ranking with inconsistent rankers. IEEE Transactions on Pattern Analysis and Machine In- telligence, 44(9):5211-5224, 2021.","title":"Multilabel ranking with inconsistent rankers","authors":"Xin Geng, Renyi Zheng, Jiaqi Lv, Yu Zhang","year":"2021","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Geng, 2016] Xin Geng. Label distribution learning. IEEE Transactions on Knowledge and Data Engineering, 28(7):1734-1748, 2016.","title":"Label distribution learning","authors":"Xin Geng, Geng","year":"2016","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hou et al., 2017] Peng Hou, Xin Geng, Zeng-Wei Huo, and Jia-Qi Lv. Semi-supervised adaptive label distribution learning for facial age estimation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.","title":"Semi-supervised adaptive label distribution learning for facial age estimation","authors":"Peng Hou, Xin Geng, Zeng-Wei Huo, Jia-Qi Lv","year":"2017","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jia et al., 2019a] Xiuyi Jia, Tingting Ren, Lei Chen, Jun Wang, Jihua Zhu, and Xianzhong Long. Weakly super- vised label distribution learning based on transductive ma- trix completion with sample correlations. Pattern Recog- nition Letters, 125:453-462, 2019.","title":"Weakly supervised label distribution learning based on transductive matrix completion with sample correlations","authors":"Jia","year":"2019","venue":"Pattern Recognition Letters","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jia et al., 2019b] Xiuyi Jia, Xiang Zheng, Weiwei Li, Changqing Zhang, and Zechao Li. Facial emotion distri- bution learning by exploiting low-rank label correlations locally. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9841- 9850, 2019.","title":"Facial emotion distribution learning by exploiting low-rank label correlations locally","authors":"Jia","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jia et al., 2021] Xiuyi Jia, Tao Wen, Weiping Ding, Huax- iong Li, and Weiwei Li. Semi-supervised label distribution learning via projection graph embedding. Information Sci- ences, 581:840-855, 2021.","title":"Semi-supervised label distribution learning via projection graph embedding","authors":"Xiuyi Jia, Tao Wen, Weiping Ding, Huaxiong Li, Weiwei Li","year":"2021","venue":"Information Sciences","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Koltchinskii, 2001] Vladimir Koltchinskii. Rademacher penalties and structural risk minimization. IEEE Trans- actions on Information Theory, 47(5):1902-1914, 2001.","title":"Rademacher penalties and structural risk minimization","authors":"Vladimir Koltchinskii","year":"2001","venue":"IEEE Transactions on Information Theory","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li and Deng, 2019] Shan Li and Weihong Deng. Blended emotion in-the-wild: Multi-label facial expression recog- nition using crowdsourced annotations and deep locality feature learning. International Journal of Computer Vi- sion, 127(6-7):884-906, 2019.","title":"Blended emotion in-the-wild: Multi-label facial expression recognition using crowdsourced annotations and deep locality feature learning","authors":"Shan Li, Weihong Deng","year":"2019","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2022] Weiwei Li, Jin Chen, Yuqing Lu, and Zhiqiu Huang. Filling missing labels in label distribution learn- ing by exploiting label-specific feature selection. In 2022 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE, 2022.","title":"Filling missing labels in label distribution learning by exploiting label-specific feature selection","authors":"Weiwei Li, Jin Chen, Yuqing Lu, Zhiqiu Huang","year":"2022","venue":"International Joint Conference on Neural Networks (IJCNN)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liang et al., 2018] Lingyu Liang, Luojun Lin, Lianwen Jin, Duorui Xie, and Mengru Li. Scut-fbp5500: A diverse benchmark dataset for multi-paradigm facial beauty pre- diction. In 2018 24th International conference on pattern recognition (ICPR), pages 1598-1603. IEEE, 2018.","title":"Scut-fbp5500: A diverse benchmark dataset for multi-paradigm facial beauty prediction","authors":"Lingyu Liang, Luojun Lin, Lianwen Jin, Duorui Xie, Mengru Li","year":"2018","venue":"24th International conference on pattern recognition (ICPR)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lu et al., 2023] Yunan Lu, Weiwei Li, Huaxiong Li, and Xiuyi Jia. Predicting label distribution from tie-allowed multi-label ranking. IEEE Transactions on Pattern Analy- sis and Machine Intelligence, 2023.","title":"Predicting label distribution from tie-allowed multi-label ranking","authors":"Yunan Lu, Weiwei Li, Huaxiong Li, Xiuyi Jia","year":"2023","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Maurer, 2016] Andreas Maurer. A vector-contraction in- equality for rademacher complexities. In Algorithmic Learning Theory: 27th International Conference, ALT 2016, Bari, Italy, October 19-21, 2016, Proceedings 27, pages 3-17. Springer, 2016.","title":"A vector-contraction inequality for rademacher complexities","authors":"Andreas Maurer","year":"2016","venue":"Algorithmic Learning Theory: 27th International Conference, ALT 2016","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[McDiarmid and others, 1989] Colin McDiarmid et al. On the method of bounded differences. Surveys in combina- torics, 141(1):148-188, 1989.","title":"On the method of bounded differences","authors":"Colin Mcdiarmid","year":"1989","venue":"Surveys in combinatorics","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Mohri et al., 2018] Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning. MIT press, 2018.","title":"Foundations of machine learning","authors":"Mehryar Mohri, Afshin Rostamizadeh, Ameet Talwalkar","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Nemenyi, 1963] Peter Bjorn Nemenyi. Distribution-free multiple comparisons. Princeton University, 1963.","title":"Distribution-free multiple comparisons","authors":"Bjorn Peter, Nemenyi","year":"1963","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Peng et al., 2015] Kuan-Chuan Peng, Tsuhan Chen, Amir Sadovnik, and Andrew C Gallagher. A mixed bag of emo- tions: Model, predict, and transfer emotion distributions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 860-868, 2015.","title":"A mixed bag of emotions: Model, predict, and transfer emotion distributions","authors":"Kuan-Chuan, Tsuhan Peng, Amir Chen, Andrew C Sadovnik, Gallagher","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Qian et al., 2022a] Wenbin Qian, Ping Dong, Shiming Dai, Jintao Huang, and Yinglong Wang. Incomplete la- bel distribution feature selection based on neighborhood- tolerance discrimination index. Applied Soft Computing, 130:109693, 2022.","title":"Incomplete label distribution feature selection based on neighborhoodtolerance discrimination index","authors":"Qian","year":"2022","venue":"Applied Soft Computing","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Qian et al., 2022b] Wenbin Qian, Ping Dong, Yinglong Wang, Shiming Dai, and Jintao Huang. Local rough set-based feature selection for label distribution learning with incomplete labels. International Journal of Machine Learning and Cybernetics, 13(8):2345-2364, 2022.","title":"Local rough set-based feature selection for label distribution learning with incomplete labels","authors":"Qian","year":"2022","venue":"International Journal of Machine Learning and Cybernetics","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Rupprecht et al., 2017] Christian Rupprecht, Iro Laina, Robert DiPietro, Maximilian Baust, Federico Tombari, Nassir Navab, and Gregory D Hager. Learning in an uncer- tain world: Representing ambiguity through multiple hy- potheses. In Proceedings of the IEEE International Con- ference on Computer Vision, pages 3591-3600, 2017.","title":"Learning in an uncertain world: Representing ambiguity through multiple hypotheses","authors":"Rupprecht","year":"2017","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Shalev-Shwartz and Ben-David, 2014] Shai Shalev- Shwartz and Shai Ben-David. Understanding machine learning: From theory to algorithms. Cambridge university press, 2014.","title":"Understanding machine learning: From theory to algorithms","authors":"Shai Shalev, -Shwartz, Shai Ben-David","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Teng and Jia, 2021] Qifa Teng and Xiuyi Jia. Incomplete la- bel distribution learning by exploiting global sample cor- relation. In Multimedia Understanding with Less Labeling on Multimedia Understanding with Less Labeling, pages 9-16. 2021.","title":"Incomplete label distribution learning by exploiting global sample correlation","authors":"Jia Teng, Qifa Teng, Xiuyi Jia","year":"2021","venue":"Multimedia Understanding with Less Labeling on Multimedia Understanding with Less Labeling","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang and Carreira-Perpinán, 2013] Weiran Wang and Miguel A Carreira-Perpinán. Projection onto the proba- bility simplex: An efficient algorithm with a simple proof, and an application. arXiv preprint arXiv:1309.1541, 2013.","title":"Projection onto the probability simplex: An efficient algorithm with a simple proof, and an application","authors":"Wang, Carreira-Perpinán ; Weiran Wang, Miguel A Carreira-Perpinán","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wang and Geng, 2023] Jing Wang and Xin Geng. Label distribution learning by exploiting label distribution man- ifold. IEEE Transactions on Neural Networks and Learn- ing Systems, 34(2):839-852, 2023.","title":"Label distribution learning by exploiting label distribution manifold","authors":"Jing Wang, Xin Geng","year":"2023","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wang et al., 2022] Jing Wang, Xin Geng, and Hui Xue. Re- weighting large margin label distribution learning for clas- sification. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 44(9):5445-5459, 2022.","title":"Reweighting large margin label distribution learning for classification","authors":"Jing Wang, Xin Geng, Hui Xue","year":"2022","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Xie et al., 2015] Duorui Xie, Lingyu Liang, Lianwen Jin, Jie Xu, and Mengru Li. Scut-fbp: A benchmark dataset for facial beauty perception. In 2015 IEEE International Con- ference on Systems, Man, and Cybernetics, pages 1821- 1826. IEEE, 2015.","title":"Scut-fbp: A benchmark dataset for facial beauty perception","authors":"Duorui Xie, Lingyu Liang, Lianwen Jin, Jie Xu, Mengru Li","year":"2015","venue":"IEEE International Conference on Systems, Man, and Cybernetics","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Xu and Zhou, 2017] Miao Xu and Zhi-Hua Zhou. Incom- plete label distribution learning. In International Joint Conferences on Artificial Intelligence, pages 3175-3181, 2017.","title":"Incomplete label distribution learning","authors":"Miao Xu, Zhi-Hua Zhou","year":"2017","venue":"International Joint Conferences on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/443"},{"position":36,"raw":"[Yang et al., 2017] Jufeng Yang, Ming Sun, and Xiaoxiao Sun. Learning visual sentiment distributions via aug- mented conditional probability neural network. In Pro- ceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.","title":"Learning visual sentiment distributions via augmented conditional probability neural network","authors":"Jufeng Yang, Ming Sun, Xiaoxiao Sun","year":"2017","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhang et al., 2022] Jing Zhang, Hong Tao, Tingjin Luo, and Chenping Hou. Safe incomplete label distribution learn- ing. Pattern Recognition, 125:108518, 2022.","title":"Safe incomplete label distribution learning","authors":"Jing Zhang, Hong Tao, Tingjin Luo, Chenping Hou","year":"2022","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}