{"type":"ScholarlyArticle","title":"Classification with Label Distribution Learning","authors":[{"name":"Jing Wang","slug":"jing-wang","given":"Jing","family":"Wang","affiliation":"MOE Key Laboratory of Computer Network and Information Integration School of Computer Science and Engineering , Southeast University , Nanjing 210096 , China"},{"name":"Xin Geng","slug":"xin-geng","given":"Xin","family":"Geng","affiliation":"MOE Key Laboratory of Computer Network and Information Integration School of Computer Science and Engineering , Southeast University , Nanjing 210096 , China"}],"abstract":"Label Distribution Learning (LDL) is a novel learning paradigm, aim of which is to minimize the distance between the model output and the ground-truth label distribution. We notice that, in real-word applications, the learned label distribution model is generally treated as a classification model, with the label corresponding to the highest model output as the predicted label, which unfortunately prompts an inconsistency between the training phrase and the test phrase. To solve the inconsistency, we propose in this paper a new Label Distribution Learning algorithm for Classification (LDL4C). Firstly, instead of KL-divergence, absolute loss is applied as the measure for LDL4C. Secondly, samples are re-weighted with information entropy. Thirdly, large margin classifier is adapted to boost discrimination precision. We then reveal that theoretically LDL4C seeks a balance between generalization and discrimination. Finally, we compare LDL4C with existing LDL algorithms on 17 real-word datasets, and experimental results demonstrate the effectiveness of LDL4C in classification.","doi":"10.24963/ijcai.2019/515","url":"https://ijcai.org/proceedings/2019/515","pdf":"https://ijcai.org/proceedings/2019/0515.pdf","markdown":"https://ijcai.org/proceedings/2019/515.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/515","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":515,"pages":{"first":3712,"last":3718},"keywords":["Machine Learning: Multi-instance;Multi-label;Multi-view learning","Machine Learning: Classification"],"videos":[],"references":[{"position":1,"raw":"PT-Bayes PT-SVM AA-BP AA-kNN LDL-SVR StructRF LDL-SCL BFGS-LDL LDL4C SJAFFE .783±.016• .785±.003• .807±.003• .483±.012• .686±.004• .569±.011• .789±.002• .437±.018 .394±.010","title":"PT-Bayes PT-SVM AA-BP AA-kNN LDL-SVR StructRF LDL-SCL BFGS-LDL LDL4","authors":null,"year":null,"venue":"C SJAFFE","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"M 2 B .645±.003• .552±.003• .487±.001 .484±.001 .495±.002 .483±.002 .498±.001 .486±.002 .481±.001","title":"M 2 B","authors":null,"year":null,"venue":"M 2 B","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"SCUT-FBP .893±.002• .517±.001 .463±.001 .457±.001 .467±.001 .467±.001 .502±.001• .467±.000 .465±.001 Natural Scene .801±.002• .649±.000• .667±.001• .521±.001• .568±.001• .520±.001• .696±.001• .597±.002• .420±.001","title":"467±.000","authors":null,"year":null,"venue":"Natural Scene","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"Yeast-alpha .945±.000• .940±.000• .886±.000• .882±.000• .925±.000• .882±.001• .903±.000• .891±.000• .787±.001","title":"Yeast-alpha","authors":null,"year":null,"venue":"Yeast-alpha","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"Yeast-cdc .935±.000• .926±.000• .825±.001 .830±.001• .858±.000• .824±.001 .825±.000 .823±.001 .818±.001","title":"Yeast-cdc","authors":null,"year":"0935","venue":"Yeast-cdc","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"Yeast-cold .730±.000• .733±.001• .580±.001 .570±.001• .609±.000• .585±.001• .586±.001• .580±.001 .575±.001","title":"Yeast-cold","authors":null,"year":null,"venue":"Yeast-cold","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"Yeast-diau .866±.000• .848±.001• .711±.001• .688±.000• .694±.001• .678±.000 .703±.001• .694±.001• .665±.000","title":"Yeast-diau","authors":null,"year":null,"venue":"Yeast-diau","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"Yeast-dtt .743±.001• .740±.001• .641±.001 .637±.001 .651±.001• .622±.001 .636±.001 .629±.001 .627±.001","title":"Yeast-dtt","authors":null,"year":null,"venue":"Yeast-dtt","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"Yeast-elu .930±.000• .924±.000• .917±.000• .867±.001• .892±.000• .875±.000• .903±.000• .899±.000• .803±.000","title":"Yeast-elu","authors":null,"year":"0930","venue":"Yeast-elu","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"Yeast-heat .824±.001• .827±.000• .707±.001• .692±.001 .710±.001• .680±.001 .717±.001• .703±.001• .675±.001","title":"Yeast-heat","authors":null,"year":null,"venue":"Yeast-heat","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"Yeast-spo .826±.001• .817±.001• .548±.000 .565±.001• .616±.000• .566±.001• .584±.000• .548±.000 .547±.000","title":"Yeast-spo","authors":null,"year":null,"venue":"Yeast-spo","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"Yeast-spo5 .667±.001• .637±.000• .543±.001 .542±.000 .559±.001 .524±.001 .622±.000• .559±.001• .534±.001","title":"Yeast-spo5","authors":null,"year":null,"venue":"Yeast-spo5","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"Yeast-spoem .482±.000• .479±.001• .441±.001• .410±.001 .409±.000 .401±.002 .439±.002• .435±.002• .401±.000","title":"Yeast-spoem","authors":null,"year":null,"venue":"Yeast-spoem","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"SBU 3DFE .818±.001• .810±.001• .675±.001• .616±.001• .633±.002• .516±.001• .514±.001• .582±.001 .569±.001 Movie .819±.000• .677±.000 • .423±.001• .427±.000• .415±.000• .410±.000 .454±.000• .416±.000• .409±.000 Human Gene .986±.000• .981±.000• .951±.001• .966±.000• .977±.000• .965±.000• .976±.000• .955±.000• .927±.000 Table 3: Experimental results (mean ± std.) of LDL4C and comparing algorithms in terms of 0/1 loss on 17 real-word datasets. In addition, •/• indicates whether LDL4C is statistically superior/inferior to the comparing algorithms. References [Bartlett and Mendelson, 2003] Peter L. Bartlett and Shahar Mendelson. Rademacher and gaussian complexities: Risk bounds and structural results. J. Mach. Learn. Res., 3:463- 482, March 2003.","title":"Experimental results (mean ± std.) of LDL4C and comparing algorithms in terms of 0/1 loss on 17 real-word datasets. In addition, •","authors":null,"year":"2003","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Chen et al., 2018] Mengting Chen, Xinggang Wang, Bin Feng, and Wenyu Liu. Structured random forest for la- bel distribution learning. Neurocomputing, 320:171 -182, 2018.","title":"Structured random forest for la-bel distribution learning","authors":"Chen","year":"2018","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Cover and Thomas, 2012] Thomas M Cover and Joy A Thomas. Elements of information theory. John Wiley & Sons, 2012. [Devroye et al., 1996] Luc Devroye, László Györfi, and Gábor Lugosi. A probabilistic theory of pattern recog- nition, volume 31. Springer Science & Business Media, 1996.","title":"Elements of information theory","authors":"M Thomas, Joy A Cover, Thomas ; Devroye","year":"1996","venue":"A probabilistic theory of pattern recognition","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Gao et al., 2017] Binbin Gao, Chao Xing, Chenwei Xie, Jianxin Wu, and Xin Geng. Deep label distribution learn- ing with label ambiguity. IEEE Transactions on Image Processing, 26(6):2825-2838, June 2017.","title":"Deep label distribution learning with label ambiguity","authors":"Binbin Gao, Chao Xing, Chenwei Xie, Jianxin Wu, Xin Geng","year":"2017","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Gao et al., 2018] Binbin Gao, Hongyu Zhou, Jianxin Wu, and Xin Geng. Age estimation using expectation of la- bel distribution learning. In Proceedings of the Twenty- Seventh International Joint Conference on Artificial In- telligence, IJCAI-18, pages 712-718. International Joint Conferences on Artificial Intelligence Organization, 7 2018.","title":"Age estimation using expectation of label distribution learning","authors":"Binbin Gao, Hongyu Zhou, Jianxin Wu, Xin Geng","year":"2018","venue":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18","doi":null,"url":null,"internal_url":"/proceedings/2018/99"},{"position":19,"raw":"[Geng and Hou, 2015] Xin Geng and Peng Hou. Pre-release prediction of crowd opinion on movies by label distribu- tion learning. In Proceedings of the 24th International Conference on Artificial Intelligence, IJCAI'15, pages 3511-3517, 2015.","title":"Pre-release prediction of crowd opinion on movies by label distribution learning","authors":"Hou ; ] Xin Geng, Peng Geng, Hou","year":"2015","venue":"Proceedings of the 24th International Conference on Artificial Intelligence, IJCAI'15","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Geng et al., 2013] Xin Geng, Chao Yin, and Zhihua Zhou. Facial age estimation by learning from label distributions. IEEE Transactions on Pattern Analysis and Machine In- telligence, 35(10):2401-2412, Oct 2013.","title":"Facial age estimation by learning from label distributions","authors":"Xin Geng, Zhihua Chao Yin, Zhou","year":"2013","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Geng, 2016] Xin Geng. Label distribution learning. IEEE Transactions on Knowledge and Data Engineering, 28(7):1734-1748, July 2016.","title":"Label distribution learning","authors":"Xin Geng, Geng","year":"2016","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Huo and Geng, 2017] Zengwei Huo and Xin Geng. Ordi- nal zero-shot learning. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pages 1916-1922, 2017.","title":"Zengwei Huo and Xin Geng. Ordinal zero-shot learning","authors":"Geng Huo","year":"2017","venue":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Kakade et al., 2009] Sham M Kakade, Karthik Sridharan, and Ambuj Tewari. On the complexity of linear predic- tion: Risk bounds, margin bounds, and regularization. In Advances in Neural Information Processing Systems, NIPS'09, pages 793-800, 2009.","title":"On the complexity of linear prediction: Risk bounds, margin bounds, and regularization","authors":"Karthik Sham M Kakade, Ambuj Sridharan, Tewari","year":"2009","venue":"Advances in Neural Information Processing Systems, NIPS'09","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Maurer, 2016] Andreas Maurer. A vector-contraction in- equality for rademacher complexities. In Algorithmic Learning Theory, pages 3-17, 2016.","title":"A vector-contraction inequality for rademacher complexities","authors":"Andreas Maurer","year":"2016","venue":"Algorithmic Learning Theory","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Mohri et al., 2012] Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning. MIT press, 2012.","title":"Foundations of machine learning","authors":"Mehryar Mohri, Afshin Rostamizadeh, Ameet Talwalkar","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Nguyen et al., 2012] Tam V. Nguyen, Si Liu, Bingbing Ni, Jun Tan, Yong Rui, and Shuicheng Yan. Sense beauty via face, dressing, and/or voice. In Proceedings of the 20th ACM International Conference on Multimedia, MM '12, pages 239-248, New York, USA, 2012. ACM.","title":"Sense beauty via face, dressing, and/or voice","authors":"V Tam, Si Nguyen, Bingbing Liu, Jun Ni, Yong Tan, Shuicheng Rui, Yan","year":"2012","venue":"Proceedings of the 20th ACM International Conference on Multimedia, MM '12","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Nocedal and Wright, 2006] Jorge Nocedal and Stephen Wright. Numerical optimization. Springer Science & Business Media, 2006.","title":"Numerical optimization","authors":"Wright ; Nocedal, Jorge Nocedal, Stephen Wright","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Ren and Geng, 2017] Yi Ren and Xin Geng. Sense beauty by label distribution learning. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pages 2648-2654, 2017.","title":"Sense beauty by label distribution learning","authors":"Geng ; ] Ren, Xin Yi Ren, Geng","year":"2017","venue":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17","doi":null,"url":null,"internal_url":"/proceedings/2017/369"},{"position":29,"raw":"[Shen et al., 2017] Wei Shen, Kai Zhao, Yilu Guo, and Alan L. Yuille. Label distribution learning forests. In Proceedings of the 30th International Conference on Neu- ral Information Processing Systems, NIPS'30, pages 834- 843.","title":"Label distribution learning forests","authors":"Wei Shen, Kai Zhao, Yilu Guo, Alan L Yuille","year":null,"venue":"Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS'30","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"Curran Associates, Inc., 2017.","title":null,"authors":"Curran Associates, Inc","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang and Geng, 2019] Jing Wang and Xin Geng. Theoret- ical analysis of label distribution learning (in press). In AAAI Conference on 33th AAAI Conference on Artificial Intelligence, AAAI'19, 2019.","title":"Theoretical analysis of label distribution learning","authors":"Wang, Geng ; ] Jing Wang, Xin Geng","year":"2019","venue":"AAAI Conference on 33th AAAI Conference on Artificial Intelligence, AAAI'19","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Xie et al., 2015] Duorui Xie, Lingyu Liang, Lianwen Jin, Jie Xu, and Mengru Li. Scut-fbp: A benchmark dataset for facial beauty perception. 2015 IEEE International Confer- ence on Systems, Man, and Cybernetics, Oct 2015.","title":"Scut-fbp: A benchmark dataset for facial beauty perception","authors":"Xie","year":"2015","venue":"IEEE International Conference on Systems, Man, and Cybernetics","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Xu et al., 2018] Ning Xu, An Tao, and Xin Geng. Label en- hancement for label distribution learning. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pages 2926-2932. Inter- national Joint Conferences on Artificial Intelligence Orga- nization, 7 2018.","title":"Label enhancement for label distribution learning","authors":"Ning Xu, An Tao, Xin Geng","year":"2018","venue":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18","doi":null,"url":null,"internal_url":"/proceedings/2018/406"},{"position":34,"raw":"[Zhang and Zhou, 2014] Minling Zhang and Zhihua Zhou. A review on multi-label learning algorithms. IEEE Trans- actions on Knowledge and Data Engineering, 26(8):1819- 1837, Aug 2014.","title":"A review on multi-label learning algorithms","authors":"Zhou Zhang, Minling Zhang, Zhihua Zhou","year":"2014","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhang et al., 2015] Zhaoxiang Zhang, Wang Mo, and Geng Xin. Crowd counting in public video surveillance by label distribution learning. Neurocomputing, 166(1):151-163, 2015.","title":"Crowd counting in public video surveillance by label distribution learning","authors":"Zhang","year":"2015","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zheng et al., 2018] Xiang Zheng, Xiuyi Jia, and Weiwei Li. Label distribution learning by exploiting sample correla- tions locally, 2018.","title":"Label distribution learning by exploiting sample correlations locally","authors":"Xiang Zheng, Xiuyi Jia, Weiwei Li","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhou and Zhang, 2006] Zhihua Zhou and Minling Zhang. Multi-instance multi-label learning with application to scene classification. In Proceedings of the 19th Interna- tional Conference on Neural Information Processing Sys- tems, NIPS'06, pages 1609-1616, Cambridge, MA, USA, 2006. MIT Press.","title":"Multi-instance multi-label learning with application to scene classification","authors":"Zhang ; ] Zhihua Zhou, Minling Zhou, Zhang","year":"2006","venue":"Proceedings of the 19th International Conference on Neural Information Processing Systems, NIPS'06","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zhou et al., 2015] Ying Zhou, Hui Xue, and Xin Geng. Emotion distribution recognition from facial expressions. In Proceedings of the 23rd ACM International Conference on Multimedia, MM '15, pages 1247-1250, New York, NY, USA, 2015. ACM.","title":"Emotion distribution recognition from facial expressions","authors":"Ying Zhou, Hui Xue, Xin Geng","year":"2015","venue":"Proceedings of the 23rd ACM International Conference on Multimedia, MM '15","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}