{"type":"ScholarlyArticle","title":"Posistive-Unlabeled Learning via Optimal Transport and Margin Distribution","authors":[{"name":"Nan Cao","slug":"nan-cao","given":"Nan","family":"Cao","affiliation":"National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab, Cluster and Grid Computing Lab , School of Computer Science and Technology , Huazhong University of Science and Technology , China"},{"name":"Teng Zhang","slug":"teng-zhang","given":"Teng","family":"Zhang","affiliation":"National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab, Cluster and Grid Computing Lab , School of Computer Science and Technology , Huazhong University of Science and Technology , China"},{"name":"Xuanhua Shi","slug":"xuanhua-shi","given":"Xuanhua","family":"Shi","affiliation":"National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab, Cluster and Grid Computing Lab , School of Computer Science and Technology , Huazhong University of Science and Technology , China"},{"name":"Hai Jin","slug":"hai-jin","given":"Hai","family":"Jin","affiliation":"National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab, Cluster and Grid Computing Lab , School of Computer Science and Technology , Huazhong University of Science and Technology , China"}],"abstract":"Positive-unlabeled (PU) learning deals with the circumstances where only a portion of positive instances are labeled, while the rest and all negative instances are unlabeled, and due to this confusion, the class prior can not be directly available. Existing PU learning methods usually estimate the class prior by training a nontraditional probabilistic classifier, which is prone to give an overestimation. Moreover, these methods learn the decision boundary by optimizing the minimum margin, which is not suitable in PU learning due to its sensitivity to label noise. In this paper, we enhance PU learning methods from the above two aspects. More specifically, we first explicitly learn a transformation from unlabeled data to positive data by entropy regularized optimal transport to achieve a much more precise estimation for class prior. Then we switch to optimizing the margin distribution, rather than the minimum margin, to obtain a label noise insensitive classifier. Extensive empirical studies on both synthetic and real-world data sets demonstrate the superiority of our proposed method.","doi":"10.24963/ijcai.2022/393","url":"https://ijcai.org/proceedings/2022/393","pdf":"https://ijcai.org/proceedings/2022/0393.pdf","markdown":"https://ijcai.org/proceedings/2022/393.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/393","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":393,"pages":{"first":2836,"last":2842},"keywords":["Machine Learning: Weakly Supervised Learning","Machine Learning: Classification"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987021","length_sec":91,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985164","length_sec":643,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bekker and Davis, 2017] Jessa Bekker and Jesse Davis. Positive and unlabeled relational classification through la- bel frequency estimation. In ILP, pages 16-30, 2017.","title":"Positive and unlabeled relational classification through label frequency estimation","authors":"Davis ; Bekker, Jessa Bekker, Jesse Davis","year":"2017","venue":"ILP","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bekker and Davis, 2020] Jessa Bekker and Jesse Davis. Learning from positive and unlabeled data: A survey. Mach. Learn., 109(4):719-760, 2020.","title":"Learning from positive and unlabeled data: A survey","authors":"Davis ; Bekker, Jessa Bekker, Jesse Davis","year":"2020","venue":"Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Blondel et al., 2018] Mathieu Blondel, Vivien Seguy, and Antoine Rolet. Smooth and sparse optimal transport. In AISTATS, pages 880-889, 2018.","title":"Smooth and sparse optimal transport","authors":"Mathieu Blondel, Vivien Seguy, Antoine Rolet","year":"2018","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cao et al., 2021] Nan Cao, Teng Zhang, and Hai Jin. Partial multi-label optimal margin distribution machine. In IJCAI, pages 2198-2204, 2021.","title":"Partial multi-label optimal margin distribution machine","authors":"Nan Cao, Teng Zhang, Hai Jin","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/303"},{"position":5,"raw":"[Chang et al., 2021] Shizhen Chang, Bo Du, and Liangpei Zhang. Positive unlabeled learning with class-prior ap- proximation. In IJCAI, pages 2014-2021, 2021.","title":"Positive unlabeled learning with class-prior approximation","authors":"Shizhen Chang, Bo Du, Liangpei Zhang","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/279"},{"position":6,"raw":"[Chaudhari and Shevade, 2012] Sneha Chaudhari and Shirish Shevade. Learning from positive and unlabelled examples using maximum margin clustering. In ICONIP, pages 465-473, 2012.","title":"Learning from positive and unlabelled examples using maximum margin clustering","authors":"Sneha Chaudhari, Shirish Shevade","year":"2012","venue":"ICONIP","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cuturi, 2013] Marco Cuturi. Sinkhorn distances: Light- speed computation of optimal transport. In NIPS 26, pages 2292-2300, 2013.","title":"Sinkhorn distances: Lightspeed computation of optimal transport","authors":"Marco Cuturi","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Du Plessis et al., 2015a] Marthinus Du Plessis, Gang Niu, and Masashi Sugiyama. Class-prior estimation for learn- ing from positive and unlabeled data. In ACML, pages 221-236, 2015.","title":"Class-prior estimation for learning from positive and unlabeled data","authors":"Du Plessis","year":"2015","venue":"ACML","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Du Plessis et al., 2015b] Marthinus Du Plessis, Gang Niu, and Masashi Sugiyama. Convex formulation for learning from positive and unlabeled data. In ICML, pages 1386- 1394, 2015.","title":"Convex formulation for learning from positive and unlabeled data","authors":"Du Plessis","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Du Plessis Marthinus and Sugiyama, 2014] Christoffel Du Plessis Marthinus and Masashi Sugiyama. Class prior estimation from positive and unlabeled data. IEICE Trans. Inf. Syst., 97(5):1358-1362, 2014.","title":"Class prior estimation from positive and unlabeled data","authors":"Du Plessis, Marthinus, Sugiyama ; Christoffel Du, Plessis Marthinus, Masashi Sugiyama","year":"2014","venue":"IEICE Trans. Inf. Syst","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Elkan and Noto, 2008] Charles Elkan and Keith Noto. Learning classifiers from only positive and unlabeled data. In SIGKDD, pages 213-220, 2008.","title":"Learning classifiers from only positive and unlabeled data","authors":"Noto Elkan, Charles Elkan, Keith Noto","year":"2008","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Frogner et al., 2015] Charlie Frogner, Chiyuan Zhang, Hos- sein Mobahi, Mauricio Araya, and Tomaso A Poggio. Learning with a wasserstein loss. In NIPS 28, pages 2053- 2061, 2015.","title":"Learning with a wasserstein loss","authors":"Charlie Frogner, Chiyuan Zhang, Hossein Mobahi, Mauricio Araya, Tomaso A Poggio","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gao and Zhou, 2013] Wei Gao and Zhi-Hua Zhou. On the doubt about margin explanation of boosting. Artif. Intell., 203:1-18, 2013.","title":"On the doubt about margin explanation of boosting","authors":"Wei Gao, Zhi-Hua Zhou","year":"2013","venue":"Artif. Intell","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ho et al., 2017] Nhat Ho, XuanLong Nguyen, Mikhail Yurochkin, Hung Hai Bui, Viet Huynh, and Dinh Phung. Multilevel clustering via wasserstein means. In ICML, pages 1501-1509, 2017.","title":"Multilevel clustering via wasserstein means","authors":"Ho","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kantorovitch, 1958] Leonid Kantorovitch. On the translo- cation of masses. Manage. Sci., 5(1):1-4, 1958.","title":"Leonid Kantorovitch. On the translocation of masses","authors":"Kantorovitch","year":"1958","venue":"Manage. Sci","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kato et al., 2018] Masahiro Kato, Takeshi Teshima, and Junya Honda. Learning from positive and unlabeled data with a selection bias. In ICLR, 2018.","title":"Learning from positive and unlabeled data with a selection bias","authors":"Masahiro Kato, Takeshi Teshima, Junya Honda","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kiryo et al., 2017] Ryuichi Kiryo, Gang Niu, Marthinus du Plessis, and Masashi Sugiyama. Positive-unlabeled learning with non-negative risk estimator. In ICONIP, pages 1674-1684, 2017.","title":"Positive-unlabeled learning with non-negative risk estimator","authors":"Ryuichi Kiryo, Gang Niu, Marthinus Du Plessis, Masashi Sugiyama","year":"2017","venue":"ICONIP","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lee and Liu, 2003] Wee Sun Lee and Bing Liu. Learning with positive and unlabeled examples using weighted lo- gistic regression. In ICML, pages 448-455, 2003.","title":"Learning with positive and unlabeled examples using weighted logistic regression","authors":"Lee, Liu ; Wee, Sun Lee, Bing Liu","year":"2003","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li and Liu, 2003] Xiaoli Li and Bing Liu. Learning to clas- sify texts using positive and unlabeled data. In IJCAI, pages 587-592, 2003.","title":"Learning to classify texts using positive and unlabeled data","authors":"Li, Liu ; ] Xiaoli Li, Bing Liu","year":"2003","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2003/87"},{"position":20,"raw":"[Liu et al., 2002] Bing Liu, Wee Sun Lee, Philip S. Yu, and Xiaoli Li. Partially supervised classification of text docu- ments. In ICML, pages 387-394, 2002.","title":"Partially supervised classification of text documents","authors":"Bing Liu, Wee Sun Lee, Philip S Yu, Xiaoli Li","year":"2002","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mordelet and Jean Philippe, 2014] Fantine Mordelet and Vert Jean Philippe. A bagging SVM to learn from positive and unlabeled examples. Pattern Recognit. Lett., 37:201- 209, 2014.","title":"Fantine Mordelet and Vert Jean Philippe. A bagging SVM to learn from positive and unlabeled examples","authors":"Jean Mordelet, Philippe","year":"2014","venue":"Pattern Recognit. Lett","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Peyré et al., 2019] Gabriel Peyré, Marco Cuturi, et al. Com- putational optimal transport: With applications to data science. Found. Trends Mach. Learn., 11(5-6):355-607, 2019.","title":"Computational optimal transport: With applications to data science","authors":"Gabriel Peyré, Marco Cuturi","year":"2019","venue":"Found. Trends Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shao et al., 2015] Yuan-Hai Shao, Wei-Jie Chen, Li-Ming Liu, and Nai-Yang Deng. Laplacian unit-hyperplane learn- ing from positive and unlabeled examples. Inf. Sci., 314:152-168, 2015.","title":"Laplacian unit-hyperplane learning from positive and unlabeled examples","authors":"Yuan-Hai Shao, Wei-Jie Chen, Li-Ming Liu, Nai-Yang Deng","year":"2015","venue":"Inf. Sci","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shi et al., 2018] Hong Shi, Shaojun Pan, Jian Yang, and Chen Gong. Positive and unlabeled learning via loss de- composition and centroid estimation. In IJCAI, pages 2689-2695, 2018.","title":"Positive and unlabeled learning via loss decomposition and centroid estimation","authors":"Shi","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/373"},{"position":25,"raw":"[Song et al., 2017] Kun Song, Feiping Nie, Junwei Han, and Xuelong Li. Parameter free large margin nearest neighbor for distance metric learning. In AAAI, pages 2555-2561, 2017.","title":"Parameter free large margin nearest neighbor for distance metric learning","authors":"Kun Song, Feiping Nie, Junwei Han, Xuelong Li","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tao et al., 2008] Peng Tao, Wanli Zuo, and Fengling He. SVM based adaptive learning method for text classifica- tion from positive and unlabeled documents. Knowl. Inf. Syst., 16(3):281-301, 2008.","title":"SVM based adaptive learning method for text classification from positive and unlabeled documents","authors":"Peng Tao, Wanli Zuo, Fengling He","year":"2008","venue":"Knowl. Inf. Syst","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Titouan et al., 2019] Vayer Titouan, Nicolas Courty, Ro- main Tavenard, and Rémi Flamary. Optimal transport for structured data with application on graphs. In ICML, pages 6275-6284, 2019.","title":"Optimal transport for structured data with application on graphs","authors":"Nicolas Vayer Titouan, Romain Courty, Rémi Tavenard, Flamary","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang and Jin, 2021] Teng Zhang and Hai Jin. Optimal margin distribution machine for multi-instance learning. In IJCAI, pages 2383-2389, 2021.","title":"Optimal margin distribution machine for multi-instance learning","authors":"Teng Zhang, Hai Jin","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/330"},{"position":29,"raw":"[Zhang and Zhou, 2014] Teng Zhang and Zhi-Hua Zhou. Large margin distribution machine. In SIGKDD, pages 313-322, 2014.","title":"Large margin distribution machine","authors":"Teng Zhang, Zhi-Hua Zhou","year":"2014","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang and Zhou, 2017] Teng Zhang and Zhi-Hua Zhou. Multi-class optimal margin distribution machine. In ICML, pages 4063-4071, 2017.","title":"Multi-class optimal margin distribution machine","authors":"Teng Zhang, Zhi-Hua Zhou","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhang and Zhou, 2018a] Teng Zhang and Zhi-Hua Zhou. Optimal margin distribution clustering. In AAAI, pages 4474-4481, 2018.","title":"Optimal margin distribution clustering","authors":"Zhou Zhang, Teng Zhang, Zhi-Hua Zhou","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhang and Zhou, 2018b] Teng Zhang and Zhi-Hua Zhou. Semi-supervised optimal margin distribution machines. In IJCAI, pages 3104-3110, 2018.","title":"Semi-supervised optimal margin distribution machines","authors":"Zhou Zhang, Teng Zhang, Zhi-Hua Zhou","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/431"},{"position":33,"raw":"[Zhang and Zhou, 2019] Teng Zhang and Zhi-Hua Zhou. Optimal margin distribution machine. IEEE Trans. Knowl. Data Eng., 32(6):1143-1156, 2019.","title":"Optimal margin distribution machine","authors":"Teng Zhang, Zhi-Hua Zhou","year":"2019","venue":"IEEE Trans. Knowl. Data Eng","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhang et al., 2019] Chuang Zhang, Dexin Ren, Tongliang Liu, Jian Yang, and Chen Gong. Positive and unlabeled learning with label disambiguation. In IJCAI, pages 4250- 4256, 2019.","title":"Positive and unlabeled learning with label disambiguation","authors":"Chuang Zhang, Dexin Ren, Tongliang Liu, Jian Yang, Chen Gong","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/590"},{"position":35,"raw":"[Zhang et al., 2020] Teng Zhang, Peng Zhao, and Hai Jin. Optimal margin distribution learning in dynamic environ- ments. In AAAI, pages 6821-6828, 2020.","title":"Optimal margin distribution learning in dynamic environments","authors":"Teng Zhang, Peng Zhao, Hai Jin","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}