{"type":"ScholarlyArticle","title":"Semi-Supervised Optimal Margin Distribution Machines","authors":[{"name":"Teng Zhang","slug":"teng-zhang","given":"Teng","family":"Zhang","affiliation":"National Key Lab for Novel Software Technology , Nanjing University , Nanjing 210023 , China"},{"name":"Zhi-Hua Zhou","slug":"zhi-hua-zhou","given":"Zhi-Hua","family":"Zhou","affiliation":"National Key Lab for Novel Software Technology , Nanjing University , Nanjing 210023 , China"}],"abstract":"Semi-supervised support vector machines is an extension of standard support vector machines with unlabeled instances, and the goal is to find a label assignment of the unlabeled instances, so that the decision boundary has the maximal \\textit{minimum margin} on both the original labeled instances and unlabeled instances. Recent studies, however, disclosed that maximizing the minimum margin does not necessarily lead to better performance, and instead, it is crucial to optimize the \\textit{margin distribution}. In this paper, we propose a novel approach SODM (Semi-supervised Optimal margin Distribution Machine), which tries to assign the label to unlabeled instances and achieve optimal margin distribution simultaneously. Specifically, we characterize the margin distribution by the first- and second-order statistics, i.e., the margin mean and variance, and extend a stochastic mirror prox method to solve the resultant minimax problem. Extensive experiments on UCI data sets show that SODM is significantly better than compared methods, which verifies the superiority of optimal margin distribution learning.","doi":"10.24963/ijcai.2018/431","url":"https://ijcai.org/proceedings/2018/431","pdf":"https://ijcai.org/proceedings/2018/0431.pdf","markdown":"https://ijcai.org/proceedings/2018/431.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/431","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":431,"pages":{"first":3104,"last":3110},"keywords":["Machine Learning: Classification","Machine Learning: Semi-Supervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Beck and Teboulle, 2003] A. Beck and M. Teboulle. Mir- ror descent and nonlinear projected subgradient methods for convex optimization. Operations Research Letters, 31(3):167-175, 2003.","title":"Mirror descent and nonlinear projected subgradient methods for convex optimization","authors":"A Beck, M Teboulle","year":"2003","venue":"Operations Research Letters","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Belkin et al., 2006] M. Belkin, P. Niyogi, and V. Sind- hwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, pages 2399-2434, 2006.","title":"Manifold regularization: A geometric framework for learning from labeled and unlabeled examples","authors":"M Belkin, P Niyogi, V Sindhwani","year":"2006","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bennett and Demiriz, 1999] K. P. Bennett and A. Demiriz. Semi-supervised support vector machines. In Proceedings of the 1998 Conference on Advances in Neural Informa- tion Processing Systems II, pages 368-374, 1999.","title":"Semi-supervised support vector machines","authors":"K P Bennett, A Demiriz","year":"1999","venue":"Proceedings of the 1998 Conference on Advances in Neural Information Processing Systems II","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chapelle and Zien, 2005] O. Chapelle and A. Zien. Semi- supervised classification by low density separation. In Pro- ceedings of the 8th International Conference on Artificial Intelligence and Statistics, pages 57-64, 2005.","title":"Semisupervised classification by low density separation","authors":"O Chapelle, A Zien","year":"2005","venue":"Proceedings of the 8th International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Collobert et al., 2006] R. Collobert, F. Sinz, J. Weston, and L. Bottou. Large scale transductive svms. Journal of Ma- chine Learning Research, 7:1687-1712, 2006.","title":"Large scale transductive svms","authors":"R Collobert, F Sinz, J Weston, L Bottou","year":"2006","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cortes and Vapnik, 1995] C. Cortes and V. Vapnik. Support-vector networks. Machine Learning, 20(3):273- 297, 1995.","title":"Support-vector networks","authors":"C Cortes, V Vapnik","year":"1995","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cristianini and Shawe-Taylor, 2000] N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press, Cambridge, UK, 2000.","title":"An Introduction to Support Vector Machines and Other Kernel-based Learning Methods","authors":"N Cristianini, J Shawe-Taylor","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gao and Zhou, 2013] W. Gao and Z.-H. Zhou. On the doubt about margin explanation of boosting. Artificial Intelli- gence, 203:1-18, 2013.","title":"On the doubt about margin explanation of boosting","authors":"W Gao, Z.-H Zhou","year":"2013","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Joachims, 1999] T. Joachims. Transductive inference for text classification using support vector machines. pages 200-209.","title":"Transductive inference for text classification using support vector machines","authors":"T Joachims","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"Morgan Kaufmann, 1999.","title":null,"authors":"Morgan Kaufmann","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Johnson and Zhang, 2013] R. Johnson and T. Zhang. Accel- erating stochastic gradient descent using predictive vari- ance reduction. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 26, pages 315- 323.","title":"Accelerating stochastic gradient descent using predictive variance reduction","authors":"R Johnson, T Zhang","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"Curran Associates, Inc., 2013.","title":null,"authors":"Curran Associates, Inc","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kim and Boyd, 2008] Seung-Jean Kim and Stephen Boyd. A minimax theorem with applications to machine learn- ing, signal processing, and finance. SIAM Journal on Op- timization, 19(3):1344-1367, 2008.","title":"A minimax theorem with applications to machine learning, signal processing, and finance","authors":"Boyd ; Seung-Jean Kim, Stephen Kim, Boyd","year":"2008","venue":"SIAM Journal on Optimization","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li and Zhou, 2011] Y.-F. Li and Z.-H. Zhou. Towards mak- ing unlabeled data never hurt. In Proceedings of the 28th International Conference on International Conference on Machine Learning, pages 1081-1088, 2011.","title":"Towards making unlabeled data never hurt","authors":"Y.-F Li, Z.-H Zhou","year":"2011","venue":"Proceedings of the 28th International Conference on International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2009a] Y.-F. Li, James T. Kwok, and Z.-H. Zhou. Semi-supervised learning using label mean. In Interna- tional Conference on Machine Learning, pages 633-640, 2009.","title":"Semi-supervised learning using label mean","authors":"Li","year":"2009","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2009b] Y.-F. Li, Ivor W. Tsang, James T. Kwok, and Z.-H. Zhou. Tighter and convex maximum margin clustering. In In Proceeding of the 12th International Con- ference on Artificial Intelligence and Statistics, volume 5, pages 344-351, 2009.","title":"Tighter and convex maximum margin clustering","authors":"Li","year":"2009","venue":"Proceeding of the 12th International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2010] Y.-F. Li, J. T. Kwok, and Z.-H. Zhou. Cost- sensitive semi-supervised support vector machine. In Pro- ceedings of the 24th AAAI Conference on Artificial Intelli- gence, pages 500-505. AAAI Press, 2010.","title":"Costsensitive semi-supervised support vector machine","authors":"Y.-F Li, J T Kwok, Z.-H Zhou","year":"2010","venue":"Proceedings of the 24th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2013] Yu-Feng Li, Ivor W. Tsang, James T. Kwok, and Zhi-Hua Zhou. Convex and scalable weakly la- beled svms. Journal of Machine Learning Research, 14(1):2151-2188, jan 2013.","title":"Convex and scalable weakly labeled svms","authors":"Yu-Feng Li, Ivor W Tsang, James T Kwok, Zhi-Hua Zhou","year":"2013","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nemirovski, 2005] A. Nemirovski. Prox-Method with Rate of Convergence O(1/t) for Variational Inequalities with Lipschitz Continuous Monotone Operators and Smooth Convex-Concave Saddle Point Problems. Society for In- dustrial and Applied Mathematics, 2005.","title":"Prox-Method with Rate of Convergence O(1/t) for Variational Inequalities with Lipschitz Continuous Monotone Operators and Smooth Convex-Concave Saddle Point Problems","authors":"A Nemirovski","year":"2005","venue":"Society for Industrial and Applied Mathematics","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Schmidt et al., 2015] Mark Schmidt, Reza Babanezhad, Mohamed Ahmed, Aaron Defazio, Ann Clifton, and Anoop Sarkar. Non-uniform stochastic average gradient method for training conditional random fields. In Guy Lebanon and S. V. N. Vishwanathan, editors, Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, volume 38 of Proceedings of Machine Learning Research, pages 819-828, San Diego, CA, 09-12 May 2015. PMLR.","title":"Non-uniform stochastic average gradient method for training conditional random fields","authors":"Mark Schmidt, Reza Babanezhad, Mohamed Ahmed, Aaron Defazio, Ann Clifton, Anoop Sarkar","year":"2015","venue":"Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Schölkopf and Smola, 2001] B. Schölkopf and A. J. Smola. Learning with kernels: support vector machines, regular- ization, optimization, and beyond. MIT Press, Cambridge, MA, 2001.","title":"Learning with kernels: support vector machines, regularization, optimization, and beyond","authors":"B Schölkopf, A J Smola","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sion, 1958] Maurice Sion. On general minimax theorems. Pacific Journal of Mathematics, 8(1):171-176, 1958.","title":"On general minimax theorems","authors":"Maurice Sion","year":"1958","venue":"Pacific Journal of Mathematics","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xu et al., 2005] L. Xu, J. Neufeld, B. Larson, and D. Schu- urmans. Maximum margin clustering. In L. K. Saul, Y. Weiss, and L. Bottou, editors, Advances in Neural In- formation Processing Systems 17, pages 1537-1544. MIT Press, 2005.","title":"Maximum margin clustering","authors":"L Xu, J Neufeld, B Larson, D Schuurmans","year":"2005","venue":"Advances in Neural Information Processing Systems 17","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang and Zhou, 2014] T. Zhang and Z.-H. Zhou. Large margin distribution machine. In Proceedings of the 20th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 313-322, New York, NY, 2014.","title":"Large margin distribution machine","authors":"T Zhang, Z.-H Zhou","year":"2014","venue":"Proceedings of the 20th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang and Zhou, 2017] T. Zhang and Z.-H. Zhou. Multi- class optimal distribution machine. In Proceedings of the 34th International Conference on Machine Learning, pages 4063-4071, Sydney, NSW, Australia, 2017.","title":"Multiclass optimal distribution machine","authors":"T Zhang, Z.-H Zhou","year":"2017","venue":"Proceedings of the 34th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang and Zhou, 2018] T. Zhang and Z.-H. Zhou. Optimal margin distribution clustering. In Proceedings of the 20th National Conference on Artificial Intelligence, AAAI'18. AAAI Press, 2018.","title":"Optimal margin distribution clustering","authors":"T Zhang, Z.-H Zhou","year":"2018","venue":"Proceedings of the 20th National Conference on Artificial Intelligence, AAAI'18","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhou, 2014] Z.-H. Zhou. Large margin distribution learn- ing. In Proceedings of the 6th IAPR International Work- shop on Artificial Neural Networks in Pattern Recognition, pages 1-11, Montreal, Canada, 2014.","title":"Large margin distribution learning","authors":"Z.-H Zhou","year":"2014","venue":"Proceedings of the 6th IAPR International Workshop on Artificial Neural Networks in Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}