{"type":"ScholarlyArticle","title":"Optimal Margin Distribution Machine for Multi-Instance Learning","authors":[{"name":"Teng Zhang","slug":"teng-zhang","given":"Teng","family":"Zhang","affiliation":"National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab , Cluster and Grid Computing Lab School of Computer Science and Technology , Huazhong University of Science and Technology , China"},{"name":"Hai Jin","slug":"hai-jin","given":"Hai","family":"Jin","affiliation":"National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab , Cluster and Grid Computing Lab School of Computer Science and Technology , Huazhong University of Science and Technology , China"}],"abstract":"Multi-instance learning (MIL) is a celebrated learning framework where each example is represented as a bag of instances. An example is negative if it has no positive instances, and vice versa if at least one positive instance is contained. During the past decades, various MIL algorithms have been proposed, among which the large margin based methods is a very popular class. Recently, the studies on margin theory disclose that the margin distribution is of more importance to generalization ability than the minimal margin. Inspired by this observation, we propose the multi-instance optimal margin distribution machine, which can identify the key instances via explicitly optimizing the margin distribution. We also extend a stochastic accelerated mirror prox method to solve the formulated minimax problem. Extensive experiments show the superiority of the proposed method.","doi":"10.24963/ijcai.2020/330","url":"https://ijcai.org/proceedings/2020/330","pdf":"https://ijcai.org/proceedings/2020/0330.pdf","markdown":"https://ijcai.org/proceedings/2020/330.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/330","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":330,"pages":{"first":2383,"last":2389},"keywords":["Machine Learning: Multi-instance;Multi-label;Multi-view learning","Machine Learning: Classification"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25445/file_25445.m3u8","length_sec":264,"provider":"hls"}],"references":[{"position":1,"raw":"[Amores, 2013] Jaume Amores. Multiple instance classifica- tion: Review, taxonomy and comparative study. Artificial Intelligence, 201(4):81-105, 2013.","title":"Multiple instance classification: Review, taxonomy and comparative study","authors":"Jaume Amores, Amores","year":"2013","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Andrews et al., 2003] Stuart Andrews, Ioannis Tsochan- taridis, and Thomas Hofmann. Support vector machines for multiple-instance learning. In Advances in Neural Information Processing Systems, pages 561-568, Cam- bridge, MA, 2003.","title":"Support vector machines for multiple-instance learning","authors":"Stuart Andrews, Ioannis Tsochantaridis, Thomas Hofmann","year":"2003","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Beck and Teboulle, 2003] Amir Beck and Marc Teboulle. Mirror descent and nonlinear projected subgradient meth- ods for convex optimization. Operations Research Letters, 31(3):167-175, 2003.","title":"Mirror descent and nonlinear projected subgradient methods for convex optimization","authors":"Amir Beck, Marc Teboulle","year":"2003","venue":"Operations Research Letters","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bi et al., 2005] Jinbo Bi, Yixin Chen, and James Z. Wang. A sparse support vector machine approach to region-based image categorization. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 1121-1128, San Diego, CA, 2005.","title":"A sparse support vector machine approach to region-based image categorization","authors":"Jinbo Bi, Yixin Chen, James Z Wang","year":"2005","venue":"Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2017] Yunmei Chen, Guanghui Lan, and Yuyuan Ouyang. Accelerated schemes for a class of variational inequalities. Mathematical Programming, 165(1):113-149, 2017.","title":"Accelerated schemes for a class of variational inequalities","authors":"Chen","year":"2017","venue":"Mathematical Programming","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cristianini and Shawe-Taylor, 2000] Nello Cristianini and John Shawe-Taylor. An Introduction to Support Vec- tor Machines and Other Kernel-based Learning Methods. Cambridge University Press, Cambridge, UK, 2000.","title":"An Introduction to Support Vector Machines and Other Kernel-based Learning Methods","authors":"Shawe-Taylor ; Nello Cristianini, John Cristianini, Shawe-Taylor","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dietterich et al., 1997] Thomas G. Dietterich, Richard H. Lathrop, and Tomás Lozano-Pérez. Solving the multiple instance problem with axis-parallel rectangles. Artificial Intelligence, 89(1-2):31-71, 1997.","title":"Solving the multiple instance problem with axis-parallel rectangles","authors":"G Thomas, Richard H Dietterich, Tomás Lathrop, Lozano-Pérez","year":"1997","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Foulds and Frank, 2010] James Richard Foulds and Eibe Frank. A review of multi-instance learning assumptions. Knowledge Engineering Review, 25(1):1-25, 2010.","title":"A review of multi-instance learning assumptions","authors":"Frank ; Foulds, Richard James, Eibe Foulds, Frank","year":"2010","venue":"Knowledge Engineering Review","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gao and Zhou, 2013] Wei Gao and Zhi-Hua Zhou. On the doubt about margin explanation of boosting. Artificial In- telligence, 203:1-18, 2013.","title":"On the doubt about margin explanation of boosting","authors":"Wei Gao, Zhi-Hua Zhou","year":"2013","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gärtner et al., 2002] Thomas Gärtner, Peter A. Flach, Adam Kowalczyk, and Alexander J. Smola. Multi-instance ker- nels. In Proceedings of the 19th International Conference on Machine Learning, pages 179-186, Sydney, Australia, 2002.","title":"Multi-instance kernels","authors":"Thomas Gärtner, Peter A Flach, Adam Kowalczyk, Alexander J Smola","year":"2002","venue":"Proceedings of the 19th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Herrera et al., 2016] Francisco Herrera, Sebastián Ventura, Rafael Bello, Chris Cornelis, Amelia Zafra, Dánel Sánchez-Tarragó, and Sarah Vluymans. Multiple Instance Learning: Foundations and Algorithms. Springer, Cham, Switzerland, 2016.","title":"Multiple Instance Learning: Foundations and Algorithms","authors":"Herrera","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Johnson and Zhang, 2013] Rie Johnson and Tong Zhang. Accelerating stochastic gradient descent using predictive variance reduction. In Advances in Neural Information Processing Systems, pages 315-323, Lake Tahoe, NV, 2013.","title":"Accelerating stochastic gradient descent using predictive variance reduction","authors":"Rie Johnson, Tong Zhang","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Juditsky et al., 2011] Anatoli Juditsky, Arkadii Nemirovski, and Claire Tauvel. Solving variational inequalities with stochastic mirror-prox algorithm. Stochastic Systems, 1(1):17-58, 2011.","title":"Solving variational inequalities with stochastic mirror-prox algorithm","authors":"Anatoli Juditsky, Arkadii Nemirovski, Claire Tauvel","year":"2011","venue":"Stochastic Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2009] Yu-Feng Li, James T. Kwok, Ivor W. Tsang, and Zhi-Hua Zhou. A convex method for locating regions of interest with multi-instance learning. In Proceedings of the 20th European Conference on Machine Learning, pages 15-30, Bled, Slovenia, 2009.","title":"A convex method for locating regions of interest with multi-instance learning","authors":"Yu-Feng Li, James T Kwok, Ivor W Tsang, Zhi-Hua Zhou","year":"2009","venue":"Proceedings of the 20th European Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Maron and Ratan, 1998] Oded Maron and Aparna Lakshmi Ratan. Multiple-instance learning for natural scene classi- fication. In Proceedings of the 15th International Confer- ence on Machine Learning, pages 341-349, Madison, WI, 1998.","title":"Multiple-instance learning for natural scene classification","authors":"Ratan Maron, Oded Maron, Aparna Lakshmi Ratan","year":"1998","venue":"Proceedings of the 15th International Confer-ence on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nemirovski, 2005] Arkadi Nemirovski. Prox-method with rate of convergence O(1/T) for variational inequalities with lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM Journal on Optimization, 15(1):229-251, 2005.","title":"Prox-method with rate of convergence O(1/T) for variational inequalities with lipschitz continuous monotone operators and smooth convex-concave saddle point problems","authors":"Arkadi Nemirovski","year":"2005","venue":"SIAM Journal on Optimization","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Nesterov, 2003] Yurii Nesterov. Introductory Lectures on Convex Optimization: A Basic Course. Springer, New York, NY, 2003.","title":"Introductory Lectures on Convex Optimization: A Basic Course","authors":"Yurii Nesterov","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sion, 1958] Maurice Sion. On general minimax theorems. Pacific Journal of Mathematics, 8(1):171-176, 1958.","title":"On general minimax theorems","authors":"Maurice Sion","year":"1958","venue":"Pacific Journal of Mathematics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Tan et al., 2020] Zhi-Hao Tan, Peng Tan, Yuan Jiang, and Zhi-Hua Zhou. Multi-label optimal margin distribution machine. Machine Learning, 109(3):623-642, 2020.","title":"Multi-label optimal margin distribution machine","authors":"Zhi-Hao Tan, Peng Tan, Yuan Jiang, Zhi-Hua Zhou","year":"2020","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xu et al., 2004] Linli Xu, James Neufeld, Bryce Larson, and Dale Schuurmans. Maximum margin clustering. In Advances in Neural Information Processing Systems, pages 1537-1544, Vancouver, Canada, 2004.","title":"Maximum margin clustering","authors":"Xu","year":"2004","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang and Goldman, 2001] Qi Zhang and Sally A. Gold- man. EM-DD: An improved multiple-instance learning technique. In Advances in Neural Information Processing Systems, pages 1073-1080, Cambridge, MA, 2001.","title":"EM-DD: An improved multiple-instance learning technique","authors":"Goldman Zhang, Qi Zhang, Sally A Goldman","year":"2001","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang and Zhou, 2018a] Teng Zhang and Zhi-Hua Zhou. Optimal margin distribution clustering. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, pages 4474-4481, New Orleans, LA, 2018.","title":"Optimal margin distribution clustering","authors":"Zhou Zhang, Teng Zhang, Zhi-Hua Zhou","year":"2018","venue":"Proceedings of the 32nd AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang and Zhou, 2018b] Teng Zhang and Zhi-Hua Zhou. Semi-supervised optimal margin distribution machines. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, pages 3104-3110, Stockholm, Swe- den, 2018.","title":"Semi-supervised optimal margin distribution machines","authors":"Zhou Zhang, Teng Zhang, Zhi-Hua Zhou","year":"2018","venue":"Proceedings of the 27th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2018/431"},{"position":24,"raw":"[Zhang and Zhou, 2019] Teng Zhang and Zhi-Hua Zhou. Optimal margin distribution machine. IEEE Transactions on Knowledge and Data Engineering, 32(6):1143-1156, 2019.","title":"Optimal margin distribution machine","authors":"Teng Zhang, Zhi-Hua Zhou","year":"2019","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhao et al., 2008] Bin Zhao, Fei Wang, and Changshui Zhang. Efficient maximum margin clustering via cutting plane algorithm. In Proceedings of the SIAM International Conference on Data Mining, pages 751-762, Atlanta, GA, 2008.","title":"Efficient maximum margin clustering via cutting plane algorithm","authors":"Bin Zhao, Fei Wang, Changshui Zhang","year":"2008","venue":"Proceedings of the SIAM International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhou et al., 2005] Zhi-Hua Zhou, Xiao-Bing Xue, and Yuan Jiang. Locating regions of interest in cbir with multi- instance learning techniques. In Proceedings of the 18th Australian Joint Conference on Advances in Artificial In- telligence, pages 92-101, Sydney, Australia, 2005.","title":"Locating regions of interest in cbir with multiinstance learning techniques","authors":"Zhi-Hua Zhou, Xiao-Bing Xue, Yuan Jiang","year":"2005","venue":"Proceedings of the 18th Australian Joint Conference on Advances in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhou et al., 2012] Zhi-Hua Zhou, Min-Ling Zhang, Sheng- Jun Huang, and Yu-Feng Li. Multi-instance multi-label learning. Artificial Intelligence, 176(1):2291-2320, 2012.","title":"Multi-instance multi-label learning","authors":"Zhi-Hua Zhou, Min-Ling Zhang, Sheng-Jun Huang, Yu-Feng Li","year":"2012","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhou, 2014] Zhi-Hua Zhou. Large margin distribution learning. In Proceedings of the 6th IAPR International Workshop on Artificial Neural Networks in Pattern Recog- nition, pages 1-11, Montreal, Canada, 2014.","title":"Large margin distribution learning","authors":"Zhi-Hua Zhou","year":"2014","venue":"Proceedings of the 6th IAPR International Workshop on Artificial Neural Networks in Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}