{"type":"ScholarlyArticle","title":"Fast SVM Trained by Divide-and-Conquer Anchors","authors":[{"name":"Meng Liu","slug":"meng-liu","given":"Meng","family":"Liu","affiliation":"† Key Laboratory of Machine Perception (MOE) , Cooperative Medianet Innovation Center ,"},{"name":"Chang Xu","slug":"chang-xu","given":"Chang","family":"Xu","affiliation":"‡ School of Electronics Engineering and Computer Science , Peking University , China; UBTech Sydney AI Institute , The School of Information Technologies , The University of Sydney"},{"name":"Chao Xu","slug":"chao-xu","given":"Chao","family":"Xu","affiliation":"† Key Laboratory of Machine Perception (MOE) , Cooperative Medianet Innovation Center ,"},{"name":"Dacheng Tao","slug":"dacheng-tao","given":"Dacheng","family":"Tao","affiliation":"‡ School of Electronics Engineering and Computer Science , Peking University , China"}],"abstract":"Supporting vector machine (SVM) is the most frequently used classifier for machine learning tasks. However, its training time could become cumbersome when the size of training data is very large. Thus, many kinds of representative subsets are chosen from the original dataset to reduce the training complexity. In this paper, we propose to choose the representative points which are noted as anchors obtained from non-negative matrix factorization (NMF) in a divide-and-conquer framework, and then use the anchors to train an approximate SVM. Our theoretical analysis shows that the solving the DCA-SVM can yield an approximate solution close to the primal SVM. Experimental results on multiple datasets demonstrate that our DCA-SVM is faster than the state-of-the-art algorithms without notably decreasing the accuracy of classification results.","doi":"10.24963/ijcai.2017/323","url":"https://ijcai.org/proceedings/2017/323","pdf":"https://ijcai.org/proceedings/2017/0323.pdf","markdown":"https://ijcai.org/proceedings/2017/323.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/323","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":323,"pages":{"first":2322,"last":2328},"keywords":["Machine Learning: Classification","Machine Learning: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Bennett and Bredensteiner, 2000] Kristin P Bennett and Erin J Bredensteiner. Duality and geometry in svm classi- fiers. In ICML, pages 57-64, 2000.","title":"Duality and geometry in svm classifiers","authors":"P Kristin, Erin J Bennett, Bredensteiner","year":"2000","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Boyd and Vandenberghe, 2004] Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cambridge universi- ty press, 2004.","title":"Convex optimization","authors":"Vandenberghe Boyd, Stephen Boyd, Lieven Vandenberghe","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chang and Lin, 2011] ChihChung Chang and ChihJen Lin. Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3):27, 2011.","title":"Libsvm: a library for support vector machines","authors":"Chang, Lin ;, Chihchung Chang, Chihjen Lin","year":"2011","venue":"ACM Transactions on Intelligent Systems and Technology","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chazelle, 1993] Bernard Chazelle. An optimal convex hull algorithm in any fixed dimension. Discrete & Computa- tional Geometry, 10(4):377-409, 1993.","title":"An optimal convex hull algorithm in any fixed dimension","authors":"Bernard Chazelle","year":"1993","venue":"Discrete & Computational Geometry","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Clarkson, 2010] Kenneth L Clarkson. Coresets, sparse greedy approximation, and the frank-wolfe algorithm. ACM Transactions on Algorithms, 6(4):63, 2010.","title":"Coresets, sparse greedy approximation, and the frank-wolfe algorithm","authors":"Kenneth L Clarkson","year":"2010","venue":"ACM Transactions on Algorithms","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cortes and Vapnik, 1995] Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine learning, 20(3):273-297, 1995.","title":"Corinna Cortes and Vladimir Vapnik. Support-vector networks","authors":"Vapnik Cortes","year":"1995","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Donoho and Stodden, 2003] David Donoho and Victoria S- todden. When does non-negative matrix factorization give a correct decomposition into parts? In Advances in neural information processing systems, page None, 2003.","title":"When does non-negative matrix factorization give a correct decomposition into parts?","authors":"David Donoho, Victoria Stodden","year":"2003","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fan et al., 2008] Rong-En Fan, Kai-Wei Chang, Cho-Jui H- sieh, Xiang-Rui Wang, and Chih-Jen Lin. Liblinear: A library for large linear classification. Journal of machine learning research, 9(Aug):1871-1874, 2008.","title":"Liblinear: A library for large linear classification","authors":"Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, Chih-Jen Lin","year":"2008","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gu et al., 2015] Bin Gu, Victor S Sheng, and Shuo Li. Bi- parameter space partition for cost-sensitive svm. In IJCAI, pages 3532-3539, 2015.","title":"Biparameter space partition for cost-sensitive svm","authors":"Bin Gu, Victor S Sheng, Shuo Li","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hsieh et al., 2014] Cho-Jui Hsieh, Si Si, and Inderjit S D- hillon. A divide-and-conquer solver for kernel support vector machines. In ICML, pages 566-574, 2014.","title":"A divide-and-conquer solver for kernel support vector machines","authors":"Cho-Jui Hsieh, Si Si, Inderjit S Dhillon","year":"2014","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Joachims, 1999] Thorsten Joachims. Svmlight: Support vector machine. SVM-Light Support Vector Machine, 19(4), 1999.","title":"Svmlight: Support vector machine","authors":"Thorsten Joachims","year":"1999","venue":"SVM-Light Support Vector Machine","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Joachims, 2005] Thorsten Joachims. A support vector method for multivariate performance measures. In Pro- ceedings of the 22nd international conference on Machine learning, pages 377-384, 2005.","title":"A support vector method for multivariate performance measures","authors":"Thorsten Joachims","year":"2005","venue":"Proceedings of the 22nd international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li and Guo, 2013] Xin Li and Yuhong Guo. Active learning with multi-label svm classification. In IJCAI, 2013.","title":"Active learning with multi-label svm classification","authors":"Guo ; ] Xin Li, Yuhong Li, Guo","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2013/221"},{"position":14,"raw":"[Li et al., 2015] Xiang Li, Huaimin Wang, Bin Gu, and Charles X Ling. Data sparseness in linear svm. In IJCAI, pages 3628-3634, 2015.","title":"Data sparseness in linear svm","authors":"Xiang Li, Huaimin Wang, Bin Gu, Charles X Ling","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/510"},{"position":15,"raw":"[Liu and Tao, 2016] Tongliang Liu and Dacheng Tao. Clas- sification with noisy labels by importance reweighting. IEEE Transactions on pattern analysis and machine in- telligence, 38(3):447-461, 2016.","title":"Tongliang Liu and Dacheng Tao. Classification with noisy labels by importance reweighting","authors":"Tao Liu","year":"2016","venue":"IEEE Transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2011] Qi Liu, Yong Ge, Zhongmou Li, and En- hong Chen. Personalized travel package recommendation. pages 407-416, 2011.","title":"Personalized travel package recommendation","authors":"Qi Liu, Yong Ge, Zhongmou Li, Enhong Chen","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Luo et al., 2016] Yong Luo, Yonggang Wen, Dacheng Tao, Jie Gui, and Chao Xu. Large margin multi-modal multi- task feature extraction for image classification. IEEE Transactions on Image Processing, 25(1):414-427, 2016.","title":"Large margin multi-modal multitask feature extraction for image classification","authors":"Yong Luo, Yonggang Wen, Dacheng Tao, Jie Gui, Chao Xu","year":"2016","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nandan et al., 2014] Manu Nandan, Pramod P Khar- gonekar, and Sachin S Talathi. Fast SVM training using approximate extreme points. Journal of Machine Learn- ing Research, 15(1):59-98, 2014.","title":"Fast SVM training using approximate extreme points","authors":"Manu Nandan, Sachin S Pramod P Khargonekar, Talathi","year":"2014","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shalev-Shwartz and Srebro, 2008] Shai Shalev-Shwartz and Nathan Srebro. Svm optimization: inverse depen- dence on training set size. In Proceedings of the 25th international conference on Machine learning, pages 928-935. ACM, 2008.","title":"Svm optimization: inverse dependence on training set size","authors":"Shalev-Shwartz, Srebro ; Shai Shalev-Shwartz, Nathan Srebro","year":"2008","venue":"Proceedings of the 25th international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shalev-Shwartz et al., 2011] Shai Shalev-Shwartz, Yoram Singer, Nathan Srebro, and Andrew Cotter. Pegasos: Pri- mal estimated sub-gradient solver for svm. Mathematical programming, 127(1):3-30, 2011.","title":"Pegasos: Primal estimated sub-gradient solver for svm","authors":"Shai Shalev-Shwartz, Yoram Singer, Nathan Srebro, Andrew Cotter","year":"2011","venue":"Mathematical programming","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tavallaee et al., 2009] Mahbod Tavallaee, Ebrahim Bagher- i, Wei Lu, and Ali-A Ghorbani. A detailed analysis of the kdd cup 99 data set. In Proceedings of the Second IEEE Symposium on Computational Intelligence for Secu- rity and Defence Applications, 2009.","title":"A detailed analysis of the kdd cup 99 data set","authors":"Mahbod Tavallaee, Ebrahim Bagheri, Wei Lu, Ali-A Ghorbani","year":"2009","venue":"Proceedings of the Second IEEE Symposium on Computational Intelligence for Security and Defence Applications","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tsang et al., 2005] Ivor W Tsang, James T Kwok, and Pak- Ming Cheung. Core vector machines: Fast SVM training on very large data sets. Journal of Machine Learning Re- search, 6(Apr):363-392, 2005.","title":"Core vector machines: Fast SVM training on very large data sets","authors":"Ivor W Tsang, James T Kwok, Pak-Ming Cheung","year":"2005","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tsang et al., 2007] Ivor W Tsang, Andras Kocsor, and James T Kwok. Simpler core vector machines with en- closing balls. In ICML, pages 911-918, 2007.","title":"Simpler core vector machines with enclosing balls","authors":"Ivor W Tsang, Andras Kocsor, James T Kwok","year":"2007","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Vavasis, 2009] Stephen A Vavasis. On the complexity of nonnegative matrix factorization. SIAM Journal on Opti- mization, 20(3):1364-1377, 2009.","title":"On the complexity of nonnegative matrix factorization","authors":"A Stephen, Vavasis","year":"2009","venue":"SIAM Journal on Optimization","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xu et al., 2015] Chang Xu, Dacheng Tao, and Chao Xu. Large-margin multi-label causal feature learning. In AAAI, pages 1924-1930, 2015.","title":"Large-margin multi-label causal feature learning","authors":"Chang Xu, Dacheng Tao, Chao Xu","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xu et al., 2016] Chang Xu, Dacheng Tao, and Chao Xu. Robust extreme multi-label learning. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowl- edge Discovery and Data Mining, San Francisco, CA, USA August, pages 13-17, 2016.","title":"Robust extreme multi-label learning","authors":"Chang Xu, Dacheng Tao, Chao Xu","year":"2016","venue":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zheng et al., 2015] Xiaodong Zheng, Shanfeng Zhu, Jun- ning Gao, and Hiroshi Mamitsuka. Instance-wise weight- ed nonnegative matrix factorization for aggregating parti- tions with locally reliable clusters. In IJCAI, pages 4091- 4097, 2015.","title":"Instance-wise weighted nonnegative matrix factorization for aggregating partitions with locally reliable clusters","authors":"Xiaodong Zheng, Shanfeng Zhu, Junning Gao, Hiroshi Mamitsuka","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/574"},{"position":28,"raw":"[Zhou et al., 2013] Tianyi Zhou, Wei Bian, and Dacheng Tao. Divide-and-conquer anchoring for near-separable nonnegative matrix factorization and completion in high dimensions. In International Conference on Data Mining, pages 917-926, 2013.","title":"Divide-and-conquer anchoring for near-separable nonnegative matrix factorization and completion in high dimensions","authors":"Tianyi Zhou, Wei Bian, Dacheng Tao","year":"2013","venue":"International Conference on Data Mining","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}