{"type":"ScholarlyArticle","title":"Feature Selection via Scaling Factor Integrated Multi-Class Support Vector Machines","authors":[{"name":"Jinglin Xu","slug":"jinglin-xu","given":"Jinglin","family":"Xu","affiliation":"Northwestern Polytechnical University , Xian , 710072 , P. R. China"},{"name":"Feiping Nie","slug":"feiping-nie","given":"Feiping","family":"Nie","affiliation":"Northwestern Polytechnical University , Xian , 710072 , P. R. China"},{"name":"Junwei Han","slug":"junwei-han","given":"Junwei","family":"Han","affiliation":"Northwestern Polytechnical University , Xian , 710072 , P. R. China"}],"abstract":"In data mining, we often encounter high dimensional and noisy features, which may not only increase the load of computational resources but also result in the problem of model overfitting. Feature selection is often adopted to address this issue. In this paper, we propose a novel feature selection method based on multi-class SVM, which introduces the scaling factor with a flexible parameter to renewedly adjust the distribution of feature weights and select the most discriminative features. Concretely, the proposed method designs a scaling factor with p/2 power to control the distribution of weights adaptively and search optimal sparsity of weighting matrix. In addition, to solve the proposed model, we provide an alternative and iterative optimization method. It not only makes solutions of weighting matrix and scaling factor independently, but also provides a better way to address the problem of solving L2,0-norm. Comprehensive experiments are conducted on six datasets to demonstrate that this work can obtain better performance compared with a number of existing state-of-the-art multi-class feature selection methods.","doi":"10.24963/ijcai.2017/442","url":"https://ijcai.org/proceedings/2017/442","pdf":"https://ijcai.org/proceedings/2017/0442.pdf","markdown":"https://ijcai.org/proceedings/2017/442.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/442","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":442,"pages":{"first":3168,"last":3174},"keywords":["Machine Learning: Feature Selection/Construction","Machine Learning: Machine Learning","Machine Learning: Classification"],"videos":[],"references":[{"position":1,"raw":"[Cai et al., 2011] Xiao Cai, Feiping Nie, Heng Huang, and Chris Ding. Multi-class ℓ 2,1 -norm support vector machine. In ICDM, pages 91-100, 2011.","title":"Multi-class ℓ 2,1 -norm support vector machine","authors":"Xiao Cai, Feiping Nie, Heng Huang, Chris Ding","year":"2011","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chang et al., 2014] X. Chang, F. Nie, Y. Yang, and H. Huang. A convex formulation for semi-supervised multi-label feature selection. In AAAI, pages 1171-1177, 2014.","title":"A convex formulation for semi-supervised multi-label feature selection","authors":"X Chang, F Nie, Y Yang, H Huang","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chapelle and Keerthi, 2008] Olivier Chapelle and S Sathiya Keerthi. Multi-class feature selection with support vector machines. In PASA, 2008.","title":"Multi-class feature selection with support vector machines","authors":"Keerthi Chapelle, Olivier Chapelle, S Sathiya, Keerthi","year":"2008","venue":"PASA","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cover and Thomas, 1991] Thomas M Cover and Joy A Thomas. Elements of information theory. New York, 1991.","title":"Elements of information theory","authors":"M Thomas, Joy A Cover, Thomas","year":"1991","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Crammer and Singer, 2001] K. Crammer and Y. Singer. On the algorithmic interpretation of multiclass kernel-based vector machines. JMLR, 2(2):2001, 2001.","title":"On the algorithmic interpretation of multiclass kernel-based vector machines","authors":"K Crammer, Y Singer","year":"2001","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cristianini and Shawe-Taylor, 2000] Nello Cristianini and John Shawe-Taylor. An introduction to support vector ma- chines, 2000.","title":"An introduction to support vector machines","authors":"Nello Cristianini, John Shawe-Taylor","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Duda et al., 1973] Richard O Duda, Peter E Hart, David G Stork, et al. Pattern classification, volume 2. Wiley New York, 1973.","title":"Pattern classification","authors":"Peter E Richard O Duda, David G Hart, Stork","year":"1973","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fan et al., 2008] Rong-En Fan, Kai-Wei Chang, Cho-Jui H- sieh, Xiang-Rui Wang, and Chih-Jen Lin. Liblinear: A library for large linear classification. JMLR, 9:1871-1874, 2008.","title":"Liblinear: A library for large linear classification","authors":"Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, Chih-Jen Lin","year":"2008","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gupta and Xiao, 2011] Mithun Das Gupta and Jing Xiao. Non-negative matrix factorization as a feature selection tool for maximum margin classifiers. In CVPR, pages 2841-2848, 2011.","title":"Non-negative matrix factorization as a feature selection tool for maximum margin classifiers","authors":"Mithun Das, Gupta, Jing Xiao","year":"2011","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Guyon and Elisseeff, 2003] Isabelle Guyon and André Elis- seeff. An introduction to variable and feature selection. JMLR, 3:1157-1182, 2003.","title":"Isabelle Guyon and André Elisseeff. An introduction to variable and feature selection","authors":"Elisseeff Guyon","year":"2003","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Guyon et al., 2002] Isabelle Guyon, Jason Weston, Stephen Barnhill, and Vladimir Vapnik. Gene selection for cancer classification using support vector machines. ML, 46(1- 3):389-422, 2002.","title":"Gene selection for cancer classification using support vector machines","authors":"Isabelle Guyon, Jason Weston, Stephen Barnhill, Vladimir Vapnik","year":"2002","venue":"ML","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Han and Kim, 2015] Dongyoon Han and Junmo Kim. Un- supervised simultaneous orthogonal basis clustering fea- ture selection. In CVPR, pages 5016-5023, 2015.","title":"Dongyoon Han and Junmo Kim. Unsupervised simultaneous orthogonal basis clustering feature selection","authors":"Kim Han","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[He et al., 2012] Ran He, Tieniu Tan, Liang Wang, and Wei- Shi Zheng. l 2,1 regularized correntropy for robust feature selection. In CVPR, pages 2504-2511, 2012.","title":"l 2,1 regularized correntropy for robust feature selection","authors":"Ran He, Tieniu Tan, Liang Wang, Wei-Shi Zheng","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hong et al., 2016] Tao Hong, Hou Chenping, Nie Feiping, Jiao Yuanyuan, and Yi Dongyun. Effective discrimina- tive feature selection with nontrivial solution. TNNLS, 27(4):796-808, 2016.","title":"Effective discriminative feature selection with nontrivial solution","authors":"Tao Hong, Hou Chenping, Nie Feiping, Jiao Yuanyuan, Yi Dongyun","year":"2016","venue":"TNNLS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hou et al., 2011] Chenping Hou, Feiping Nie, Dongyun Yi, and Yi Wu. Feature selection via joint embedding learning and sparse regression. In IJCAI, pages 1324-1329, 2011.","title":"Feature selection via joint embedding learning and sparse regression","authors":"Hou","year":"2011","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2011/224"},{"position":16,"raw":"[Huang et al., 2012] Heng Huang, C. Ding, Deguang Kong, and Haifeng Zhao. Multi-label relieff and f-statistic feature selections for image annotation. In CVPR, pages 2352- 2359, 2012.","title":"Multi-label relieff and f-statistic feature selections for image annotation","authors":"C Heng Huang, Deguang Ding, Haifeng Kong, Zhao","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kohavi and John, 1997] Ron Kohavi and George H John. Wrappers for feature subset selection. AI, 97(1):273-324, 1997.","title":"Wrappers for feature subset selection","authors":"John ; Kohavi, Ron Kohavi, George H John","year":"1997","venue":"AI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kusakunniran et al., 2010] Worapan Kusakunniran, Qiang Wu, Jian Zhang, and Hongdong Li. Support vector regres- sion for multi-view gait recognition based on local motion feature selection. In CVPR, pages 974-981, 2010.","title":"Support vector regression for multi-view gait recognition based on local motion feature selection","authors":"Kusakunniran","year":"2010","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lal et al., 2006] Thomas Navin Lal, Olivier Chapelle, Jason Weston, and André Elisseeff. Embedded methods. In FE, pages 137-165. 2006.","title":"Embedded methods","authors":"Olivier Thomas Navin Lal, Jason Chapelle, André Weston, Elisseeff","year":"2006","venue":"FE","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu and Motoda, 2007] Huan Liu and Hiroshi Motoda. Computational methods of feature selection. CRC Press, 2007.","title":"Computational methods of feature selection","authors":"Huan Liu, Hiroshi Motoda","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu and Setiono, 1995] Huan Liu and Rudy Setiono. Chi2: Feature selection and discretization of numeric attributes. In TWAI, pages 388-391, 1995.","title":"Chi2: Feature selection and discretization of numeric attributes","authors":"Setiono Liu, Huan Liu, Rudy Setiono","year":"1995","venue":"TWAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu et al., 2013] Xinwang Liu, Lei Wang, Jian Zhang, Jian- ping Yin, and Huan Liu. Global and local structure preser- vation for feature selection. TNNLS, 25(6):1083-1095, 2013.","title":"Global and local structure preservation for feature selection","authors":"Xinwang Liu, Lei Wang, Jian Zhang, Jianping Yin, Huan Liu","year":"2013","venue":"TNNLS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Mangasarian, 2006] Olvi L Mangasarian. Exact ℓ 1 -norm support vector machines via unconstrained convex differ- entiable minimization. JMLR, 7:1517-1530, 2006.","title":"Exact ℓ 1 -norm support vector machines via unconstrained convex differentiable minimization","authors":"; Mangasarian, Olvi L Mangasarian","year":"2006","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Montgomery et al., 2009] Douglas C Montgomery, George C Runger, and Norma F Hubele. Engineer- ing statistics. John Wiley & Sons, 2009.","title":"Engineering statistics","authors":"George C Douglas C Montgomery, Norma F Runger, Hubele","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Nie et al., 2010] Feiping Nie, Heng Huang, Xiao Cai, and Chris H. Q. Ding. Efficient and robust feature selection via joint ℓ 2,1 -norms minimization. In NIPS, pages 1813- 1821, 2010.","title":"Efficient and robust feature selection via joint ℓ 2,1 -norms minimization","authors":"Nie","year":"2010","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Obozinski et al., 2006] Guillaume Obozinski, Ben Taskar, and Michael Jordan. Multi-task feature selection. SD, 2, 2006.","title":"Multi-task feature selection","authors":"Guillaume Obozinski, Ben Taskar, Michael Jordan","year":"2006","venue":"SD","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Padungweang et al., 2012] P Padungweang, C Lursinsap, and K Sunat. A discrimination analysis for unsupervised feature selection via optic diffraction principle. TNNLS, 23(10):1587-1600, 2012.","title":"A discrimination analysis for unsupervised feature selection via optic diffraction principle","authors":"Padungweang, Lursinsap, Sunat","year":"2012","venue":"TNNLS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Peng et al., 2005] Hanchuan Peng, Fuhui Long, and Chris Ding. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min- redundancy. TPAMI, 27(8):1226-1238, 2005.","title":"Feature selection based on mutual information criteria of max-dependency, max-relevance, and minredundancy","authors":"Peng","year":"2005","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2007] Li Wang, Ji Zhu, and Hui Zou. Hybrid huberized support vector machines for microarray classifi- cation. In ICML, pages 983-990, 2007.","title":"Hybrid huberized support vector machines for microarray classification","authors":"Li Wang, Ji Zhu, Hui Zou","year":"2007","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yang et al., 2011] Yi Yang, Heng Tao Shen, Zhigang Ma, Zi Huang, and Xiaofang Zhou. l 2,1 -norm regularized dis- criminative feature selection for unsupervised learning. In IJCAI, pages 1589-1594, 2011.","title":"l 2,1 -norm regularized discriminative feature selection for unsupervised learning","authors":"Yi Yang, Heng Tao Shen, Zhigang Ma, Zi Huang, Xiaofang Zhou","year":"2011","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhao et al., 2010] Zheng Zhao, Fred Morstatter, Shashvata Sharma, Salem Alelyani, Aneeth Anand, and Huan Liu. Advancing feature selection research. ASU, pages 1-28, 2010.","title":"Advancing feature selection research","authors":"Zheng Zhao, Fred Morstatter, Shashvata Sharma, Salem Alelyani, Aneeth Anand, Huan Liu","year":"2010","venue":"ASU","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhu et al., 2014] Xiaofeng Zhu, Heung-Il Suk, and Ding- gang Shen. Matrix-similarity based loss function and fea- ture selection for alzheimer's disease diagnosis. In CVPR, pages 3089-3096, 2014.","title":"Matrix-similarity based loss function and feature selection for alzheimer's disease diagnosis","authors":"Xiaofeng Zhu, Heung-Il Suk, Dinggang Shen","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}