{"type":"ScholarlyArticle","title":"Semi-supervised Feature Selection via Rescaled Linear Regression","authors":[{"name":"Xiaojun Chen","slug":"xiaojun-chen","given":"Xiaojun","family":"Chen","affiliation":"College of Computer Science and Software , Shenzhen University , Shenzhen 518060 , P.R. China"},{"name":"Guowen Yuan","slug":"guowen-yuan","given":"Guowen","family":"Yuan","affiliation":"School of Computer Science and Center for OPTical IMagery Analysis and Learning (OPTIMAL) , Northwestern Polytechnical University , Xi ′ an 710072 , P. R. China"},{"name":"Feiping Nie","slug":"feiping-nie","given":"Feiping","family":"Nie","affiliation":"College of Computer Science and Software , Shenzhen University , Shenzhen 518060 , P.R. China"},{"name":"Joshua Zhexue Huang","slug":"joshua-zhexue-huang","given":"Joshua Zhexue","family":"Huang","affiliation":"College of Computer Science and Software , Shenzhen University , Shenzhen 518060 , P.R. China"}],"abstract":"With the rapid increase of complex and high-dimensional sparse data, demands for new methods to select features by exploiting both labeled and unlabeled data have increased. Least regression based feature selection methods usually learn a projection matrix and evaluate the importances of features using the projection matrix, which is lack of theoretical explanation. Moreover, these methods cannot find both global and sparse solution of the projection matrix. In this paper, we propose a novel semi-supervised feature selection method which can learn both global and sparse solution of the projection matrix. The new method extends the least square regression model by rescaling the regression coefficients in the least square regression with a set of scale factors, which are used for ranking the features. It has shown that the new model can learn global and sparse solution. Moreover, the introduction of scale factors provides a theoretical explanation for why we can use the projection matrix to rank the features. A simple yet effective algorithm with proved convergence is proposed to optimize the new model. Experimental results on eight real-life data sets show the superiority of the method.","doi":"10.24963/ijcai.2017/211","url":"https://ijcai.org/proceedings/2017/211","pdf":"https://ijcai.org/proceedings/2017/0211.pdf","markdown":"https://ijcai.org/proceedings/2017/211.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/211","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":211,"pages":{"first":1525,"last":1531},"keywords":["Machine Learning: Machine Learning","Machine Learning: Feature Selection/Construction"],"videos":[],"references":[{"position":1,"raw":"[Cai et al., 2010] Deng Cai, Chiyuan Zhang, and Xiaofei He. Unsupervised feature selection for multi-cluster data. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 333-342, 2010.","title":"Unsupervised feature selection for multi-cluster data","authors":"Deng Cai, Chiyuan Zhang, Xiaofei He","year":"2010","venue":"ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chang et al., 2014] Xiaojun Chang, Feiping Nie, Yi Yang, and Heng Huang. A convex formulation for semi- supervised multi-label feature selection. In Twenty-Eighth AAAI Conference on Artificial Intelligence, pages 1171- 1177, 2014.","title":"A convex formulation for semisupervised multi-label feature selection","authors":"Xiaojun Chang, Feiping Nie, Yi Yang, Heng Huang","year":"2014","venue":"Twenty-Eighth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cristianini and Shawe-Taylor, 2000] Nello Cristianini and John Shawe-Taylor. An Introduction to Support Vector Machines: And Other Kernel-based Learning Methods. Cambridge University Press, New York, NY, USA, 2000.","title":"An Introduction to Support Vector Machines: And Other Kernel-based Learning Methods","authors":"Shawe-Taylor ; Nello Cristianini, John Cristianini, Shawe-Taylor","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Doquire and Verleysen, 2013] Gauthier Doquire and Michel Verleysen. A graph laplacian based approach to semi-supervised feature selection for regression problems. Neurocomputing, 121:5-13, 2013.","title":"A graph laplacian based approach to semi-supervised feature selection for regression problems","authors":"Gauthier Doquire, Michel Verleysen","year":"2013","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Guyon and Elisseef, 2003] Isabelle Guyon and André Elis- seef. An introduction to variable and feature selection. Journal of Machine Learning Research, 3:1157-1182, 2003.","title":"Isabelle Guyon and André Elisseef. An introduction to variable and feature selection","authors":"Elisseef Guyon","year":"2003","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2005] Xiaofei He, Deng Cai, and Partha Niyogi. Laplacian score for feature selection. In Advances in neu- ral information processing systems, pages 507-514, 2005.","title":"Laplacian score for feature selection","authors":"Xiaofei He, Deng Cai, Partha Niyogi","year":"2005","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Huang, 2015] Samuel H Huang. Supervised feature selec- tion: A tutorial. Artificial Intelligence Research, 4(2):22, 2015.","title":"Supervised feature selection: A tutorial","authors":"H Samuel, Huang","year":"2015","venue":"Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kira and Rendell, 1992] Kenji Kira and Larry A. Rendell. A practical approach to feature selection. In The ninth inter- national workshop on Machine learning, pages 249-256, 1992.","title":"A practical approach to feature selection","authors":"Kenji Kira, Larry A Rendell","year":"1992","venue":"The ninth international workshop on Machine learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kong and Yu, 2010] Xiangnan Kong and Philip S Yu. Semi- supervised feature selection for graph classification. In Proceedings of the 16th ACM SIGKDD international con- ference on Knowledge discovery and data mining, pages 793-802. ACM, 2010.","title":"Semisupervised feature selection for graph classification","authors":"Yu ; Kong, Xiangnan Kong, Philip S Yu","year":"2010","venue":"Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kononenko, 1994] Igor Kononenko. Estimating attributes: Analysis and extensions of relief. In Proceedings of ECML-94, pages 171-182, Italy, 1994. Springer Berlin Heidelberg.","title":"Estimating attributes: Analysis and extensions of relief","authors":"Igor Kononenko, Kononenko","year":"1994","venue":"Proceedings of ECML-94","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu and Yu, 2005] Huan Liu and Lei Yu. Toward integrating feature selection algorithms for classification and cluster- ing. IEEE Transactions on Knowledge and Data Engineer- ing, 17(4):491-502, 2005.","title":"Toward integrating feature selection algorithms for classification and clustering","authors":"Huan Liu, Lei Yu","year":"2005","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Luo et al., 2013] Yong Luo, Dacheng Tao, Chang Xu, Dongchen Li, and Chao Xu. Vector-valued multi-view semi-supervised learning for multi-label image classifica- tion. In Proceedings of the Twenty-Seventh AAAI Confer- ence on Artificial Intelligence, AAAI'13, pages 647-653. AAAI Press, 2013.","title":"Vector-valued multi-view semi-supervised learning for multi-label image classification","authors":"Yong Luo, Dacheng Tao, Chang Xu, Dongchen Li, Chao Xu","year":"2013","venue":"Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence, AAAI'13","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Nie et al., 2010] Feiping Nie, Heng Huang, Xiao Cai, and Chris H Ding. Efficient and robust feature selection via joint ℓ 2,1 -norms minimization. In Advances in neural in- formation processing systems, pages 1813-1821, 2010.","title":"Efficient and robust feature selection via joint ℓ 2,1 -norms minimization","authors":"Nie","year":"2010","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ren et al., 2008] Jiangtao Ren, Zhengyuan Qiu, Wei Fan, Hong Cheng, and Philip S. Yu. Forward semi-supervised feature selection. In Proceedings of the 12th Pacific-Asia Conference in Knowledge Discovery and Data Mining, pages 970-976, Berlin, Heidelberg, 2008. Springer Berlin Heidelberg.","title":"Forward semi-supervised feature selection","authors":"Jiangtao Ren, Zhengyuan Qiu, Wei Fan, Hong Cheng, Philip S Yu","year":"2008","venue":"Proceedings of the 12th Pacific-Asia Conference in Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Richard et al., 2010] D. Richard, P. E. Hart, and D. G Stork. Pattern classification. Wiley-Interscience, 2010.","title":"Pattern classification","authors":"D Richard, P E Hart, D G Stork","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Shi et al., 2014] Lei Shi, Liang Du, and Yi-Dong Shen. Ro- bust spectral learning for unsupervised feature selection. In IEEE International Conference on Data Mining, pages 977-982, Dec 2014.","title":"Robust spectral learning for unsupervised feature selection","authors":"Lei Shi, Liang Du, Yi-Dong Shen","year":"2014","venue":"IEEE International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Strutz, 2010] Tilo Strutz. Data fitting and uncertainty: A practical introduction to weighted least squares and be- yond. Vieweg and Teubner, Germany, 2010.","title":"Data fitting and uncertainty: A practical introduction to weighted least squares and beyond","authors":"Tilo Strutz","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2015] D. Wang, F. Nie, and H. Huang. Feature selection via global redundancy minimization. IEEE Transactions on Knowledge and Data Engineering, 27(10):2743-2755, Oct 2015.","title":"Feature selection via global redundancy minimization","authors":"D Wang, F Nie, H Huang","year":"2015","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wold et al., 1984] Svante Wold, Arnold Ruhe, Herman Wold, and WJ Dunn, III. The collinearity problem in lin- ear regression. the partial least squares (pls) approach to generalized inverses. SIAM Journal on Scientific and Sta- tistical Computing, 5(3):735-743, 1984.","title":"The collinearity problem in linear regression. the partial least squares (pls) approach to generalized inverses","authors":"Svante Wold, Arnold Ruhe, Herman Wold, Dunn, Iii","year":"1984","venue":"SIAM Journal on Scientific and Statistical Computing","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xiang et al., 2012] Shiming Xiang, Feiping Nie, Gaofeng Meng, Chunhong Pan, and Changshui Zhang. Discrimi- native least squares regression for multiclass classification and feature selection. IEEE transactions on neural net- works and learning systems, 23(11):1738-1754, 2012.","title":"Discriminative least squares regression for multiclass classification and feature selection","authors":"Shiming Xiang, Feiping Nie, Gaofeng Meng, Chunhong Pan, Changshui Zhang","year":"2012","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xu et al., 2010] Zenglin Xu, Irwin King, Michael Rung- Tsong Lyu, and Rong Jin. Discriminative semi-supervised feature selection via manifold regularization. IEEE Trans- actions on Neural Networks, 21(7):1033-1047, July 2010.","title":"Discriminative semi-supervised feature selection via manifold regularization","authors":"Xu","year":"2010","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":"/proceedings/2009/219"},{"position":22,"raw":"[Xu et al., 2016] J. Xu, B. Tang, H. He, and H. Man. Semisu- pervised feature selection based on relevance and redun- dancy criteria. IEEE Transactions on Neural Networks and Learning Systems, PP(99):1-11, 2016.","title":"Semisupervised feature selection based on relevance and redundancy criteria","authors":"J Xu, B Tang, H He, H Man","year":"2016","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yang et al., 2011] Yi Yang, Heng Tao Shen, Zhigang Ma, Zi Huang, and Xiaofang Zhou. ℓ 2,1 -norm regularized dis- criminative feature selection for unsupervised learning. In International Joint Conference on Artificial Intelligence, pages 1589-1594, 2011.","title":"ℓ 2,1 -norm regularized discriminative feature selection for unsupervised learning","authors":"Yi Yang, Heng Tao Shen, Zhigang Ma, Zi Huang, Xiaofang Zhou","year":"2011","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhao and Liu, 2007] Zheng Zhao and Huan Liu. Semi- supervised feature selection via spectral analysis. In Pro- ceedings of the 2007 SIAM International Conference on Data Mining, pages 641-646, 2007.","title":"Semisupervised feature selection via spectral analysis","authors":"Zheng Zhao, Huan Liu","year":"2007","venue":"Proceedings of the 2007 SIAM International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhao et al., 2008] Jidong Zhao, Ke Lu, and Xiaofei He. Lo- cality sensitive semi-supervised feature selection. Neuro- computing, 71:1842 -1849, 2008.","title":"Locality sensitive semi-supervised feature selection","authors":"Jidong Zhao, Ke Lu, Xiaofei He","year":"2008","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}