{"type":"ScholarlyArticle","title":"Top-k Supervise Feature Selection via ADMM for Integer Programming","authors":[{"name":"Mingyu Fan","slug":"mingyu-fan","given":"Mingyu","family":"Fan","affiliation":"School of Maths & Info. Science , Wenzhou University , Wenzhou 325035 , China"},{"name":"Xiaojun Chang","slug":"xiaojun-chang","given":"Xiaojun","family":"Chang","affiliation":"School of Computer Science , Carnegie Mellon University , PA 15213 , USA"},{"name":"Xiaoqin Zhang","slug":"xiaoqin-zhang","given":"Xiaoqin","family":"Zhang","affiliation":"School of Maths & Info. Science , Wenzhou University , Wenzhou 325035 , China"},{"name":"Di Wang","slug":"di-wang","given":"Di","family":"Wang","affiliation":"School of Maths & Info. Science , Wenzhou University , Wenzhou 325035 , China"},{"name":"Liang Du","slug":"liang-du","given":"Liang","family":"Du","affiliation":"School of Computer & Information Technology , Shanxi University , Taiyuan 030006 China"}],"abstract":"Recently, structured sparsity inducing based feature selection has become a hot topic in machine learning and pattern recognition. Most of the sparsity inducing feature selection methods are designed to rank all features by certain criterion and then select the k top ranked features, where k is an integer. However, the k top features are usually not the top k features and therefore maybe a suboptimal result. In this paper, we propose a novel supervised feature selection method to directly identify the top k features. The new method is formulated as a classic regularized least squares regression model with two groups of variables. The problem with respect to one group of the variables turn out to be a 0-1 integer programming, which had been considered very hard to solve. To address this, we utilize an efficient optimization method to solve the integer programming, which first replaces the discrete 0-1 constraints with two continuous constraints and then utilizes the alternating direction method of multipliers to optimize the equivalent problem. The obtained result is the top subset with k features under the proposed criterion rather than the subset of k top features. Experiments have been conducted on benchmark data sets to show the effectiveness of proposed method.","doi":"10.24963/ijcai.2017/228","url":"https://ijcai.org/proceedings/2017/228","pdf":"https://ijcai.org/proceedings/2017/0228.pdf","markdown":"https://ijcai.org/proceedings/2017/228.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/228","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":228,"pages":{"first":1646,"last":1653},"keywords":["Machine Learning: Data Mining","Machine Learning: Feature Selection/Construction","Machine Learning: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Amaldi and Kann, 1998] Edoardo Amaldi and Viggo Kann. On the approximability of minimizing nonzero variables or unsatis- fied relations in linear systems. Theoretical Computer Science, 209(1):237 -260, 1998.","title":"On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems","authors":"Kann Amaldi, Edoardo Amaldi, Viggo Kann","year":"1998","venue":"Theoretical Computer Science","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Argyriou et al., 2007] Andreas Argyriou, Theodoros Evgeniou, and Massimiliano Pontil. Multi-task feature learning. In NIP- S. 2007.","title":"Multi-task feature learning","authors":"Andreas Argyriou, Theodoros Evgeniou, Massimiliano Pontil","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Belhumeur et al., 1997] P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman. Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):711-720, Jul 1997.","title":"Eigenfaces vs. fisherfaces: recognition using class specific linear projection","authors":"P N Belhumeur, J P Hespanha, D J Kriegman","year":"1997","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cai et al., 2010] Deng Cai, Chiyuan Zhang, and Xiaofei He. Un- supervised feature selection for multi-cluster data. In KDD, 2010.","title":"Unsupervised feature selection for multi-cluster data","authors":"Deng Cai, Chiyuan Zhang, Xiaofei He","year":"2010","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cai et al., 2013] Xiao Cai, Feiping Nie, and Heng Huang. Exact top-k feature selection via l2,0-norm constraint. In IJCAI, 2013.","title":"Exact top-k feature selection via l2,0-norm constraint","authors":"Xiao Cai, Feiping Nie, Heng Huang","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2013/187"},{"position":6,"raw":"[Chang et al., 2014] Xiaojun Chang, Feiping Nie, Yi Yang, and Heng Huang. A convex formulation for semi-supervised multi- label feature selection. In AAAI, 2014.","title":"A convex formulation for semi-supervised multilabel feature selection","authors":"Xiaojun Chang, Feiping Nie, Yi Yang, Heng Huang","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chang et al., 2016] Xiaojun Chang, Feiping Nie, Sen Wang, Y- i Yang, Xiaofang Zhou, and Chengqi Zhang. Compound rank- k projections for bilinear analysis. IEEE Trans. Neural Netw. Learning Syst., 27(7):1502-1513, 2016.","title":"Compound rankk projections for bilinear analysis","authors":"Xiaojun Chang, Feiping Nie, Sen Wang, Yi Yang, Xiaofang Zhou, Chengqi Zhang","year":"2016","venue":"IEEE Trans. Neural Netw. Learning Syst","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chang et al., 2017] Xiaojun Chang, Zhigang Ma, Yi Yang, Zhiqiang Zeng, and Alexander G. Hauptmann. Bi-level seman- tic representation analysis for multimedia event detection. IEEE Trans. Cybernetics, 47(5):1180-1197, 2017.","title":"Bi-level semantic representation analysis for multimedia event detection","authors":"Xiaojun Chang, Zhigang Ma, Yi Yang, Zhiqiang Zeng, Alexander G Hauptmann","year":"2017","venue":"IEEE Trans. Cybernetics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Chatfield et al., 2011] K. Chatfield, V. Lempitsky, A. Vedaldi, and A. Zisserman. The devil is in the details: an evaluation of recent feature encoding methods. In BMVC, 2011.","title":"The devil is in the details: an evaluation of recent feature encoding methods","authors":"K Chatfield, V Lempitsky, A Vedaldi, A Zisserman","year":"2011","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Destrero et al., 2009] A. Destrero, C. De Mol, F. Odone, and A. Verri. A sparsity-enforcing method for learning face fea- tures. IEEE Transactions on Image Processing, 18(1):188-201, Jan 2009.","title":"A sparsity-enforcing method for learning face features","authors":"A Destrero, C De Mol, F Odone, A Verri","year":"2009","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Donoho, 2004] David L. Donoho. For most large underdetermined systems of linear equations the minimal ℓ1-norm solution is al- so the sparsest solution. Communications on Pure and Applied Mathematics, 59:797-829, 2004.","title":"For most large underdetermined systems of linear equations the minimal ℓ1-norm solution is also the sparsest solution","authors":"L David, Donoho","year":"2004","venue":"Communications on Pure and Applied Mathematics","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Du and Shen, 2015] Liang Du and Yi-Dong Shen. Unsupervised feature selection with adaptive structure learning. In KDD, 2015.","title":"Unsupervised feature selection with adaptive structure learning","authors":"Liang Du, Yi-Dong Shen","year":"2015","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Duda et al., 2000] Richard O. Duda, Peter E. Hart, and David G. Stork. Pattern Classification (2Nd Edition). Wiley-Interscience, 2000.","title":"Pattern Classification (2Nd Edition)","authors":"Richard O Duda, Peter E Hart, David G Stork","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Elad, 2010] Michael Elad. Sparse and Redundant Representation- s: From Theory to Applications in Signal and Image Processing. Springer Publishing Company, Incorporated, 1st edition, 2010.","title":"Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing","authors":"Michael Elad","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Guyon and Elisseeff, 2003] Isabelle Guyon and André Elisseeff. An introduction to variable and feature selection. Journal of Ma- chine Learning Research, 3:1157-1182, March 2003.","title":"An introduction to variable and feature selection","authors":"Elisseeff Guyon, Isabelle Guyon, André Elisseeff","year":"2003","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Guyon et al., 2002] Isabelle Guyon, Jason Weston, Stephen Barn- hill, and Vladimir Vapnik. Gene selection for cancer clas- sification using support vector machines. Machine Learning, 46(1):389-422, 2002.","title":"Gene selection for cancer classification using support vector machines","authors":"Isabelle Guyon, Jason Weston, Stephen Barnhill, Vladimir Vapnik","year":"2002","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Hall and Smith, 1999] Mark A. Hall and Lloyd A. Smith. Feature selection for machine learning: Comparing a correlation-based filter approach to the wrapper. In IFAIRSC, 1999.","title":"Feature selection for machine learning: Comparing a correlation-based filter approach to the wrapper","authors":"Mark A Hall, Lloyd A Smith","year":"1999","venue":"IFAIRSC","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Han et al., 2015] Y. Han, Y. Yang, Y. Yan, Z. Ma, N. Sebe, and X. Zhou. Semisupervised feature selection via spline regression for video semantic recognition. IEEE Transactions on Neural Networks and Learning Systems, 26(2):252-264, Feb 2015.","title":"Semisupervised feature selection via spline regression for video semantic recognition","authors":"Y Han, Y Yang, Y Yan, Z Ma, N Sebe, X Zhou","year":"2015","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[He et al., 2006] Xiaofei He, Deng Cai, and Partha Niyogi. Lapla- cian score for feature selection. In NIPS. 2006.","title":"Laplacian score for feature selection","authors":"Xiaofei He, Deng Cai, Partha Niyogi","year":"2006","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Jenatton et al., 2011] Rodolphe Jenatton, Jean-Yves Audibert, and Francis Bach. Structured variable selection with sparsity- inducing norms. J. Mach. Learn. Res., 12:2777-2824, November 2011.","title":"Structured variable selection with sparsityinducing norms","authors":"Rodolphe Jenatton, Jean-Yves Audibert, Francis Bach","year":"2011","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kira and Rendell, 1992] Kenji Kira and Larry A. Rendell. A prac- tical approach to feature selection. In IWML, 1992.","title":"A practical approach to feature selection","authors":"Kenji Kira, Larry A Rendell","year":"1992","venue":"IWML","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lan et al., 2015] Zhenzhong Lan, Ming Lin, Xuanchong Li, A. G. Hauptmann, and B. Raj. Beyond gaussian pyramid: Multi-skip feature stacking for action recognition. In 2015 IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR), pages 204-212, June 2015.","title":"Beyond gaussian pyramid: Multi-skip feature stacking for action recognition","authors":"Zhenzhong Lan, Ming Lin, Xuanchong Li, A G Hauptmann, B Raj","year":"2015","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Nie et al., 2010] Feiping Nie, Heng Huang, Xiao Cai, and Chris H. Ding. Efficient and robust feature selection via joint ℓ2,1-norms minimization. In NIPS, 2010.","title":"Efficient and robust feature selection via joint ℓ2,1-norms minimization","authors":"Feiping Nie, Heng Huang, Xiao Cai, Chris H Ding","year":"2010","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Peng et al., 2005] Hanchuan Peng, Fuhui Long, and C. Ding. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8):1226-1238, Aug 2005.","title":"Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy","authors":"Peng","year":"2005","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tao et al., 2016] Hong Tao, Chenping Hou, Feiping Nie, Yuanyuan Jiao, and Dongyun Yi. Effective discriminative feature selection with nontrivial solution. IEEE Transactions on Neural Networks and Learning Systems, 27(4):796-808, April 2016.","title":"Effective discriminative feature selection with nontrivial solution","authors":"Tao","year":"2016","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tibshirani, 1994] Robert Tibshirani. Regression shrinkage and s- election via the lasso. Journal of the Royal Statistical Society, Series B, 58:267-288, 1994.","title":"Regression shrinkage and selection via the lasso","authors":"Robert Tibshirani","year":"1994","venue":"Journal of the Royal Statistical Society, Series B","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2007] Li Wang, Ji Zhu, and Hui Zou. Hybrid huber- ized support vector machines for microarray classification. In ICML, 2007.","title":"Hybrid huberized support vector machines for microarray classification","authors":"Li Wang, Ji Zhu, Hui Zou","year":"2007","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2014] Liping Wang, Songcan Chen, and Yuanping Wang. A unified algorithm for mixed ℓ2,p-minimizations and its application in feature selection. Computational Optimization and Applications, 58(2):409-421, June 2014.","title":"A unified algorithm for mixed ℓ2,p-minimizations and its application in feature selection","authors":"Liping Wang, Songcan Chen, Yuanping Wang","year":"2014","venue":"Computational Optimization and Applications","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wu and Ghanem, 2016] Baoyuan Wu and Bernard Ghanem. ℓp- box ADMM: A versatile framework for integer programming. CoRR, abs/1604.07666, 2016.","title":"ℓpbox ADMM: A versatile framework for integer programming","authors":"Ghanem Wu, Baoyuan Wu, Bernard Ghanem","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xiang et al., 2012] S. Xiang, F. Nie, G. Meng, C. Pan, and C. Zhang. Discriminative least squares regression for multiclass classification and feature selection. IEEE Transactions on Neural Networks and Learning Systems, 23(11):1738-1754, Nov 2012.","title":"Discriminative least squares regression for multiclass classification and feature selection","authors":"S Xiang, F Nie, G Meng, C Pan, C Zhang","year":"2012","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhang et al., 2012] S. Zhang, J. Huang, H. Li, and D. N. Metaxas. Automatic image annotation and retrieval using group sparsity. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 42(3):838-849, June 2012.","title":"Automatic image annotation and retrieval using group sparsity","authors":"S Zhang, J Huang, H Li, D N Metaxas","year":"2012","venue":"IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhao and Liu, 2007] Zheng Zhao and Huan Liu. Spectral feature selection for supervised and unsupervised learning. In ICML, 2007.","title":"Spectral feature selection for supervised and unsupervised learning","authors":"Zheng Zhao, Huan Liu","year":"2007","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zou and Hastie, 2005] Hui Zou and Trevor Hastie. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society. Series B: Statistical Methodology, 67(2):301- 320, 2005.","title":"Regularization and variable selection via the elastic net","authors":"Hastie ; ] Hui Zou, Trevor Zou, Hastie","year":"2005","venue":"Journal of the Royal Statistical Society. Series B: Statistical Methodology","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}