{"type":"ScholarlyArticle","title":"Multiple Indefinite Kernel Learning for Feature Selection","authors":[{"name":"Hui Xue","slug":"hui-xue","given":"Hui","family":"Xue","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing , 210096 , China; MOE Key Laboratory of Computer Network and Information Integration ( Southeast University) , China"},{"name":"Yu Song","slug":"yu-song","given":"Yu","family":"Song","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing , 210096 , China; MOE Key Laboratory of Computer Network and Information Integration ( Southeast University) , China"},{"name":"Hai-Ming Xu","slug":"hai-ming-xu","given":"Hai-Ming","family":"Xu","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing , 210096 , China; MOE Key Laboratory of Computer Network and Information Integration ( Southeast University) , China"}],"abstract":"Multiple kernel learning for feature selection (MKL-FS) utilizes kernels to explore complex properties of features and performs better in embedded methods. However, the kernels in MKL-FS are generally limited to be positive definite. In fact, indefinite kernels often emerge in actual applications and can achieve better empirical performance. But due to the non-convexity of indefinite kernels, existing MKL-FS methods are usually inapplicable and the corresponding research is also relatively little. In this paper, we propose a novel multiple indefinite kernel feature selection method (MIK-FS) based on the primal framework of indefinite kernel support vector machine (IKSVM), which applies an indefinite base kernel for each feature and then exerts an l1-norm constraint on kernel combination coefficients to select features automatically. A two-stage algorithm is further presented to optimize the coefficients of IKSVM and kernel combination alternately. In the algorithm, we reformulate the non-convex optimization problem of primal IKSVM as a difference of convex functions (DC) programming and transform the non-convex problem into a convex one with the affine minorization approximation. Experiments on real-world datasets demonstrate that MIK-FS is superior to some related state-of-the-art methods in both feature selection and classification performance.","doi":"10.24963/ijcai.2017/448","url":"https://ijcai.org/proceedings/2017/448","pdf":"https://ijcai.org/proceedings/2017/0448.pdf","markdown":"https://ijcai.org/proceedings/2017/448.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/448","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":448,"pages":{"first":3210,"last":3216},"keywords":["Machine Learning: Feature Selection/Construction","Machine Learning: Kernel Methods","Machine Learning: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Bradley and Mangasarian, 1998] Paul S Bradley and Olvi L Mangasarian. Feature selection via concave minimization and support vector machines. In Proceedings of Fifteenth International Conference on Machine Learning, volume 98, pages 82-90, 1998.","title":"Feature selection via concave minimization and support vector machines","authors":"Mangasarian ; ] Bradley, S Paul, Bradley, Olvi L Mangasarian","year":"1998","venue":"Proceedings of Fifteenth International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chandrashekar and Sahin, 2014] Girish Chandrashekar and Ferat Sahin. A survey on feature selection methods. Com- puters & Electrical Engineering, 40(1):16-28, 2014.","title":"A survey on feature selection methods","authors":"Sahin Chandrashekar, Girish Chandrashekar, Ferat Sahin","year":"2014","venue":"Computers & Electrical Engineering","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2007] Zhenyu Chen, Jianping Li, and Liwei Wei. A multiple kernel support vector machine scheme for fea- ture selection and rule extraction from gene expression data of cancer tissue. Artificial Intelligence in Medicine, 41(2):161-175, 2007.","title":"A multiple kernel support vector machine scheme for feature selection and rule extraction from gene expression data of cancer tissue","authors":"Zhenyu Chen, Jianping Li, Liwei Wei","year":"2007","venue":"Artificial Intelligence in Medicine","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2009] Yihua Chen, Maya R Gupta, and Ben- jamin Recht. Learning kernels from indefinite similarities. In Proceedings of the Twenty-sixth Annual International Conference on Machine Learning, pages 145-152. ACM, 2009.","title":"Learning kernels from indefinite similarities","authors":"Yihua Chen, Maya R Gupta, Benjamin Recht","year":"2009","venue":"Proceedings of the Twenty-sixth Annual International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dileep and Sekhar, 2009] Aroor Dinesh Dileep and C Chan- dra Sekhar. Representation and feature selection using multiple kernel learning. In Proceedings of the Twenty- second International Joint Conference on Neural Networks, pages 717-722. IEEE, 2009.","title":"Representation and feature selection using multiple kernel learning","authors":"Aroor Dinesh, Dileep, C Chandra, Sekhar","year":"2009","venue":"Proceedings of the Twentysecond International Joint Conference on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dinh and Le Thi, 2014] Tao Pham Dinh and Hoai An Le Thi. Recent advances in dc programming and dca. In Trans- actions on Computational Intelligence XIII, pages 1-37. Springer, 2014.","title":"Recent advances in dc programming and dca","authors":"Le Dinh, ; Thi, Pham Tao, Hoai Dinh, Le An, Thi","year":"2014","venue":"Transactions on Computational Intelligence XIII","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Haasdonk and Pekalska, 2010] Bernard Haasdonk and Elżbieta Pekalska. Indefinite kernel discriminant analysis. In Proceedings of the Sixteenth International Conference on Computational Statistics, pages 221-230. Springer, 2010.","title":"Indefinite kernel discriminant analysis","authors":"Pekalska Haasdonk, Bernard Haasdonk, Elżbieta Pekalska","year":"2010","venue":"Proceedings of the Sixteenth International Conference on Computational Statistics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2016] Xiaolin Huang, Andreas Maier, Joachim Hornegger, and Johan AK Suykens. Indefinite kernels in least squares support vector machines and principal com- ponent analysis. Applied and Computational Harmonic Analysis, 2016.","title":"Indefinite kernels in least squares support vector machines and principal component analysis","authors":"Xiaolin Huang, Andreas Maier, Joachim Hornegger, Johan Ak Suykens","year":"2016","venue":"Applied and Computational Harmonic Analysis","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kowalski et al., 2009] Matthieu Kowalski, Marie Szafranski, and Liva Ralaivola. Multiple indefinite kernel learning with mixed norm regularization. In International Conference on Machine Learning, pages 545-552, 2009.","title":"Multiple indefinite kernel learning with mixed norm regularization","authors":"Matthieu Kowalski, Marie Szafranski, Liva Ralaivola","year":"2009","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liu, 2004] Chengjun Liu. Gabor-based kernel pca with frac- tional power polynomial models for face recognition. IEEE transactions on pattern analysis and machine intelligence, 26(5):572-581, 2004.","title":"Gabor-based kernel pca with fractional power polynomial models for face recognition","authors":"Chengjun Liu","year":"2004","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liwicki et al., 2012] Stephan Liwicki, Stefanos Zafeiriou, Georgios Tzimiropoulos, and Maja Pantic. Efficient online subspace learning with an indefinite kernel for visual track- ing and recognition. IEEE transactions on neural networks and learning systems, 23(10):1624-1636, 2012.","title":"Efficient online subspace learning with an indefinite kernel for visual tracking and recognition","authors":"Stephan Liwicki, Stefanos Zafeiriou, Georgios Tzimiropoulos, Maja Pantic","year":"2012","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ong et al., 2004] Cheng Soon Ong, Xavier Mary, Stéphane Canu, and Alexander J Smola. Learning with non-positive kernels. In Proceedings of the twenty-first international conference on Machine learning, page 81. ACM, 2004.","title":"Learning with non-positive kernels","authors":"Soon Cheng, Xavier Ong, Stéphane Mary, Alexander J Canu, Smola","year":"2004","venue":"Proceedings of the twenty-first international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Saigo et al., 2004] Hiroto Saigo, Jean-Philippe Vert, Nobuhisa Ueda, and Tatsuya Akutsu. Protein homology detection using string alignment kernels. Bioinformatics, 20(11):1682-1689, 2004.","title":"Protein homology detection using string alignment kernels","authors":"Hiroto Saigo, Jean-Philippe Vert, Nobuhisa Ueda, Tatsuya Akutsu","year":"2004","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Tan et al., 2010] Mingkui Tan, Li Wang, and Ivor W. Tsang. Learning sparse svm for feature selection on very high dimensional datasets. In Proceedings of the 27th Interna- tional Conference on Machine Learning (ICML-10), pages 1047-1054, 2010.","title":"Learning sparse svm for feature selection on very high dimensional datasets","authors":"Mingkui Tan, Li Wang, Ivor W Tsang","year":"2010","venue":"Proceedings of the 27th International Conference on Machine Learning (ICML-10)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Tao and An, 1997] Pham Dinh Tao and Le Thi Hoai An. Convex analysis approach to dc programming: Theory, al- gorithms and applications. Acta Mathematica Vietnamica, 22(1):289-355, 1997.","title":"Convex analysis approach to dc programming: Theory, algorithms and applications","authors":"Dinh Pham, Le Tao, Hoai Thi, An","year":"1997","venue":"Acta Mathematica Vietnamica","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Varma and Babu, 2009] Manik Varma and Bodla Rakesh Babu. More generality in efficient multiple kernel learning. In Proceedings of the Twenty-sixth Annual International Conference on Machine Learning, pages 1065-1072. ACM, 2009.","title":"More generality in efficient multiple kernel learning","authors":"Babu Varma, Manik Varma, Bodla Rakesh, Babu","year":"2009","venue":"Proceedings of the Twenty-sixth Annual International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2006] Li Wang, Ji Zhu, and Hui Zou. The dou- bly regularized support vector machine. Statistica Sinica, pages 589-615, 2006.","title":"The doubly regularized support vector machine","authors":"Li Wang, Ji Zhu, Hui Zou","year":"2006","venue":"Statistica Sinica","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Xu et al., 2009] Zenglin Xu, Rong Jin, Jieping Ye, Michael R Lyu, and Irwin King. Non-monotonic feature se- lection. In Proceedings of the Twenty-sixth Annual Interna- tional Conference on Machine Learning, pages 1145-1152. ACM, 2009.","title":"Non-monotonic feature selection","authors":"Xu","year":"2009","venue":"Proceedings of the Twenty-sixth Annual International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xu et al., 2017] Hai-Ming Xu, Hui Xue, Xiao-Hong Chen, and Yun-Yun Wang. Solving indefinite kernel support vec- tor machine with difference of convex functions program- ming. In Proceedings of the Thirty-first AAAI Conference on Artificial Intelligence, 2017.","title":"Solving indefinite kernel support vector machine with difference of convex functions programming","authors":"Hai-Ming Xu, Hui Xue, Xiao-Hong Chen, Yun-Yun Wang","year":"2017","venue":"Proceedings of the Thirty-first AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xue and Chen, 2014] Hui Xue and Songcan Chen. Discriminality-driven regularization framework for indefinite kernel machine. Neurocomputing, 133:209-221, 2014.","title":"Discriminality-driven regularization framework for indefinite kernel machine","authors":"Chen ; ] Xue, Songcan Hui Xue, Chen","year":"2014","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yamada et al., 2014] Makoto Yamada, Wittawat Jitkrittum, Leonid Sigal, Eric P Xing, and Masashi Sugiyama. High- dimensional feature selection by feature-wise kernelized lasso. Neural computation, 26(1):185-207, 2014.","title":"Highdimensional feature selection by feature-wise kernelized lasso","authors":"Makoto Yamada, Wittawat Jitkrittum, Leonid Sigal, Eric P Xing, Masashi Sugiyama","year":"2014","venue":"Neural computation","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}