{"type":"ScholarlyArticle","title":"Discriminative Feature Selection via A Structured Sparse Subspace Learning Module","authors":[{"name":"Zheng Wang","slug":"zheng-wang","given":"Zheng","family":"Wang","affiliation":"School of Computer Science and Center for OPTical IMagery Analysis and Learning (OPTIMAL) , Northwestern Polytechnical University , Xi'an , 710072 , P. R. China"},{"name":"Feiping Nie","slug":"feiping-nie","given":"Feiping","family":"Nie","affiliation":"School of Computer Science and Center for OPTical IMagery Analysis and Learning (OPTIMAL) , Northwestern Polytechnical University , Xi'an , 710072 , P. R. China"},{"name":"Lai Tian","slug":"lai-tian","given":"Lai","family":"Tian","affiliation":"School of Computer Science and Center for OPTical IMagery Analysis and Learning (OPTIMAL) , Northwestern Polytechnical University , Xi'an , 710072 , P. R. China"},{"name":"Rong Wang","slug":"rong-wang","given":"Rong","family":"Wang","affiliation":"School of Computer Science and Center for OPTical IMagery Analysis and Learning (OPTIMAL) , Northwestern Polytechnical University , Xi'an , 710072 , P. R. China; School of Cybersecurity , Northwestern Polytechnical University , Xi'an , 710072 , P. R. China"},{"name":"Xuelong Li","slug":"xuelong-li","given":"Xuelong","family":"Li","affiliation":"School of Computer Science and Center for OPTical IMagery Analysis and Learning (OPTIMAL) , Northwestern Polytechnical University , Xi'an , 710072 , P. R. China"}],"abstract":"In this paper, we first propose a novel Structured Sparse Subspace Learning S^3L module to address the long-standing subspace sparsity issue. Elicited by proposed module, we design a new discriminative feature selection method, named Subspace Sparsity Discriminant Feature Selection S^2DFS which enables the following new functionalities: 1) Proposed S^2DFS method directly joints trace ratio objective and  structured sparse subspace constraint via L2,0-norm to learn a row-sparsity subspace, which improves the discriminability of model and overcomes the parameter-tuning trouble with comparison to the methods used L2,1-norm regularization; 2) An alternative iterative optimization algorithm based on the proposed S^3L module is presented to explicitly solve the proposed problem with a closed-form solution and strict convergence proof. To our best knowledge, such objective function and solver are first proposed in this paper, which provides a new though for the development of feature selection methods. Extensive experiments conducted on several high-dimensional datasets demonstrate the discriminability of selected features via S^2DFS with comparison to several related SOTA feature selection methods. Source matlab code: https://github.com/StevenWangNPU/L20-FS.","doi":"10.24963/ijcai.2020/416","url":"https://ijcai.org/proceedings/2020/416","pdf":"https://ijcai.org/proceedings/2020/0416.pdf","markdown":"https://ijcai.org/proceedings/2020/416.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/416","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":416,"pages":{"first":3009,"last":3015},"keywords":["Data Mining: Feature Extraction, Selection and Dimensionality Reduction","Machine Learning: Feature Selection; Learning Sparse Models"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24451/file_24451.m3u8","length_sec":298,"provider":"hls"}],"references":[{"position":1,"raw":"[Abid et al., 2019] Abubakar Abid, Muhammad Fatih Balin, and James Zou. Concrete autoencoders for differen- tiable feature selection and reconstruction. arXiv preprint arXiv:1901.09346, 2019.","title":"Concrete autoencoders for differentiable feature selection and reconstruction","authors":"Abubakar Abid, Muhammad Fatih Balin, James Zou","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bishop, 2006] Christopher M Bishop. Pattern recognition and machine learning. springer, 2006.","title":"Pattern recognition and machine learning","authors":"M Christopher, Bishop","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai et al., 2013] Xiao Cai, Feiping Nie, and Heng Huang. Exact top-k feature selection via l2, 0-norm constraint. In Proceedings of the Twenty-Third international joint con- ference on Artificial Intelligence, pages 1240-1246, 2013.","title":"Exact top-k feature selection via l2, 0-norm constraint","authors":"Xiao Cai, Feiping Nie, Heng Huang","year":"2013","venue":"Proceedings of the Twenty-Third international joint conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2013/187"},{"position":4,"raw":"[Du et al., 2018] Xingzhong Du, Feiping Nie, Weiqing Wang, Yi Yang, and Xiaofang Zhou. Exploiting combina- tion effect for unsupervised feature selection by ℓ 2,0 -norm. IEEE transactions on neural networks and learning sys- tems, 30(1):201-214, 2018.","title":"Exploiting combination effect for unsupervised feature selection by ℓ 2,0 -norm","authors":"Du","year":"2018","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gu et al., 2012] Quanquan Gu, Zhenhui Li, and Jiawei Han. Generalized fisher score for feature selection. arXiv preprint arXiv:1202.3725, 2012.","title":"Generalized fisher score for feature selection","authors":"Gu","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Han et al., 2018] Kai Han, Yunhe Wang, Chao Zhang, Chao Li, and Chao Xu. Autoencoder inspired unsupervised fea- ture selection. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2941-2945. IEEE, 2018.","title":"Autoencoder inspired unsupervised feature selection","authors":"Kai Han, Yunhe Wang, Chao Zhang, Chao Li, Chao Xu","year":"2018","venue":"IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2012] Ran He, Tieniu Tan, Liang Wang, and Wei- Shi Zheng. ℓ 2,1 regularized correntropy for robust feature selection. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 2504-2511. IEEE, 2012.","title":"ℓ 2,1 regularized correntropy for robust feature selection","authors":"Ran He, Tieniu Tan, Liang Wang, Wei-Shi Zheng","year":"2012","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Horn and Johnson, 2012] Roger A Horn and Charles R Johnson. Matrix analysis. Cambridge university press, 2012.","title":"Matrix analysis","authors":"A Roger, Charles R Horn, Johnson","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Liao et al., 2015] Shengcai Liao, Yang Hu, Xiangyu Zhu, and Stan Z Li. Person re-identification by local maximal occurrence representation and metric learning. In Proceed- ings of the IEEE conference on computer vision and pat- tern recognition, pages 2197-2206, 2015.","title":"Person re-identification by local maximal occurrence representation and metric learning","authors":"Liao","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Maldonado and Weber, 2009] Sebastián Maldonado and Richard Weber. A wrapper method for feature selection using support vector machines. Information Sciences, 179(13):2208-2217, 2009.","title":"A wrapper method for feature selection using support vector machines","authors":"Weber ; Maldonado, Sebastián Maldonado, Richard Weber","year":"2009","venue":"Information Sciences","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ming and Ding, 2019] Di Ming and Chris Ding. Robust flexible feature selection via exclusive l21 regularization. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pages 3158-3164. AAAI Press, 2019.","title":"Robust flexible feature selection via exclusive l21 regularization","authors":"Di Ming, Chris Ding","year":"2019","venue":"Proceedings of the 28th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2019/438"},{"position":12,"raw":"[Nie et al., 2010] Feiping Nie, Heng Huang, Xiao Cai, and Chris H Ding. Efficient and robust feature selection via joint ℓ 2,1 -norms minimization. In Advances in neural in- formation processing systems, pages 1813-1821, 2010.","title":"Efficient and robust feature selection via joint ℓ 2,1 -norms minimization","authors":"Nie","year":"2010","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Nie et al., 2017] Feiping Nie, Rui Zhang, and Xuelong Li. A generalized power iteration method for solving quadratic problem on the stiefel manifold. Science China Information Sciences, 60(11):112101, 2017.","title":"A generalized power iteration method for solving quadratic problem on the stiefel manifold","authors":"Nie","year":"2017","venue":"Science China Information Sciences","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Nie et al., 2019] Feiping Nie, Zheng Wang, Rong Wang, Zhen Wang, and Xuelong Li. Towards robust discrimi- native projections learning via non-greedy l 2,1 -norm min- max. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, DOI: 10.1109/TPAMI.2019.2961877.","title":"Towards robust discriminative projections learning via non-greedy l 2,1 -norm minmax","authors":"Nie","year":"2019","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":"10.1109/tpami.2019.2961877","url":null,"internal_url":null},{"position":15,"raw":"[Pang et al., 2018] Tianji Pang, Feiping Nie, Junwei Han, and Xuelong Li. Efficient feature selection via ℓ 2,0 - norm constrained sparse regression. IEEE Transactions on Knowledge and Data Engineering, 31(5):880-893, 2018.","title":"Efficient feature selection via ℓ 2,0norm constrained sparse regression","authors":"Tianji Pang, Feiping Nie, Junwei Han, Xuelong Li","year":"2018","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Roffo et al., 2015] Giorgio Roffo, Simone Melzi, and Marco Cristani. Infinite feature selection. In Proceedings of the IEEE International Conference on Computer Vision, pages 4202-4210, 2015.","title":"Infinite feature selection","authors":"Giorgio Roffo, Simone Melzi, Marco Cristani","year":"2015","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sun et al., 2016] Ying Sun, Prabhu Babu, and Daniel P Palomar. Majorization-minimization algorithms in signal processing, communications, and machine learning. IEEE Transactions on Signal Processing, 65(3):794-816, 2016.","title":"Majorization-minimization algorithms in signal processing, communications, and machine learning","authors":"Ying Sun, Prabhu Babu, Daniel P Palomar","year":"2016","venue":"IEEE Transactions on Signal Processing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Tian et al., 2019] Lai Tian, Feiping Nie, and Xuelong Li. Learning feature sparse principal components. arXiv preprint arXiv:1904.10155, 2019.","title":"Learning feature sparse principal components","authors":"Lai Tian, Feiping Nie, Xuelong Li","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Vu et al., 2013] Vincent Q Vu, Jing Lei, et al. Minimax sparse principal subspace estimation in high dimensions. The Annals of Statistics, 41(6):2905-2947, 2013.","title":"Minimax sparse principal subspace estimation in high dimensions","authors":"Q Vincent, Jing Vu, Lei","year":"2013","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2007] Huan Wang, Shuicheng Yan, Dong Xu, Xiaoou Tang, and Thomas Huang. Trace ratio vs. ratio trace for dimensionality reduction. In 2007 IEEE Confer- ence on Computer Vision and Pattern Recognition, pages 1-8. IEEE, 2007.","title":"Trace ratio vs. ratio trace for dimensionality reduction","authors":"Huan Wang, Shuicheng Yan, Dong Xu, Xiaoou Tang, Thomas Huang","year":"2007","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2014] Zhaoran Wang, Huanran Lu, and Han Liu. Tighten after relax: Minimax-optimal sparse pca in polynomial time. In Advances in neural information pro- cessing systems, pages 3383-3391, 2014.","title":"Tighten after relax: Minimax-optimal sparse pca in polynomial time","authors":"Wang","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xiang et al., 2012] Shiming Xiang, Feiping Nie, Gaofeng Meng, Chunhong Pan, and Changshui Zhang. Discrimi- native least squares regression for multiclass classification and feature selection. IEEE transactions on neural net- works and learning systems, 23(11):1738-1754, 2012.","title":"Discriminative least squares regression for multiclass classification and feature selection","authors":"Shiming Xiang, Feiping Nie, Gaofeng Meng, Chunhong Pan, Changshui Zhang","year":"2012","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xu et al., 2018] Jie Xu, Lei Luo, Cheng Deng, and Heng Huang. Bilevel distance metric learning for robust image recognition. In Advances in Neural Information Process- ing Systems, pages 4198-4207, 2018.","title":"Bilevel distance metric learning for robust image recognition","authors":"Jie Xu, Lei Luo, Cheng Deng, Heng Huang","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yan et al., 2016] Hui Yan, Jian Yang, and Jingyu Yang. Robust joint feature weights learning framework. IEEE Transactions on Knowledge and Data Engineering, 28(5):1327-1339, 2016.","title":"Robust joint feature weights learning framework","authors":"Hui Yan, Jian Yang, Jingyu Yang","year":"2016","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yang and Xu, 2015] Wenzhuo Yang and Huan Xu. Stream- ing sparse principal component analysis. In International Conference on Machine Learning, pages 494-503, 2015.","title":"Streaming sparse principal component analysis","authors":"Yang, Xu ; ] Wenzhuo Yang, Huan Xu","year":"2015","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2020] H. Zhang, J. Wang, Z. Sun, J. M. Zu- rada, and N. R. Pal. Feature selection for neural networks using group lasso regularization. IEEE Transactions on Knowledge and Data Engineering, 32(4):659-673, 2020.","title":"Feature selection for neural networks using group lasso regularization","authors":"H Zhang, J Wang, Z Sun, J M Zurada, N R","year":"2020","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}