{"type":"ScholarlyArticle","title":"Robust Flexible Feature Selection via Exclusive L21 Regularization","authors":[{"name":"Di Ming","slug":"di-ming","given":"Di","family":"Ming","affiliation":"Department of Computer Science and Engineering , University of Texas at Arlington , USA"},{"name":"Chris Ding","slug":"chris-ding","given":"Chris","family":"Ding","affiliation":"Department of Computer Science and Engineering , University of Texas at Arlington , USA"}],"abstract":"Recently, exclusive lasso has demonstrated its promising results in selecting discriminative features for each class. The sparsity is enforced on each feature across all the classes via L12-norm. However, the exclusive sparsity of L12-norm could not screen out a large amount of irrelevant and redundant noise features in high-dimensional data space, since each feature belongs to at least one class. Thus, in this paper, we introduce a novel regularization called \"exclusive L21\", which is short for \"L21 with exclusive lasso\", towards robust flexible feature selection. The exclusive L21 regularization is the mix of L21-norm and L12-norm, which brings out joint sparsity at inter-group level and exclusive sparsity at intra-group level simultaneously. An efficient augmented Lagrange multipliers based optimization algorithm is proposed to iteratively solve the exclusive L21 regularization in a row-wise fashion. Extensive experiments on twelve benchmark datasets demonstrate the effectiveness of the proposed regularization and the optimization algorithm as compared to state-of-the-arts.","doi":"10.24963/ijcai.2019/438","url":"https://ijcai.org/proceedings/2019/438","pdf":"https://ijcai.org/proceedings/2019/0438.pdf","markdown":"https://ijcai.org/proceedings/2019/438.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/438","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":438,"pages":{"first":3158,"last":3164},"keywords":["Machine Learning: Feature Selection ; Learning Sparse Models","Machine Learning: Classification"],"videos":[],"references":[{"position":1,"raw":"[Bhattacharjee et al., 2001] Arindam Bhattacharjee, William G. Richards, Jane Staunton, Cheng Li, et al. Classification of human lung carcinomas by mrna expres- sion profiling reveals distinct adenocarcinoma subclasses. Proceedings of the National Academy of Sciences, 98(24):13790-13795, 2001.","title":"Classification of human lung carcinomas by mrna expression profiling reveals distinct adenocarcinoma subclasses","authors":"Arindam Bhattacharjee, William G Richards, Jane Staunton, Cheng Li","year":"2001","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Campbell and Allen, 2017] Frederick Campbell and Genev- era I. Allen. Within group variable selection through the exclusive lasso. Electronic Journal of Statistics, 11(2):4220-4257, 2017.","title":"Frederick Campbell and Genevera I. Allen. Within group variable selection through the exclusive lasso","authors":"Allen Campbell","year":"2017","venue":"Electronic Journal of Statistics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cavazza et al., 2018] Jacopo Cavazza, Pietro Morerio, Ben- jamin Haeffele, Connor Lane, et al. Dropout as a low-rank regularizer for matrix factorization. In Proceedings of the Twenty-First International Conference on Artificial Intel- ligence and Statistics, pages 435-444, 2018.","title":"Dropout as a low-rank regularizer for matrix factorization","authors":"Jacopo Cavazza, Pietro Morerio, Benjamin Haeffele, Connor Lane","year":"2018","venue":"Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chang and Lin, 2011] Chih-Chung Chang and Chih-Jen Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1-27:27, 2011.","title":"LIBSVM: A library for support vector machines","authors":"Chang, Lin ; ] Chih-Chung Chang, Chih-Jen Lin","year":"2011","venue":"ACM Transactions on Intelligent Systems and Technology","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ciarelli and Oliveira, 2009] Patrick M. Ciarelli and Elias Oliveira. Agglomeration and elimination of terms for di- mensionality reduction. In 2009 Ninth International Con- ference on Intelligent Systems Design and Applications, pages 547-552, Nov 2009.","title":"Agglomeration and elimination of terms for dimensionality reduction","authors":"Patrick M Ciarelli, Elias Oliveira","year":"2009","venue":"Ninth International Conference on Intelligent Systems Design and Applications","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ding and Peng, 2003] Chris Ding and Hanchuan Peng. Minimum redundancy feature selection from microarray gene expression data. In Computational Systems Bioin- formatics. Proceedings of the 2003 IEEE Bioinformatics Conference. CSB2003, pages 523-528, Aug 2003.","title":"Minimum redundancy feature selection from microarray gene expression data","authors":"Chris Ding, Hanchuan Peng","year":"2003","venue":"Proceedings of the 2003 IEEE Bioinformatics Conference","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gui et al., 2017] Jie Gui, Zhenan Sun, Shuiwang Ji, Dacheng Tao, and Tieniu Tan. Feature selection based on structured sparsity: A comprehensive study. IEEE Transactions on Neural Networks and Learning Systems, 28(7):1490-1507, July 2017.","title":"Feature selection based on structured sparsity: A comprehensive study","authors":"Jie Gui, Zhenan Sun, Shuiwang Ji, Dacheng Tao, Tieniu Tan","year":"2017","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guyon and Elisseeff, 2003] Isabelle Guyon and André Elis- seeff. An introduction to variable and feature selec- tion. Journal of Machine Learning Research, 3:1157- 1182, March 2003.","title":"Isabelle Guyon and André Elisseeff. An introduction to variable and feature selection","authors":"Elisseeff Guyon","year":"2003","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Guyon et al., 2002] Isabelle Guyon, Jason Weston, Stephen Barnhill, and Vladimir Vapnik. Gene selection for can- cer classification using support vector machines. Machine Learning, 46(1):389-422, Jan 2002.","title":"Gene selection for cancer classification using support vector machines","authors":"Isabelle Guyon, Jason Weston, Stephen Barnhill, Vladimir Vapnik","year":"2002","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kong et al., 2014] Deguang Kong, Ryohei Fujimaki, Ji Liu, Feiping Nie, and Chris Ding. Exclusive feature learning on arbitrary structures via ℓ 1,2 -norm. In Advances in Neu- ral Information Processing Systems 27, pages 1655-1663, 2014.","title":"Exclusive feature learning on arbitrary structures via ℓ 1,2 -norm","authors":"Kong","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lecun et al., 1998] Yann Lecun, Léon Bottou, Yoshua Ben- gio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, Nov 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2009] Jun Liu, Shuiwang Ji, and Jieping Ye. Multi-task feature learning via efficient ℓ 2,1 -norm mini- mization. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, pages 339-348, 2009.","title":"Multi-task feature learning via efficient ℓ 2,1 -norm minimization","authors":"Liu","year":"2009","venue":"Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ming et al., 2019] Di Ming, Chris Ding, and Feiping Nie. A probabilistic derivation of lasso and l12-norm feature selections. In Proceedings of the Thirty-Third AAAI Con- ference on Artificial Intelligence (AAAI-19), 2019.","title":"A probabilistic derivation of lasso and l12-norm feature selections","authors":"Di Ming, Chris Ding, Feiping Nie","year":"2019","venue":"Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Nie et al., 2010] Feiping Nie, Heng Huang, Xiao Cai, and Chris Ding. Efficient and robust feature selection via joint ℓ 2,1 -norms minimization. In Advances in Neural Informa- tion Processing Systems 23, pages 1813-1821, 2010.","title":"Efficient and robust feature selection via joint ℓ 2,1 -norms minimization","authors":"Nie","year":"2010","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nutt et al., 2003] Catherine L. Nutt, D. R. Mani, Re- becca A. Betensky, Pablo Tamayo, et al. Gene expression- based classification of malignant gliomas correlates better with survival than histological classification. Cancer Re- search, 63(7):1602-1607, 2003.","title":"Gene expressionbased classification of malignant gliomas correlates better with survival than histological classification","authors":"Catherine L Nutt, D R Mani, ;, Pablo Tamayo","year":"2003","venue":"Cancer Research","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Peng et al., 2005] Hanchuan Peng, Fuhui Long, and Chris Ding. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min- redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8):1226-1238, Aug 2005.","title":"Feature selection based on mutual information criteria of max-dependency, max-relevance, and minredundancy","authors":"Peng","year":"2005","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Quattoni et al., 2009] Ariadna Quattoni, Xavier Carreras, Michael Collins, and Trevor Darrell. An efficient projec- tion for ℓ 1,∞ regularization. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 857-864, 2009.","title":"An efficient projection for ℓ 1,∞ regularization","authors":"Ariadna Quattoni, Xavier Carreras, Michael Collins, Trevor Darrell","year":"2009","venue":"Proceedings of the 26th Annual International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ramaswamy et al., 2001] Sridhar Ramaswamy, Pablo Tamayo, Ryan Rifkin, Sayan Mukherjee, et al. Multiclass cancer diagnosis using tumor gene expression signa- tures. Proceedings of the National Academy of Sciences, 98(26):15149-15154, 2001.","title":"Multiclass cancer diagnosis using tumor gene expression signatures","authors":"Sridhar Ramaswamy, Pablo Tamayo, Ryan Rifkin, Sayan Mukherjee","year":"2001","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Robnik-Šikonja and Kononenko, 2003] Marko Robnik- Šikonja and Igor Kononenko. Theoretical and empirical analysis of relieff and rrelieff. Machine Learning, 53(1):23-69, Oct 2003.","title":"Theoretical and empirical analysis of relieff and rrelieff","authors":"Marko Robnik, -Šikonja, Igor Kononenko","year":"2003","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sim et al., 2002] Terence Sim, Simon Baker, and Maan Bsat. The cmu pose, illumination, and expression (pie) database. In Proceedings of Fifth IEEE International Con- ference on Automatic Face Gesture Recognition, pages 53-58, 2002.","title":"The cmu pose, illumination, and expression (pie) database","authors":"Terence Sim, Simon Baker, Maan Bsat","year":"2002","venue":"Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tibshirani, 1996] Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), 58(1):267-288, 1996.","title":"Regression shrinkage and selection via the lasso","authors":"Robert Tibshirani","year":"1996","venue":"Journal of the Royal Statistical Society. Series B (Methodological)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2007] Li Wang, Ji Zhu, and Hui Zou. Hybrid huberized support vector machines for microarray classi- fication. In Proceedings of the 24th International Confer- ence on Machine Learning, pages 983-990, 2007.","title":"Hybrid huberized support vector machines for microarray classification","authors":"Li Wang, Ji Zhu, Hui Zou","year":"2007","venue":"Proceedings of the 24th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yang et al., 2006] Kun Yang, Zhipeng Cai, Jianzhong Li, and Guohui Lin. A stable gene selection in microarray data analysis. BMC Bioinformatics, 7(1):228, Apr 2006.","title":"A stable gene selection in microarray data analysis","authors":"Kun Yang, Zhipeng Cai, Jianzhong Li, Guohui Lin","year":"2006","venue":"BMC Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhao et al., 2009] Peng Zhao, Guilherme Rocha, and Bin Yu. The composite absolute penalties family for grouped and hierarchical variable selection. The Annals of Statis- tics, 37(6A):3468-3497, 2009.","title":"The composite absolute penalties family for grouped and hierarchical variable selection","authors":"Peng Zhao, Guilherme Rocha, Bin Yu","year":"2009","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhou et al., 2010] Yang Zhou, Rong Jin, and Steven C.H. Hoi. Exclusive lasso for multi-task feature selection. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pages 988-995, 2010.","title":"Exclusive lasso for multi-task feature selection","authors":"Yang Zhou, Rong Jin, Steven C H Hoi","year":"2010","venue":"Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhu et al., 2003] Ji Zhu, Saharon Rosset, Trevor Hastie, and Rob Tibshirani. 1-norm support vector machines. In Pro- ceedings of the 16th International Conference on Neural Information Processing Systems, pages 49-56, 2003.","title":"1-norm support vector machines","authors":"Ji Zhu, Saharon Rosset, Trevor Hastie, Rob Tibshirani","year":"2003","venue":"Proceedings of the 16th International Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}