{"type":"ScholarlyArticle","title":"Worst-Case Discriminative Feature Selection","authors":[{"name":"Shuangli Liao","slug":"shuangli-liao","given":"Shuangli","family":"Liao","affiliation":"State Key Laboratory of Integrated Services Networks , Xidian University , Xi'an , 710071 , China"},{"name":"Quanxue Gao","slug":"quanxue-gao","given":"Quanxue","family":"Gao","affiliation":"State Key Laboratory of Integrated Services Networks , Xidian University , Xi'an , 710071 , China"},{"name":"Feiping Nie","slug":"feiping-nie","given":"Feiping","family":"Nie","affiliation":"School of Computer Science , OPTIMAL , Northwestern Polytechnical University , Xi'an , 710072 , China"},{"name":"Yang Liu","slug":"yang-liu","given":"Yang","family":"Liu","affiliation":"State Key Laboratory of Integrated Services Networks , Xidian University , Xi'an , 710071 , China"},{"name":"Xiangdong Zhang","slug":"xiangdong-zhang","given":"Xiangdong","family":"Zhang","affiliation":"State Key Laboratory of Integrated Services Networks , Xidian University , Xi'an , 710071 , China"}],"abstract":"Feature selection plays a critical role in data mining,\r\ndriven by increasing feature dimensionality in target problems. In this paper, we propose a new criterion for\r\ndiscriminative feature selection, worst-case discriminative feature selection (WDFS).\r\nUnlike Fisher Score and other methods based on the discriminative criteria\r\nconsidering the overall (or average) separation\r\nof data, WDFS adopts a new perspective called\r\nworst-case view which arguably is more\r\nsuitable for classification applications. Specifically, WDFS directly maximizes the ratio of the\r\nminimum of between-class variance of all class pairs over the maximum of within-class\r\nvariance, and thus it duly considers the separation of all classes. Otherwise, we\r\ntake a greedy strategy by finding one feature at a time, but it is very easy to\r\nimplement. Moreover, we also utilize the correlation between\r\nfeatures to help reduce the redundancy and extend WDFS to uncorrelated WDFS (UWDFS).\r\nTo evaluate the effectiveness of the proposed algorithm, we conduct\r\nclassification experiments on many real data sets. In the experiment, we respectively\r\nuse the original features and the score vectors of features over all class\r\npairs to calculate the correlation coefficients, and analyze the experimental\r\nresults in these two ways. Experimental results demonstrate the effectiveness\r\nof WDFS and UWDFS.","doi":"10.24963/ijcai.2019/412","url":"https://ijcai.org/proceedings/2019/412","pdf":"https://ijcai.org/proceedings/2019/0412.pdf","markdown":"https://ijcai.org/proceedings/2019/412.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/412","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":412,"pages":{"first":2973,"last":2979},"keywords":["Machine Learning: Feature Selection ; Learning Sparse Models","Machine Learning: Classification"],"videos":[],"references":[{"position":1,"raw":"[Battiti, 1994] Roberto Battiti. Using mutual information for selecting features in supervised neural net learning. IEEE Transactions on Neural Networks, 5(4):537-550, 1994.","title":"Using mutual information for selecting features in supervised neural net learning","authors":"Roberto Battiti","year":"1994","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ben-Bassat, 1982] Moshe Ben-Bassat. Pattern recognition and reduction of dimensionality. 1982.","title":"Pattern recognition and reduction of dimensionality","authors":"Ben-Bassat ;, Moshe Ben-Bassat","year":"1982","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bian and Tao, 2011] Wei Bian and Dacheng Tao. Max-min distance analysis by using sequential sdp relaxation for di- mension reduction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(5):1037-1050, 2011.","title":"Max-min distance analysis by using sequential sdp relaxation for dimension reduction","authors":"Tao ; ] Bian, Dacheng Wei Bian, Tao","year":"2011","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cai et al., 2013] Xiao Cai, Feiping Nie, and Heng Huang. Exact top-k feature selection via ℓ 2,0 -norm constraint. In IJCAI, pages 1240-1246, 2013.","title":"Exact top-k feature selection via ℓ 2,0 -norm constraint","authors":"Xiao Cai, Feiping Nie, Heng Huang","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dash and Liu, 1997] Manoranjan Dash and Huan Liu. Fea- ture selection for classification. Intelligent data analysis, 1(3):131-156, 1997.","title":"Feature selection for classification","authors":"Manoranjan Dash, Huan Liu","year":"1997","venue":"Intelligent data analysis","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fisher, 1936] Ronald Aylmer Fisher. The use of multiple measuremens in taxonomic problems. Annals of Human Genetics, 7(2):179-188, 1936.","title":"The use of multiple measuremens in taxonomic problems","authors":"Aylmer Ronald, Fisher","year":"1936","venue":"Annals of Human Genetics","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2005] Xiaofei He, Deng Cai, and Partha Niyogi. Laplacian score for feature selection. In Advances in Neu- ral Information Processing Systems, pages 507-514, 2005.","title":"Laplacian score for feature selection","authors":"Xiaofei He, Deng Cai, Partha Niyogi","year":"2005","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kohavi and John, 1997] Ron Kohavi and George H. John. Wrappers for feature subset selection. Artificial Intelli- gence, 97(1):273-324, 1997.","title":"Wrappers for feature subset selection","authors":"Ron Kohavi, George H John","year":"1997","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Loog et al., 2001] Marco Loog, Robert P. W. Duin, and Reinhold Haeb-Umbach. Multiclass linear dimension re- duction by weighted pairwise fisher criteria. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 23(7):762-766, 2001.","title":"Multiclass linear dimension reduction by weighted pairwise fisher criteria","authors":"Marco Loog, P W Robert, Reinhold Duin, Haeb-Umbach","year":"2001","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Masaeli et al., 2010] Mahdokht Masaeli, Jennifer G. Dy, and Glenn M. Fung. From transformation-based dimen- sionality reduction to feature selection. In ICML, pages 751-758, 2010.","title":"From transformation-based dimensionality reduction to feature selection","authors":"Masaeli","year":"2010","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Nie et al., 2008] Feiping Nie, Shiming Xiang, Yangqing Jia, Changshui Zhang, and Shuicheng Yan. Trace ratio crite- rion for feature selection. In AAAI, volume 2, pages 671- 676, 2008.","title":"Trace ratio criterion for feature selection","authors":"Nie","year":"2008","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Nie et al., 2010] Feiping Nie, Heng Huang, Xiao Cai, and Chris Ding. Efficient and robust feature selection via join- t ℓ 2,1 -norms minimization. In International Conference on Neural Information Processing Systems, pages 1813- 1821, 2010.","title":"Efficient and robust feature selection via joint ℓ 2,1 -norms minimization","authors":"Nie","year":"2010","venue":"International Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Pena and Nilsson, 2010] Jose M. Pena and Roland Nilsson. On the complexity of discrete feature selection for optimal classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(8):1517-1522, 2010.","title":"On the complexity of discrete feature selection for optimal classification","authors":"Nilsson ; ] Pena, M Jose, Roland Pena, Nilsson","year":"2010","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Peng et al., 2005] Hanchuan Peng, Fuhui Long, and Chris H. Q. Ding. Feature selection based on mutual informa- tion criteria of max-dependency, max-relevance, and min- redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8):1226-1238, 2005.","title":"Feature selection based on mutual information criteria of max-dependency, max-relevance, and minredundancy","authors":"Peng","year":"2005","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Raileanu and Stoffel, 2004] Laura Elena Raileanu and Kil- ian Stoffel. Theoretical comparison between the gini index and information gain criteria. Annals of Mathematics and Artificial Intelligence, 41(1):77-93, 2004.","title":"Theoretical comparison between the gini index and information gain criteria","authors":"Stoffel Raileanu","year":"2004","venue":"Annals of Mathematics and Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rao, 1948] C. Radhakrishna Rao. The utilization of mul- tiple measurements in problems of biological classifica- tion. Journal of the royal statistical society series b- methodological, 10(2):159-193, 1948.","title":"The utilization of multiple measurements in problems of biological classification","authors":"C Radhakrishna Rao","year":"1948","venue":"Journal of the royal statistical society series bmethodological","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Robnik et al., 2003] Robnik, Marko Ikonja, and Igor Kononenko. Theoretical and empirical analysis of relieff and rrelieff. Machine Learning, 53(1-2):23-69, 2003.","title":"Robnik, Marko Ikonja, and Igor Kononenko. Theoretical and empirical analysis of relieff and rrelieff","authors":"Robnik","year":"2003","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Romero and Sopena, 2008] Enrique Romero and Josep Maria Sopena. Performing feature selection with multilayer perceptrons. IEEE Transactions on Neural Networks, 19(3):431-441, 2008.","title":"Performing feature selection with multilayer perceptrons","authors":"Enrique Romero, Josep Maria, Sopena","year":"2008","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Su et al., 2018] Bing Su, Xiaoqing Ding, Changsong Liu, and Ying Wu. Heteroscedastic max-min distance analy- sis for dimensionality reduction. IEEE Transactions on Image Processing, 27(8):4052-4065, 2018.","title":"Heteroscedastic max-min distance analysis for dimensionality reduction","authors":"Bing Su, Xiaoqing Ding, Changsong Liu, Ying Wu","year":"2018","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tao et al., 2016] Hong Tao, Chenping Hou, Feiping Nie, Yuanyuan Jiao, and Dongyun Yi. Effective discriminative feature selection with nontrivial solution. IEEE Transac- tions on Neural Networks, 27(4):796-808, 2016.","title":"Effective discriminative feature selection with nontrivial solution","authors":"Tao","year":"2016","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2008] Lipo Wang, Nina Zhou, and Feng Chu. A general wrapper approach to selection of class- dependent features. IEEE Transactions on Neural Net- works, 19(7):1267-1278, 2008.","title":"A general wrapper approach to selection of classdependent features","authors":"Lipo Wang, Nina Zhou, Feng Chu","year":"2008","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yu and Liu, 2004] Lei Yu and Huan Liu. Efficient feature s- election via analysis of relevance and redundancy. Journal of Machine Learning Research, 5:1205-1224, 2004.","title":"Efficient feature selection via analysis of relevance and redundancy","authors":"Yu, Liu ; ] Lei Yu, Huan Liu","year":"2004","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yu and Yeung, 2010] Zhang Yu and Dit Yan Yeung. Worst- case linear discriminant analysis. In International Confer- ence on Neural Information Processing Systems, 2010.","title":"Worstcase linear discriminant analysis","authors":"Yu, Yeung ; ] Zhang Yu, Dit Yan, Yeung","year":"2010","venue":"International Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang et al., 2018] Rui Zhang, Feiping Nie, and Xuelong Li. Self-weighted supervised discriminative feature selec- tion. IEEE Transactions on Neural Networks, 29:3913- 3918, 2018.","title":"Self-weighted supervised discriminative feature selection","authors":"Rui Zhang, Feiping Nie, Xuelong Li","year":"2018","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}