{"type":"ScholarlyArticle","title":"Reconstruction-based Unsupervised Feature Selection: An Embedded Approach","authors":[{"name":"Jundong Li","slug":"jundong-li","given":"Jundong","family":"Li","affiliation":"† Computer Science and Engineering , Arizona State University , USA"},{"name":"Jiliang Tang","slug":"jiliang-tang","given":"Jiliang","family":"Tang","affiliation":"‡ Computer Science and Engineering , Michigan State University , USA"},{"name":"Huan Liu","slug":"huan-liu","given":"Huan","family":"Liu","affiliation":"† Computer Science and Engineering , Arizona State University , USA"}],"abstract":"Feature selection has been proven to be effective and efficient in preparing high-dimensional data for data mining and machine learning problems. Since real-world data is usually unlabeled, unsupervised feature selection has received increasing attention in recent years. Without label information, unsupervised feature selection needs alternative criteria to define feature relevance. Recently, data reconstruction error emerged as a new criterion for unsupervised feature selection, which defines feature relevance as the capability of features to approximate original data via a reconstruction function. Most existing algorithms in this family assume predefined, linear reconstruction functions. However, the reconstruction function should be data dependent and may not always be linear especially when the original data is high-dimensional. In this paper, we investigate how to learn the reconstruction function from the data automatically for unsupervised feature selection, and propose a novel reconstruction-based unsupervised feature selection framework REFS, which embeds the reconstruction function learning process into feature selection. Experiments on various types of real-world datasets demonstrate the effectiveness of the proposed framework REFS.","doi":"10.24963/ijcai.2017/300","url":"https://ijcai.org/proceedings/2017/300","pdf":"https://ijcai.org/proceedings/2017/0300.pdf","markdown":"https://ijcai.org/proceedings/2017/300.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/300","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra, IIIA-CSIC","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":300,"pages":{"first":2159,"last":2165},"keywords":["Machine Learning: Unsupervised Learning","Machine Learning: Feature Selection/Construction"],"videos":[],"references":[{"position":1,"raw":"[Bhattacharjee et al., 2001] Arindam Bhattacharjee et al. Classification of human lung carcinomas by mrna expres- sion profiling reveals distinct adenocarcinoma subclasses. PNAS, 2001.","title":"Classification of human lung carcinomas by mrna expression profiling reveals distinct adenocarcinoma subclasses","authors":"Bhattacharjee","year":"2001","venue":"PNAS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Boutsidis et al., 2008] Christos Boutsidis, Michael W Ma- honey, and Petros Drineas. Unsupervised feature selection for principal components analysis. In KDD, 2008.","title":"Unsupervised feature selection for principal components analysis","authors":"Christos Boutsidis, Michael W Mahoney, Petros Drineas","year":"2008","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai et al., 2010] Deng Cai, Chiyuan Zhang, and Xiaofei He. Unsupervised feature selection for multi-cluster data. In KDD, 2010.","title":"Unsupervised feature selection for multi-cluster data","authors":"Deng Cai, Chiyuan Zhang, Xiaofei He","year":"2010","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2010] Chun Chen, Zhengguang Chen, Jiajun Bu, Can Wang, Lijun Zhang, and Cheng Zhang. G-optimal design with laplacian regularization. In AAAI, 2010.","title":"G-optimal design with laplacian regularization","authors":"Chun Chen, Zhengguang Chen, Jiajun Bu, Can Wang, Lijun Zhang, Cheng Zhang","year":"2010","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cover and Thomas, 2012] Thomas M Cover and Joy A Thomas. Elements of information theory. 2012.","title":"Elements of information theory","authors":"M Thomas, Joy A Cover, Thomas","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ding and Peng, 2005] Chris Ding and Hanchuan Peng. Minimum redundancy feature selection from microarray gene expression data. JBCB, 2005.","title":"Minimum redundancy feature selection from microarray gene expression data","authors":"Chris Ding, Hanchuan Peng","year":"2005","venue":"JBCB","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Du and Shen, 2015] Liang Du and Yi-Dong Shen. Unsuper- vised feature selection with adaptive structure learning. In KDD, 2015.","title":"Unsupervised feature selection with adaptive structure learning","authors":"Liang Du, Yi-Dong Shen","year":"2015","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Duda et al., 2012] Richard O Duda, Peter E Hart, and David G Stork. Pattern classification. 2012.","title":"Pattern classification","authors":"Peter E Richard O Duda, David G Hart, Stork","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Farahat et al., 2011] Ahmed K Farahat, Ali Ghodsi, and Mohamed S Kamel. An efficient greedy method for un- supervised feature selection. In ICDM, 2011.","title":"An efficient greedy method for unsupervised feature selection","authors":"Ali Ahmed K Farahat, Mohamed S Ghodsi, Kamel","year":"2011","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Golub and Van Loan, 2012] Gene H Golub and Charles F Van Loan. Matrix computations. 2012.","title":"Matrix computations","authors":"H Gene, Charles F Golub, Van Loan","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Guyon and Elisseeff, 2003] Isabelle Guyon and André Elis- seeff. An introduction to variable and feature selection. JMLR, 2003.","title":"Isabelle Guyon and André Elisseeff. An introduction to variable and feature selection","authors":"Elisseeff Guyon","year":"2003","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[He et al., 2005] Xiaofei He, Deng Cai, and Partha Niyogi. Laplacian score for feature selection. In NIPS, 2005.","title":"Laplacian score for feature selection","authors":"Xiaofei He, Deng Cai, Partha Niyogi","year":"2005","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[He, 2010] Xiaofei He. Laplacian regularized d-optimal de- sign for active learning and its application to image re- trieval. TIP, 2010.","title":"Laplacian regularized d-optimal design for active learning and its application to image retrieval","authors":"Xiaofei He","year":"2010","venue":"TIP","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hull, 1994] Jonathan J. Hull. A database for handwritten text recognition research. TPAMI, 1994.","title":"A database for handwritten text recognition research","authors":"Jonathan J Hull","year":"1994","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jain and Zongker, 1997] Anil Jain and Douglas Zongker. Feature selection: Evaluation, application, and small sam- ple performance. TPAMI, 1997.","title":"Feature selection: Evaluation, application, and small sample performance","authors":"Anil Jain, Douglas Zongker","year":"1997","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Jian et al., 2016] Ling Jian, Jundong Li, Kai Shu, and Huan Liu. Multi-label informed feature selection. In IJCAI, 2016.","title":"Multi-label informed feature selection","authors":"Ling Jian, Jundong Li, Kai Shu, Huan Liu","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/233"},{"position":17,"raw":"[Li and Liu, 2017] Jundong Li and Huan Liu. Challenges of feature selection for big data analytics. IEEE Intelligent Systems, 2017.","title":"Challenges of feature selection for big data analytics","authors":"Li, Liu, Jundong Li, Huan Liu","year":"2017","venue":"IEEE Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2012] Zechao Li, Yi Yang, Jing Liu, Xiaofang Zhou, and Hanqing Lu. Unsupervised feature selection using nonnegative spectral analysis. In AAAI, 2012.","title":"Unsupervised feature selection using nonnegative spectral analysis","authors":"Zechao Li, Yi Yang, Jing Liu, Xiaofang Zhou, Hanqing Lu","year":"2012","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2015] Jundong Li, Xia Hu, Jiliang Tang, and Huan Liu. Unsupervised streaming feature selection in social media. In CIKM, 2015.","title":"Unsupervised streaming feature selection in social media","authors":"Li","year":"2015","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Li et al., 2016a] Jundong Li, Kewei Cheng, Suhang Wang, Fred Morstatter, Robert P Trevino, Jiliang Tang, and Huan Liu. Feature selection: A data perspective. arXiv preprint arXiv:1601.07996, 2016.","title":"Feature selection: A data perspective","authors":"Li","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Li et al., 2016b] Jundong Li, Xia Hu, Ling Jian, and Huan Liu. Toward time-evolving feature selection on dynamic networks. In ICDM, 2016.","title":"Toward time-evolving feature selection on dynamic networks","authors":"Li","year":"2016","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Li et al., 2017] Jundong Li, Liang Wu, Osmar R Zaïane, and Huan Liu. Toward personalized relational learning. In SDM, 2017.","title":"Toward personalized relational learning","authors":"Jundong Li, Liang Wu, Huan Osmar R Zaïane, Liu","year":"2017","venue":"SDM","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Liu and Motoda, 2007] Huan Liu and Hiroshi Motoda. Computational methods of feature selection. 2007.","title":"Computational methods of feature selection","authors":"Huan Liu, Hiroshi Motoda","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Masaeli et al., 2010] Mahdokht Masaeli, Yan Yan, Ying Cui, Glenn Fung, and Jennifer G Dy. Convex principal feature selection. In ICDM, 2010.","title":"Convex principal feature selection","authors":"Masaeli","year":"2010","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Mitra et al., 2002] Pabitra Mitra, CA Murthy, and Sankar K. Pal. Unsupervised feature selection using feature similar- ity. TPAMI, 2002.","title":"Unsupervised feature selection using feature similarity","authors":"Pabitra Mitra, Sankar K Murthy, Pal","year":"2002","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Nene et al., 1996] Sameer A Nene, Shree K Nayar, Hiroshi Murase, et al. Columbia object image library (coil-20). Technical report, Technical Report CUCS-005-96, 1996.","title":"Columbia object image library (coil-20)","authors":"Sameer A Nene, Hiroshi Shree K Nayar, Murase","year":"1996","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Nie et al., 2008] Feiping Nie, Shiming Xiang, Yangqing Jia, Changshui Zhang, and Shuicheng Yan. Trace ratio crite- rion for feature selection. In AAAI, 2008.","title":"Trace ratio criterion for feature selection","authors":"Nie","year":"2008","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Nie et al., 2010] Feiping Nie, Heng Huang, Xiao Cai, and Chris H Ding. Efficient and robust feature selection via joint 2, 1-norms minimization. In NIPS, 2010.","title":"Efficient and robust feature selection via joint 2, 1-norms minimization","authors":"Nie","year":"2010","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Nutt et al., 2003] Catherine L Nutt et al. Gene expression- based classification of malignant gliomas correlates better with survival than histological classification. 2003.","title":"Gene expressionbased classification of malignant gliomas correlates better with survival than histological classification","authors":"Catherine L Nutt","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Qian and Zhai, 2013] Mingjie Qian and Chengxiang Zhai. Robust unsupervised feature selection. In IJCAI, 2013.","title":"Robust unsupervised feature selection","authors":"Mingjie Qian, Chengxiang Zhai","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2013/241"},{"position":31,"raw":"[Tang et al., 2014] Jiliang Tang, Salem Alelyani, and Huan Liu. Feature selection for classification: A review. Data Classification: Algorithms and Applications, 2014.","title":"Feature selection for classification: A review","authors":"Tang","year":"2014","venue":"Data Classification: Algorithms and Applications","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Tibshirani, 1996] Robert Tibshirani. Regression shrinkage and selection via the lasso. JRSS(B), 1996.","title":"Regression shrinkage and selection via the lasso","authors":"Robert Tibshirani","year":"1996","venue":"JRSS","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wei and Philip, 2016] Xiaokai Wei and S Yu Philip. Unsu- pervised feature selection by preserving stochastic neigh- bors. In AISTATS, 2016.","title":"Unsupervised feature selection by preserving stochastic neighbors","authors":"Philip ; Wei, Xiaokai Wei, S Yu, Philip","year":"2016","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yang et al., 2011] Yi Yang, Heng Tao Shen, Zhigang Ma, Zi Huang, and Xiaofang Zhou. l2, 1-norm regularized dis- criminative feature selection for unsupervised learning. In IJCAI, 2011.","title":"l2, 1-norm regularized discriminative feature selection for unsupervised learning","authors":"Yi Yang, Heng Tao Shen, Zhigang Ma, Zi Huang, Xiaofang Zhou","year":"2011","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2011/267"},{"position":35,"raw":"[Zhang et al., 2011] Lijun Zhang, Chun Chen, Jiajun Bu, Deng Cai, Xiaofei He, and Thomas S Huang. Active learn- ing based on locally linear reconstruction. TPAMI, 2011.","title":"Active learning based on locally linear reconstruction","authors":"Lijun Zhang, Chun Chen, Jiajun Bu, Deng Cai, Xiaofei He, Thomas S Huang","year":"2011","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhao and Liu, 2007] Zheng Zhao and Huan Liu. Spectral feature selection for supervised and unsupervised learning. In ICML, 2007.","title":"Spectral feature selection for supervised and unsupervised learning","authors":"Zheng Zhao, Huan Liu","year":"2007","venue":"ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}