{"type":"ScholarlyArticle","title":"Ensemble-based Ultrahigh-dimensional Variable Screening","authors":[{"name":"Wei Tu","slug":"wei-tu","given":"Wei","family":"Tu","affiliation":"Department of Mathematical and Statistical Sciences , University of Alberta"},{"name":"Dong Yang","slug":"dong-yang","given":"Dong","family":"Yang","affiliation":"Department of Mathematical and Statistical Sciences , University of Alberta"},{"name":"Linglong Kong","slug":"linglong-kong","given":"Linglong","family":"Kong","affiliation":"Department of Mathematical and Statistical Sciences , University of Alberta"},{"name":"Menglu Che","slug":"menglu-che","given":"Menglu","family":"Che","affiliation":"Department of Statistics and Actuarial Science , University of Waterloo"},{"name":"Qian Shi","slug":"qian-shi","given":"Qian","family":"Shi","affiliation":"Department of Mathematical and Statistical Sciences , University of Alberta"},{"name":"Guodong Li","slug":"guodong-li","given":"Guodong","family":"Li","affiliation":"Department of Statistics and Actuarial Science , University of Hong Kong"},{"name":"Guangjian Tian","slug":"guangjian-tian","given":"Guangjian","family":"Tian","affiliation":"Huawei Noah's Ark Lab , Hong Kong , China"}],"abstract":"Since the sure independence screening (SIS) method by Fan and Lv, many different variable screening methods have been proposed based on different measures under different models. However, most of these methods are designed for specific models. In practice, we often have very little information about the data generating process and different methods can result in very different sets of features. The heterogeneity presented here motivates us to combine various screening methods simultaneously. In this paper, we introduce a general ensemble-based framework to efficiently combine results from multiple variable screening methods. The consistency and sure screening property of proposed framework has been established. Extensive simulation studies confirm our intuition that the proposed ensemble-based method is more robust against model specification than using single variable screening method. The proposed ensemble-based method is used to predict attention deficit hyperactivity disorder (ADHD) status using brain function connectivity (FC).","doi":"10.24963/ijcai.2019/501","url":"https://ijcai.org/proceedings/2019/501","pdf":"https://ijcai.org/proceedings/2019/0501.pdf","markdown":"https://ijcai.org/proceedings/2019/501.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/501","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":501,"pages":{"first":3613,"last":3619},"keywords":["Machine Learning: Feature Selection ; Learning Sparse Models","Machine Learning: Ensemble Methods"],"videos":[],"references":[{"position":1,"raw":"[Bach, 2008] Francis R Bach. Bolasso: model consistent lasso estimation through the bootstrap. In Proceedings of the 25th international conference on Machine learning, pages 33-40. ACM, 2008.","title":"Bolasso: model consistent lasso estimation through the bootstrap","authors":"Francis R Bach","year":"2008","venue":"Proceedings of the 25th international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bolón-Canedo and Alonso-Betanzos, 2019] Verónica Bolón-Canedo and Amparo Alonso-Betanzos. Ensem- bles for feature selection: A review and future trends. Information Fusion, 52:1-12, 2019.","title":"Verónica Bolón-Canedo and Amparo Alonso-Betanzos. Ensembles for feature selection: A review and future trends","authors":"Bolón-Canedo, Alonso-Betanzos","year":"2019","venue":"Information Fusion","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Fan and Li, 2001] Jianqing Fan and Runze Li. Variable se- lection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Associa- tion, 96(456):1348-1360, 2001.","title":"Jianqing Fan and Runze Li. Variable selection via nonconcave penalized likelihood and its oracle properties","authors":"Fan, Li","year":"2001","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fan and Lv, 2008] Jianqing Fan and Jinchi Lv. Sure in- dependence screening for ultrahigh dimensional feature space. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 70(5):849-911, 2008.","title":"Sure independence screening for ultrahigh dimensional feature space","authors":"Jianqing Fan, Jinchi Lv","year":"2008","venue":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fan et al., 2009] Jianqing Fan, Richard Samworth, and Yichao Wu. Ultrahigh dimensional feature selection: be- yond the linear model. Journal of Machine Learning Re- search, 10:2013-2038, December 2009.","title":"Ultrahigh dimensional feature selection: beyond the linear model","authors":"Jianqing Fan, Richard Samworth, Yichao Wu","year":"2009","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fan et al., 2010] Jianqing Fan, Rui Song, et al. Sure in- dependence screening in generalized linear models with np-dimensionality. The Annals of Statistics, 38(6):3567- 3604, 2010.","title":"Sure independence screening in generalized linear models with np-dimensionality","authors":"Jianqing Fan, Rui Song","year":"2010","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fan et al., 2011] Jianqing Fan, Yang Feng, and Rui Song. Nonparametric independence screening in sparse ultra- high-dimensional additive models. Journal of the Amer- ican Statistical Association, 106(494):544-557, 2011.","title":"Nonparametric independence screening in sparse ultrahigh-dimensional additive models","authors":"Jianqing Fan, Yang Feng, Rui Song","year":"2011","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Greicius et al., 2007] Michael D Greicius, Benjamin H Flo- res, Vinod Menon, Gary H Glover, Hugh B Solvason, Heather Kenna, Allan L Reiss, and Alan F Schatzberg. Resting-state functional connectivity in major depres- sion: abnormally increased contributions from subgen- ual cingulate cortex and thalamus. Biological Psychiatry, 62(5):429-437, 2007.","title":"Resting-state functional connectivity in major depression: abnormally increased contributions from subgenual cingulate cortex and thalamus","authors":"Benjamin H Michael D Greicius, Vinod Flores, Gary H Menon, Hugh B Glover, Heather Solvason, Allan L Kenna, Alan F Reiss, Schatzberg","year":"2007","venue":"Biological Psychiatry","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2013] Xuming He, Lan Wang, and Hyoky- oung Grace Hong. Quantile-adaptive model-free variable screening for high-dimensional heterogeneous data. The Annals of Statistics, 41(1):342-369, 2013.","title":"Xuming He, Lan Wang, and Hyokyoung Grace Hong. Quantile-adaptive model-free variable screening for high-dimensional heterogeneous data","authors":"He","year":"2013","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hearst et al., 1998] Marti Hearst, Susan T. Dumais, E Os- man, John Platt, and Bernhard Scholkopf. Support vec- tor machines. IEEE Intelligent Systems, 13(4):18-28, July 1998.","title":"Support vector machines","authors":"Marti Hearst, Susan T Dumais, John Osman, Bernhard Platt, Scholkopf","year":"1998","venue":"IEEE Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Huettel et al., 2004] Scott A Huettel, Allen W Song, and Gregory McCarthy. Functional magnetic resonance imag- ing, volume 1. Sinauer Associates Sunderland, MA, 2004.","title":"Functional magnetic resonance imaging","authors":"A Scott, Allen W Huettel, Gregory Song, Mccarthy","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2012a] Gaorong Li, Heng Peng, Jun Zhang, and Lixing Zhu. Robust rank correlation based screening. The Annals of Statistics, pages 1846-1877, 2012.","title":"Robust rank correlation based screening","authors":"Li","year":"2012","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2012b] Runze Li, Wei Zhong, and Liping Zhu. Feature screening via distance correlation learn- ing. Journal of the American Statistical Association, 107(499):1129-1139, 2012.","title":"Feature screening via distance correlation learning","authors":"Li","year":"2012","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2015] JingYuan Liu, Wei Zhong, and RunZe Li. A selective overview of feature screening for ultrahigh- dimensional data. Science China Mathematics, 58(10):1- 22, 2015.","title":"A selective overview of feature screening for ultrahighdimensional data","authors":"Liu","year":"2015","venue":"Science China Mathematics","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Milham et al., 2012] Michael P Milham, Damien Fair, Maarten Mennes, and Stewart Mostofsky. The ADHD-200 consortium: a model to advance the translational potential of neuroimaging in clinical neuroscience. Frontiers in Sys- tems Neuroscience, 6:62, 2012.","title":"The ADHD-200 consortium: a model to advance the translational potential of neuroimaging in clinical neuroscience","authors":"Damien Michael P Milham, Maarten Fair, Stewart Mennes, Mostofsky","year":"2012","venue":"Frontiers in Systems Neuroscience","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nguyen et al., 2018] Tien Thanh Nguyen, Xuan Cuong Pham, Alan Wee-Chung Liew, and Witold Pedrycz. Ag- gregation of classifiers: a justifiable information granular- ity approach. IEEE Transactions on Cybernetics, pages 1-10, 2018.","title":"Aggregation of classifiers: a justifiable information granularity approach","authors":"Thanh Tien, Xuan Cuong Nguyen, Alan Wee-Chung Pham, Witold Liew, Pedrycz","year":"2018","venue":"IEEE Transactions on Cybernetics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Tian et al., 2006] Lixia Tian, Tianzi Jiang, Yufeng Wang, Yufeng Zang, Yong He, Meng Liang, Manqiu Sui, Qingjiu Cao, Siyuan Hu, Miao Peng, et al. Altered resting-state functional connectivity patterns of anterior cingulate cor- tex in adolescents with attention deficit hyperactivity dis- order. Neuroscience letters, 400(1-2):39-43, 2006.","title":"Altered resting-state functional connectivity patterns of anterior cingulate cortex in adolescents with attention deficit hyperactivity disorder","authors":"Lixia Tian, Tianzi Jiang, Yufeng Wang, Yufeng Zang, Yong He, Meng Liang, Manqiu Sui, Qingjiu Cao, Siyuan Hu, Miao Peng","year":"2006","venue":"Neuroscience letters","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Tibshirani, 1996] Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), (1):267-288, 1996.","title":"Regression shrinkage and selection via the lasso","authors":"Robert Tibshirani","year":"1996","venue":"Journal of the Royal Statistical Society. Series B (Methodological)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Tzourio-Mazoyer et al., 2002] Nathalie Tzourio-Mazoyer, Brigitte Landeau, Dimitri Papathanassiou, Fabrice Criv- ello, Olivier Etard, Nicolas Delcroix, Bernard Mazoyer, and Marc Joliot. Automated anatomical labeling of acti- vations in SPM using a macroscopic anatomical parcella- tion of the MNI MRI single-subject brain. Neuroimage, 15(1):273-289, 2002.","title":"Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain","authors":"Nathalie Tzourio-Mazoyer, Brigitte Landeau, Dimitri Papathanassiou, Fabrice Crivello, Olivier Etard, Nicolas Delcroix, Bernard Mazoyer, Marc Joliot","year":"2002","venue":"Neuroimage","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Uddin et al., 2008] Lucina Q Uddin, AM Clare Kelly, Bharat B Biswal, Daniel S Margulies, Zarrar Shehzad, David Shaw, Manely Ghaffari, John Rotrosen, Lenard A Adler, F Xavier Castellanos, et al. Network homogene- ity reveals decreased integrity of default-mode network in adhd. Journal of neuroscience methods, 169(1):249-254, 2008.","title":"Network homogeneity reveals decreased integrity of default-mode network in adhd","authors":"Clare Lucina Q Uddin, Bharat B Kelly, Biswal, Zarrar Daniel S Margulies, David Shehzad, Manely Shaw, John Ghaffari, Lenard A Rotrosen, F Adler, Xavier Castellanos","year":"2008","venue":"Journal of neuroscience methods","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang, 2010] Cun-Hui Zhang. Nearly unbiased variable se- lection under minimax concave penalty. The Annals of Statistics, 38(2):894-942, 2010.","title":"Nearly unbiased variable selection under minimax concave penalty","authors":"Cun-Hui Zhang","year":"2010","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhou, 2012] Zhi-Hua Zhou. Ensemble methods: founda- tions and algorithms. Chapman and Hall/CRC, 2012.","title":"Ensemble methods: foundations and algorithms","authors":"Zhi-Hua Zhou","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhu et al., 2011] Li-Ping Zhu, Lexin Li, Runze Li, and Li- Xing Zhu. Model-free feature screening for ultrahigh- dimensional data. Journal of the American Statistical As- sociation, 106(496):1464-1475, 2011.","title":"Model-free feature screening for ultrahighdimensional data","authors":"Li-Ping Zhu, Lexin Li, Runze Li, Li-Xing Zhu","year":"2011","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zou and Hastie, 2005] Hui Zou and Trevor Hastie. Regular- ization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Method- ology), 67(2):301-320, 2005.","title":"Regularization and variable selection via the elastic net","authors":"Hastie Zou, Hui Zou, Trevor Hastie","year":"2005","venue":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zou, 2006] Hui Zou. The adaptive lasso and its oracle prop- erties. Journal of the American statistical association, 101(476):1418-1429, 2006.","title":"The adaptive lasso and its oracle properties","authors":"Hui Zou","year":"2006","venue":"Journal of the American statistical association","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}