{"type":"ScholarlyArticle","title":"SVD-Based Screening for the Graphical Lasso","authors":[{"name":"Yasuhiro Fujiwara","slug":"yasuhiro-fujiwara","given":"Yasuhiro","family":"Fujiwara","affiliation":"† NTT Communication Science Laboratories , NTT Software Innovation Center , Osaka University"},{"name":"Naoki Marumo","slug":"naoki-marumo","given":"Naoki","family":"Marumo","affiliation":"‡ NTT Communication Science Laboratories"},{"name":"Mathieu Blondel","slug":"mathieu-blondel","given":"Mathieu","family":"Blondel","affiliation":"‡ NTT Communication Science Laboratories"},{"name":"Koh Takeuchi","slug":"koh-takeuchi","given":"Koh","family":"Takeuchi","affiliation":"‡ NTT Communication Science Laboratories"},{"name":"Hideaki Kim","slug":"hideaki-kim","given":"Hideaki","family":"Kim","affiliation":"‡ NTT Communication Science Laboratories"},{"name":"Tomoharu Iwata","slug":"tomoharu-iwata","given":"Tomoharu","family":"Iwata","affiliation":"‡ NTT Communication Science Laboratories"},{"name":"Naonori Ueda","slug":"naonori-ueda","given":"Naonori","family":"Ueda","affiliation":"‡ NTT Communication Science Laboratories"}],"abstract":"The graphical lasso is the most popular approach to estimating the inverse covariance matrix of high-dimension data. It iteratively estimates each row and column of the matrix in a round-robin style until convergence. However, the graphical lasso is infeasible due to its high computation cost for large size of datasets. This paper proposes Sting, a fast approach to the graphical lasso. In order to reduce the computation cost, it efficiently identifies blocks in the estimated matrix that have nonzero elements before entering the iterations by exploiting the singular value decomposition of data matrix. In addition, it selectively updates elements of the estimated matrix expected to have nonzero values. Theoretically, it guarantees to converge to the same result as the original algorithm of the graphical lasso. Experiments show that our approach is faster than existing approaches.","doi":"10.24963/ijcai.2017/233","url":"https://ijcai.org/proceedings/2017/233","pdf":"https://ijcai.org/proceedings/2017/0233.pdf","markdown":"https://ijcai.org/proceedings/2017/233.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/233","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":233,"pages":{"first":1682,"last":1688},"keywords":["Machine Learning: Learning Graphical Models","Multidisciplinary Topics and Applications: Multidisciplinary Topics and Applications","Machine Learning: Structured Learning","Machine Learning: Data Mining"],"videos":[],"references":[{"position":1,"raw":"[Blitzstein and Hwang, 2014] Joseph K. Blitzstein and Jes- sica Hwang. Introduction to Probability. Chapman and Hall/CRC, 2014.","title":"Introduction to Probability","authors":"Joseph K Blitzstein, Jessica Hwang","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2010] Xi Chen, Yan Liu, Han Liu, and Jaime G. Carbonell. Learning Spatial-temporal Varying Graphs with Applications to Climate Data Analysis. In AAAI, 2010.","title":"Learning Spatial-temporal Varying Graphs with Applications to Climate Data Analysis","authors":"Xi Chen, Yan Liu, Han Liu, Jaime G Carbonell","year":"2010","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Friedman et al., 2008] Jerome Friedman, Trevor Hastie, and Robert Tibshirani. Sparse Inverse Covariance Estima- tion with the Graphical Lasso. Biostatistics, 9(3):432-441, July 2008.","title":"Sparse Inverse Covariance Estimation with the Graphical Lasso","authors":"Jerome Friedman, Trevor Hastie, Robert Tibshirani","year":"2008","venue":"Biostatistics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Friedman et al., 2015] Jerome Friedman, Trevor Hastie, and Rob Tibshirani. Package 'glasso'. ftp://cran.r- project.org/pub/R/web/packages/glasso/glasso.pdf, 2015.","title":"Package 'glasso","authors":"Jerome Friedman, Trevor Hastie, Rob Tibshirani","year":"2015","venue":null,"doi":null,"url":"ftp://cran.r-project.org/pub/R/web/packages/glasso/glasso.pdf","internal_url":null},{"position":5,"raw":"[Fujiwara et al., 2014] Yasuhiro Fujiwara, Go Irie, Shari Kuroyama, and Makoto Onizuka. Scaling Manifold Rank- ing bBsed Image Retrieval. PVLDB, 8(4):341-352, 2014.","title":"Scaling Manifold Ranking bBsed Image Retrieval","authors":"Yasuhiro Fujiwara, Go Irie, Shari Kuroyama, Makoto Onizuka","year":"2014","venue":"PVLDB","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fujiwara et al., 2016a] Yasuhiro Fujiwara, Yasutoshi Ida, Junya Arai, Mai Nishimura, and Sotetsu Iwamura. Fast Algorithm for the Lasso Based L1-graph Construction. PVLDB, 10(3):229-240, 2016.","title":"Fast Algorithm for the Lasso Based L1-graph Construction","authors":"Fujiwara","year":"2016","venue":"PVLDB","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fujiwara et al., 2016b] Yasuhiro Fujiwara, Yasutoshi Ida, Hiroaki Shiokawa, and Sotetsu Iwamura. Fast Lasso Al- gorithm via Selective Coordinate Descent. In Proceed- ings of the Thirtieth AAAI Conference on Artificial Intel- ligence, February 12-17, 2016, Phoenix, Arizona, USA., pages 1561-1567, 2016.","title":"Fast Lasso Algorithm via Selective Coordinate Descent","authors":"Fujiwara","year":"2016","venue":"Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fujiwara et al., 2017] Yasuhiro Fujiwara, Naoki Marumo, Mathieu Blondel, Koh Takeuchi, Hideaki Kim, Tomoharu Iwata, and Naonori Ueda. Scaling Locally Linear Embed- ding. In ACM SIGMOD, pages 1479-1492, 2017.","title":"Scaling Locally Linear Embedding","authors":"Yasuhiro Fujiwara, Naoki Marumo, Mathieu Blondel, Koh Takeuchi, Hideaki Kim, Tomoharu Iwata, Naonori Ueda","year":"2017","venue":"ACM SIGMOD","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Grechkin et al., 2015] Maxim Grechkin, Maryam Fazel, Daniela M. Witten, and Su-In Lee. Pathway Graphical Lasso. In AAAI, pages 2617-2623, 2015.","title":"Pathway Graphical Lasso","authors":"Grechkin","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Halko et al., 2011] Nathan Halko, Per-Gunnar Martinsson, and Joel A. Tropp. Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions. SIAM Review, 53(2):217-288, 2011.","title":"Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions","authors":"Halko","year":"2011","venue":"SIAM Review","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hsieh et al., 2013] Cho-Jui Hsieh, Mátyás A. Sustik, Inder- jit S. Dhillon, Pradeep Ravikumar, and Russell A. Pol- drack. BIG & QUIC: Sparse Inverse Covariance Estima- tion for a Million Variables. In NIPS, pages 3165-3173, 2013.","title":"BIG & QUIC: Sparse Inverse Covariance Estimation for a Million Variables","authors":"Cho-Jui Hsieh, Mátyás A Sustik, Inderjit S Dhillon, Pradeep Ravikumar, Russell A Poldrack","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hsieh et al., 2015] Cho-Jui Hsieh, Matyas A. Sustik, Inder- jit S. Dhillon, and Pradeep Ravikumar. Package 'QUIC'. https://cran.r-project.org/web/packages/QUIC/index.html, 2015.","title":"Package 'QUIC","authors":"Cho-Jui Hsieh, Matyas A Sustik, Inderjit S Dhillon, Pradeep Ravikumar","year":"2015","venue":null,"doi":null,"url":"https://cran.r-project.org/web/packages/QUIC/index.html","internal_url":null},{"position":13,"raw":"[Langfelder and Horvath, 2008] Peter Langfelder and Steve Horvath. WGCNA: an R Package for Weighted Correla- tion Network Analysis. BMC Bioinformatics, 9, 2008.","title":"WGCNA: an R Package for Weighted Correlation Network Analysis","authors":"Horvath Langfelder, Peter Langfelder, Steve Horvath","year":"2008","venue":"BMC Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2016] Mei Li, Ying Lin, Shuai Huang, and Craig Crossland. The Use of Sparse Inverse Covariance Estima- tion for Relationship Detection and Hypothesis Generation in Strategic Management. Strategic Management Journal, 37(1):86-97, 2016.","title":"The Use of Sparse Inverse Covariance Estimation for Relationship Detection and Hypothesis Generation in Strategic Management","authors":"Mei Li, Ying Lin, Shuai Huang, Craig Crossland","year":"2016","venue":"Strategic Management Journal","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2010] Han Liu, Kathryn Roeder, and Larry A. Wasserman. Stability Approach to Regularization Selec- tion (StARS) for High Dimensional Graphical Models. In NIPS, pages 1432-1440, 2010.","title":"Stability Approach to Regularization Selection (StARS) for High Dimensional Graphical Models","authors":"Han Liu, Kathryn Roeder, Larry A Wasserman","year":"2010","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2012] Han Liu, Fang Han, Ming Yuan, John D. Lafferty, and Larry A. Wasserman. High Dimensional Semiparametric Gaussian Copula Graphical Models. In ICML, 2012.","title":"High Dimensional Semiparametric Gaussian Copula Graphical Models","authors":"Han Liu, Fang Han, Ming Yuan, John D Lafferty, Larry A Wasserman","year":"2012","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mishima and Fujiwara, 2015] Takeshi Mishima and Ya- suhiro Fujiwara. Madeus: Database Live Migration Mid- dleware under Heavy Workloads for Cloud Environment. In ACM SIGMOD, pages 315-329, 2015.","title":"Madeus: Database Live Migration Middleware under Heavy Workloads for Cloud Environment","authors":"Fujiwara Mishima, Takeshi Mishima, Yasuhiro Fujiwara","year":"2015","venue":"ACM SIGMOD","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nakatsuji et al., 2014] Makoto Nakatsuji, Yasuhiro Fuji- wara, Hiroyuki Toda, Hiroshi Sawada, Jinguang Zheng, and James Alexander Hendler. Semantic Data Represen- tation for Improving Rensor Factorization. In AAAI, pages 2004-2012, 2014.","title":"Semantic Data Representation for Improving Rensor Factorization","authors":"Makoto Nakatsuji, Yasuhiro Fujiwara, Hiroyuki Toda, Hiroshi Sawada, Jinguang Zheng, James Alexander Hendler","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Neuberg and Glasserman, 2016] Richard Neuberg and Paul Glasserman. The Correlation Structure of Credit Default Swaps. Columbia Business School Research Paper, 2016.","title":"The Correlation Structure of Credit Default Swaps","authors":"Richard Neuberg, Paul Glasserman","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Press et al., 2007] William H. Press, Saul A. Teukolsky, William T Vetterling, and Brian P. Flannery. Numerical Recipes 3rd Edition. Cambridge University Press, 2007.","title":"Numerical Recipes 3rd Edition","authors":"H William, Saul A Press, William T Teukolsky, Brian P Vetterling, Flannery","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rosa et al., 2015] Maria J. Rosa, Liana Portugal, Tim Hahn, Andreas J. Fallgatter, Marta I. Garrido, John Shawe- Taylor, and Janaina Mourão Miranda. Sparse Network- based Models for Patient Classification Using fMRI. Neu- roImage, 105:493-506, 2015.","title":"Sparse Networkbased Models for Patient Classification Using fMRI","authors":"Maria J Rosa, Liana Portugal, Tim Hahn, Andreas J Fallgatter, Marta I Garrido, John Shawe-Taylor, Janaina Mourão, Miranda","year":"2015","venue":"Neu-roImage","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shiokawa et al., 2015] Hiroaki Shiokawa, Yasuhiro Fuji- wara, and Makoto Onizuka. SCAN++: Efficient Algo- rithm for Finding Clusters, Hubs and Outliers on Large- scale Graphs. PVLDB, 8(11):1178-1189, 2015.","title":"SCAN++: Efficient Algorithm for Finding Clusters, Hubs and Outliers on Largescale Graphs","authors":"Hiroaki Shiokawa, Yasuhiro Fujiwara, Makoto Onizuka","year":"2015","venue":"PVLDB","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tan et al., 2015] Kean Ming Tan, Daniela M. Witten, and Ali Shojaie. The Cluster Graphical Lasso for Improved Estimation of Gaussian Graphical Models. Computational Statistics & Data Analysis, 85:23-36, 2015.","title":"The Cluster Graphical Lasso for Improved Estimation of Gaussian Graphical Models","authors":"Ming Kean, Daniela M Tan, Ali Witten, Shojaie","year":"2015","venue":"Computational Statistics & Data Analysis","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Vinciotti et al., 2016] Veronica Vinciotti, Ernst C. Wit, Rick Jansen, Eco J. C. N. de Geus, Brenda W. J. H. Penninx, Dorret I. Boomsma, and Peter A. C. 't Hoen. Consistency of Biological Networks Inferred from Microarray and Se- quencing Data. BMC Bioinformatics, 17(1):254, 2016.","title":"Consistency of Biological Networks Inferred from Microarray and Sequencing Data","authors":"Veronica Vinciotti, Ernst C Wit, Rick Jansen, J C N Eco, Brenda W J H De Geus, Penninx, I Dorret, Peter A C Boomsma, 't Hoen","year":"2016","venue":"BMC Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Witten et al., 2011] Daniela M. Witten, Jerome H. Fried- man, and Noah Simon. New Insights and Faster Compu- tations for the Graphical Lasso. Journal of Computational and Graphical Statistics, 20(4):892-900, 2011.","title":"New Insights and Faster Computations for the Graphical Lasso","authors":"Daniela M Witten, Jerome H Friedman, Noah Simon","year":"2011","venue":"Journal of Computational and Graphical Statistics","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xu et al., 2016] Zenglin Xu, Shandian Zhe, Yuan Qi, and Peng Yu. Association Discovery and Diagnosis of Alzheimer's Disease with Bayesian Multiview Learning. JAIR, 56:247-268, 2016.","title":"Association Discovery and Diagnosis of Alzheimer's Disease with Bayesian Multiview Learning","authors":"Xu","year":"2016","venue":"JAIR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhao et al., 2015] Tuo Zhao, Xingguo Li, Han Liu, Kathryn Roeder, John Lafferty, and Larry Wasserman. Package 'huge'. https://cran.r- project.org/web/packages/huge/huge.pdf, 2015.","title":"Package 'huge","authors":"Tuo Zhao, Xingguo Li, Han Liu, Kathryn Roeder, John Lafferty, Larry Wasserman","year":"2015","venue":null,"doi":null,"url":"https://cran.r-project.org/web/packages/huge/huge.pdf","internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}