{"type":"ScholarlyArticle","title":"Fast Sparse Gaussian Markov Random Fields Learning Based on Cholesky Factorization","authors":[{"name":"Ivan Stojkovic","slug":"ivan-stojkovic","given":"Ivan","family":"Stojkovic","affiliation":"Center for Data Analytics and Biomedical Informatics , Temple University , Philadelphia , PA , USA; School of Electrical Engineering , University of Belgrade , Belgrade , Serbia"},{"name":"Vladisav Jelisavcic","slug":"vladisav-jelisavcic","given":"Vladisav","family":"Jelisavcic","affiliation":"Mathematical Institute , Serbian Academy of Sciences and Arts , Belgrade , Serbia; School of Electrical Engineering , University of Belgrade , Belgrade , Serbia"},{"name":"Veljko Milutinovic","slug":"veljko-milutinovic","given":"Veljko","family":"Milutinovic","affiliation":"School of Electrical Engineering , University of Belgrade , Belgrade , Serbia"},{"name":"Zoran Obradovic","slug":"zoran-obradovic","given":"Zoran","family":"Obradovic","affiliation":"Center for Data Analytics and Biomedical Informatics , Temple University , Philadelphia , PA , USA"}],"abstract":"Learning the sparse Gaussian Markov Random Field, or conversely, estimating the sparse inverse covariance matrix is an approach to uncover the underlying dependency structure in data. Most of the current methods solve the problem by optimizing the maximum likelihood objective with a Laplace prior L1 on entries of a precision matrix. We propose a novel objective with a regularization term which penalizes an approximate product of the Cholesky decomposed precision matrix. This new reparametrization of the penalty term allows efficient coordinate descent optimization, which in synergy with an active set approach results in a very fast and efficient method for learning the sparse inverse covariance matrix. We evaluated the speed and solution quality of the newly proposed SCHL method on problems consisting of up to 24,840 variables. Our approach was several times faster than three state-of-the-art approaches. We also demonstrate that SCHL can be used to discover interpretable networks, by applying it to a high impact problem from the health informatics domain.","doi":"10.24963/ijcai.2017/384","url":"https://ijcai.org/proceedings/2017/384","pdf":"https://ijcai.org/proceedings/2017/0384.pdf","markdown":"https://ijcai.org/proceedings/2017/384.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/384","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":384,"pages":{"first":2758,"last":2764},"keywords":["Machine Learning: Learning Graphical Models","Multidisciplinary Topics and Applications: Computational Biology and e-Health","Machine Learning: Unsupervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Banerjee et al., 2008] Onureena Banerjee, Laurent El Ghaoui, and Alexandre d'Aspremont. Model se- lection through sparse maximum likelihood estimation for multivariate gaussian or binary data. The Journal of Machine Learning Research, 9:485-516, 2008.","title":"Onureena Banerjee, Laurent El Ghaoui, and Alexandre d'Aspremont. Model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data","authors":"Banerjee","year":"2008","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Banerjee et al., 2014] Sudipto Banerjee, Bradley P Carlin, and Alan E Gelfand. Hierarchical modeling and analysis for spatial data. Crc Press, 2014.","title":"Hierarchical modeling and analysis for spatial data","authors":"Sudipto Banerjee, Alan E Bradley P Carlin, Gelfand","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Boyd and Vandenberghe, 2004] Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cambridge, 2004.","title":"Convex optimization","authors":"Vandenberghe Boyd, Stephen Boyd, Lieven Vandenberghe","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dahl et al., 2008] Joachim Dahl, Lieven Vandenberghe, and Vwani Roychowdhury. Covariance selection for non- chordal graphs via chordal embedding. Optimization Methods & Software, 23(4):501-520, 2008.","title":"Covariance selection for nonchordal graphs via chordal embedding","authors":"Joachim Dahl, Lieven Vandenberghe, Vwani Roychowdhury","year":"2008","venue":"Optimization Methods & Software","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dempster, 1972] Arthur P Dempster. Covariance selection. Biometrics, pages 157-175, 1972.","title":"Covariance selection","authors":"P Arthur, Dempster","year":"1972","venue":"Biometrics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dobra et al., 2004] Adrian Dobra, Chris Hans, Beatrix Jones, Joseph Nevins, Guang Yao, and Mike West. Sparse graphical models for exploring gene expression data. Jour- nal of Multivariate Analysis, 90(1):196-212, 2004.","title":"Sparse graphical models for exploring gene expression data","authors":"Adrian Dobra, Chris Hans, Beatrix Jones, Joseph Nevins, Guang Yao, Mike West","year":"2004","venue":"Journal of Multivariate Analysis","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Duchi et al., 2008] John Duchi, Stephen Gould, Daphne Koller, et al. Projected subgradient methods for learn- ing sparse gaussians. In Proceedings of the Twenty-Fourth Conference on Uncertainty in Artificial Intelligence, 2008.","title":"Projected subgradient methods for learning sparse gaussians","authors":"John Duchi, Stephen Gould, Daphne Koller","year":"2008","venue":"Proceedings of the Twenty-Fourth Conference on Uncertainty in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Friedman et al., 2008] Jerome Friedman, Trevor Hastie, and Robert Tibshirani. Sparse inverse covariance estima- tion with the graphical lasso. Biostatistics, 9(3), 2008.","title":"Sparse inverse covariance estimation with the graphical lasso","authors":"Jerome Friedman, Trevor Hastie, Robert Tibshirani","year":"2008","venue":"Biostatistics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hsieh et al., 2011] Cho-Jui Hsieh, Inderjit S Dhillon, Pradeep K Ravikumar, and Mátyás A Sustik. Sparse in- verse covariance matrix estimation using quadratic ap- proximation. In Advances in Neural Information Process- ing Systems, pages 2330-2338, 2011.","title":"Sparse inverse covariance matrix estimation using quadratic approximation","authors":"Cho-Jui Hsieh, Pradeep K Inderjit S Dhillon, Mátyás A Ravikumar, Sustik","year":"2011","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hsieh et al., 2013] Cho-Jui Hsieh, Mátyás A Sustik, Inder- jit S Dhillon, Pradeep K Ravikumar, and Russell Poldrack. Big & quic: Sparse inverse covariance estimation for a million variables. In Advances in Neural Information Pro- cessing Systems, pages 3165-3173, 2013.","title":"Big & quic: Sparse inverse covariance estimation for a million variables","authors":"Cho-Jui Hsieh, Mátyás A Sustik, Pradeep K Inderjit S Dhillon, Russell Ravikumar, Poldrack","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Huang et al., 2006] Jianhua Z Huang, Naiping Liu, Mohsen Pourahmadi, and Linxu Liu. Covariance matrix se- lection and estimation via penalised normal likelihood. Biometrika, 93(1):85-98, 2006.","title":"Covariance matrix selection and estimation via penalised normal likelihood","authors":"Z Jianhua, Naiping Huang, Mohsen Liu, Linxu Pourahmadi, Liu","year":"2006","venue":"Biometrika","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Koller and Friedman, 2009] Daphne Koller and Nir Fried- man. Probabilistic graphical models: principles and tech- niques. MIT press, 2009.","title":"Probabilistic graphical models: principles and techniques","authors":"Friedman Koller","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lauritzen, 1996] Steffen L Lauritzen. Graphical models. Clarendon Press, 1996.","title":"Graphical models","authors":"Steffen L Lauritzen","year":"1996","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li, 2012] Stan Z Li. Markov random field modeling in com- puter vision. Springer Science & Business Media, 2012.","title":"Markov random field modeling in computer vision","authors":"Stan Z Li","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Meinshausen and Bühlmann, 2006] Nicolai Meinshausen and Peter Bühlmann. High-dimensional graphs and variable selection with the lasso. The annals of statistics, 34(3):1436-1462, 2006.","title":"High-dimensional graphs and variable selection with the lasso","authors":"Bühlmann Meinshausen, Nicolai Meinshausen, Peter Bühlmann","year":"2006","venue":"The annals of statistics","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Parnell et al., 2013] Grant P Parnell, Benjamin M Tang, Marek Nalos, Nicola J Armstrong, Stephen J Huang, David R Booth, and Anthony S McLean. Identifying key regulatory genes in the whole blood of septic pa- tients to monitor underlying immune dysfunctions. Shock, 40(3):166-174, 2013.","title":"Identifying key regulatory genes in the whole blood of septic patients to monitor underlying immune dysfunctions","authors":"Parnell","year":"2013","venue":"Shock","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rolfs et al., 2012] Benjamin Rolfs, Bala Rajaratnam, Do- minique Guillot, Ian Wong, and Arian Maleki. Iterative thresholding algorithm for sparse inverse covariance esti- mation. In Advances in Neural Information Processing Systems, pages 1574-1582, 2012.","title":"Iterative thresholding algorithm for sparse inverse covariance estimation","authors":"Benjamin Rolfs, Bala Rajaratnam, Dominique Guillot, Ian Wong, Arian Maleki","year":"2012","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Scheinberg and Rish, 2010] Katya Scheinberg and Irina Rish. Learning sparse gaussian markov networks using a greedy coordinate ascent approach. In Machine Learning and Knowledge Discovery in Databases, pages 196-212. Springer, 2010.","title":"Learning sparse gaussian markov networks using a greedy coordinate ascent approach","authors":"Katya Scheinberg, Irina Rish","year":"2010","venue":"Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Stojkovic et al., 2016a] Ivan Stojkovic, Mohamed F. Ghal- wash, Xi Hang Cao, and Zoran Obradovic. Effective- ness of Multiple Blood-Cleansing Interventions in Sepsis, Characterized in Rats. Scientific Reports, 6(24719):1-11, 2016.","title":"Effectiveness of Multiple Blood-Cleansing Interventions in Sepsis, Characterized in Rats","authors":"Stojkovic","year":"2016","venue":"Scientific Reports","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Stojkovic et al., 2016b] Ivan Stojkovic, Vladisav Jelisavcic, Veljko Milutinovic, and Zoran Obradovic. Distance based modeling of interactions in structured regression. In Pro- cedeengs of the 25th International Joint Conference on Ar- tificial Intelligence (IJCAI-16), pages 2032-2038, 2016.","title":"Distance based modeling of interactions in structured regression","authors":"Stojkovic","year":"2016","venue":"Procedeengs of the 25th International Joint Conference on Artificial Intelligence (IJCAI-16)","doi":null,"url":null,"internal_url":"/proceedings/2016/290"},{"position":21,"raw":"[Stuart et al., 2003] Joshua M Stuart, Eran Segal, Daphne Koller, and Stuart K Kim. A gene-coexpression network for global discovery of conserved genetic modules. Sci- ence, 302(5643):249-255, 2003.","title":"A gene-coexpression network for global discovery of conserved genetic modules","authors":"Joshua M Stuart, Eran Segal, Daphne Koller, Stuart K Kim","year":"2003","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tibshirani, 1996] Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), pages 267-288, 1996.","title":"Regression shrinkage and selection via the lasso","authors":"Robert Tibshirani","year":"1996","venue":"Journal of the Royal Statistical Society. Series B (Methodological)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Treister and Turek, 2014] Eran Treister and Javier Turek. A block-coordinate descent approach for large-scale sparse inverse covariance estimation. In Advances in Neural In- formation Processing Systems, pages 927-935, 2014.","title":"A block-coordinate descent approach for large-scale sparse inverse covariance estimation","authors":"Turek Treister, Eran Treister, Javier Turek","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2016] Min Wang, Jingjun Yan, Xingxing He, Qiang Zhong, Chengye Zhan, and Shusheng Li. Can- didate genes and pathogenesis investigation for sepsis- related acute respiratory distress syndrome based on gene expression profile. Biological research, 49(1):1-9, 2016.","title":"Candidate genes and pathogenesis investigation for sepsisrelated acute respiratory distress syndrome based on gene expression profile","authors":"Wang","year":"2016","venue":"Biological research","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wen et al., 2012] Zaiwen Wen, Wotao Yin, Hongchao Zhang, and Donald Goldfarb. On the convergence of an active-set method for l1 minimization. Optimization Meth- ods and Software, 27(6):1127-1146, 2012.","title":"On the convergence of an active-set method for l1 minimization","authors":"Wen","year":"2012","venue":"Optimization Methods and Software","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wytock and Kolter, 2013] Matt Wytock and Zico Kolter. Sparse gaussian conditional random fields: Algorithms, theory, and application to energy forecasting. In Interna- tional conference on machine learning, pages 1265-1273, 2013.","title":"Sparse gaussian conditional random fields: Algorithms, theory, and application to energy forecasting","authors":"Kolter Wytock, Matt Wytock, Zico Kolter","year":"2013","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yuan and Lin, 2007] Ming Yuan and Yi Lin. Model se- lection and estimation in the gaussian graphical model. Biometrika, 94(1):19-35, 2007.","title":"Model selection and estimation in the gaussian graphical model","authors":"Lin ; Yuan, Ming Yuan, Yi Lin","year":"2007","venue":"Biometrika","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}