{"type":"ScholarlyArticle","title":"Entropy-Penalized Semidefinite Programming","authors":[{"name":"Mikhail Krechetov","slug":"mikhail-krechetov","given":"Mikhail","family":"Krechetov","affiliation":"Skolkovo Institute of Science and Technology , Nobel Street 3 , Moscow , Russia"},{"name":"Jakub Marecek","slug":"jakub-marecek","given":"Jakub","family":"Marecek","affiliation":"IBM Research -Ireland , Technology Campus Damastown , Dublin D15 , Ireland"},{"name":"Yury Maximov","slug":"yury-maximov","given":"Yury","family":"Maximov","affiliation":"Skolkovo Institute of Science and Technology , Nobel Street 3 , Moscow , Russia; Los Alamos National Laboratory , MS-B284 , Los Alamos , NM 87545 , USA"},{"name":"Martin Takac","slug":"martin-takac","given":"Martin","family":"Takac","affiliation":"Lehigh University , 200 West Packer Avenue , Bethlehem , PA 18015 , USA"}],"abstract":"Low-rank methods for semi-definite programming (SDP) have gained a lot of interest recently, especially in machine learning applications. Their analysis often involves determinant-based or Schatten-norm penalties, which are difficult to implement in practice due to high computational efforts. In this paper, we propose Entropy-Penalized Semi-Definite Programming (EP-SDP), which provides a unified framework for a broad class of penalty functions used in practice to promote a low-rank solution. We show that EP-SDP problems admit an efficient numerical algorithm, having (almost) linear time complexity of the gradient computation; this makes it useful for many machine learning and optimization problems. We illustrate the practical efficiency of our approach on several combinatorial optimization and machine learning problems.","doi":"10.24963/ijcai.2019/157","url":"https://ijcai.org/proceedings/2019/157","pdf":"https://ijcai.org/proceedings/2019/0157.pdf","markdown":"https://ijcai.org/proceedings/2019/157.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/157","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Constraints and SAT","track":"Main track","number":157,"pages":{"first":1123,"last":1129},"keywords":["Uncertainty in AI: Uncertainty in AI","Constraints and SAT: Constraint Satisfaction","Heuristic Search and Game Playing: Combinatorial Search and Optimisation"],"videos":[],"references":[{"position":1,"raw":"[Arora et al., 2012] Sanjeev Arora, Rong Ge, Ravindran Kan- nan, and Ankur Moitra. Computing a nonnegative matrix factorization-provably. In Proceedings of the forty-fourth annual ACM Symposium on Theory of Computing, pages 145-162. ACM, 2012.","title":"Computing a nonnegative matrix factorization-provably","authors":"Sanjeev Arora, Rong Ge, Ravindran Kannan, Ankur Moitra","year":"2012","venue":"Proceedings of the forty-fourth annual ACM Symposium on Theory of Computing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Boumal et al., 2016] Nicolas Boumal, Vlad Voroninski, and Afonso Bandeira. The non-convex Burer-Monteiro ap- proach works on smooth semidefinite programs. In Ad- vances in Neural Information Processing Systems, pages 2757-2765, 2016.","title":"Afonso Bandeira. The non-convex Burer-Monteiro approach works on smooth semidefinite programs","authors":"Boumal","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Burer and Monteiro, 2003] Samuel Burer and Renato DC Monteiro. A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization. Mathe- matical Programming, 95:329-357, 2003.","title":"A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization","authors":"Monteiro Burer, Samuel Burer, Renato Dc Monteiro","year":"2003","venue":"Mathematical Programming","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Burer and Monteiro, 2005] Samuel Burer and Renato DC Monteiro. Local minima and convergence in low-rank semidefinite programming. Mathematical Programming, 103(3):427-444, 2005.","title":"Local minima and convergence in low-rank semidefinite programming","authors":"Monteiro Burer, Samuel Burer, Renato Dc Monteiro","year":"2005","venue":"Mathematical Programming","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Erdogdu et al., 2017] Murat Erdogdu, Yash Deshpande, and Andrea Montanari. Inference in graphical models via semidefinite programming hierarchies. In Advances in Neural Information Processing Systems, pages 416-424, 2017.","title":"Inference in graphical models via semidefinite programming hierarchies","authors":"Murat Erdogdu, Yash Deshpande, Andrea Montanari","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Erdogdu et al., 2018] Murat A. Erdogdu, Asuman Ozdaglar, Pablo A. Parrilo, and Nuri Denizcan Vanli. Con- vergence rate of block-coordinate maximization Burer- Monteiro method for solving large sdps. arXiv preprint arXiv:1807.04428, 2018.","title":"Convergence rate of block-coordinate maximization Burer-Monteiro method for solving large sdps","authors":"A Murat, Asuman Erdogdu, Pablo A Ozdaglar, Nuri Denizcan Parrilo, Vanli","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fawzi et al., 2019] Hamza Fawzi, James Saunderson, and Pablo A. Parrilo. Semidefinite approximations of the matrix logarithm. Foundations of Computational Mathematics, 19(2):259-296, Apr 2019.","title":"Semidefinite approximations of the matrix logarithm","authors":"Hamza Fawzi, James Saunderson, Pablo A Parrilo","year":"2019","venue":"Foundations of Computational Mathematics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Frostig et al., 2014] Roy Frostig, Sida Wang, Percy S. Liang, and Christopher D. Manning. Simple MAP inference via low-rank relaxations. In Advances in Neural Information Processing Systems, pages 3077-3085, 2014.","title":"Simple MAP inference via low-rank relaxations","authors":"Roy Frostig, Sida Wang, Percy S Liang, Christopher D Manning","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ghaddar et al., 2016] Bissan Ghaddar, Jakub Marecek, and Martin Mevissen. Optimal power flow as a polynomial op- timization problem. IEEE Transactions on Power Systems, 31(1):539-546, Jan 2016.","title":"Optimal power flow as a polynomial optimization problem","authors":"Ghaddar","year":"2016","venue":"IEEE Transactions on Power Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Goemans and Williamson, 1995] Michel Goemans and David P. Williamson. Improved approximation algorithms for maximum cut and satisfiability problems using semidef- inite programming. Journal of the ACM, 42(6):1115-1145, 1995.","title":"Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming","authors":"Michel Goemans, David P Williamson","year":"1995","venue":"Journal of the ACM","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Holmes et al., 2007] Michael Holmes, Alexander Gray, and Charles Isbell. Fast SVD for large-scale matrices. In Work- shop on Efficient Machine Learning at NIPS, volume 58, pages 249-252. 2007.","title":"Fast SVD for large-scale matrices","authors":"Michael Holmes, Alexander Gray, Charles Isbell","year":"2007","venue":"Workshop on Efficient Machine Learning at NIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hu et al., 2013] Yao Hu, Debing Zhang, Jieping Ye, Xuelong Li, and Xiaofei He. Fast and accurate matrix completion via truncated nuclear norm regularization. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 2117- 2130, 2013.","title":"Fast and accurate matrix completion via truncated nuclear norm regularization","authors":"Yao Hu, Debing Zhang, Jieping Ye, Xuelong Li, Xiaofei He","year":"2013","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Keshavan et al., 2010] Raghunandan Keshavan, Andrea Montanari, and Sewoong Oh. Matrix completion from a few entries. IEEE Transactions on Information Theory, 56(6):2980-2998, 2010.","title":"Matrix completion from a few entries","authors":"Raghunandan Keshavan, Andrea Montanari, Sewoong Oh","year":"2010","venue":"IEEE Transactions on Information Theory","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Krislock et al., 2014] Nathan Krislock, Jérôme Malick, and Frédéric Roupin. Improved semidefinite bounding proce- dure for solving max-cut problems to optimality. Mathe- matical Programming, 143(1-2):61-86, 2014.","title":"Improved semidefinite bounding procedure for solving max-cut problems to optimality","authors":"Nathan Krislock, Jérôme Malick, Frédéric Roupin","year":"2014","venue":"Mathematical Programming","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lasserre, 2015] Jean-Bernard Lasserre. An introduction to polynomial and semi-algebraic optimization, volume 52. Cambridge University Press, 2015.","title":"An introduction to polynomial and semi-algebraic optimization","authors":"Jean-Bernard Lasserre","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lemon et al., 2016] Alex Lemon, Anthony Man-Cho So, and Yinyu Ye. Low-rank semidefinite programming: Theory and applications. Foundations and Trends R in Optimiza- tion, 2016.","title":"Low-rank semidefinite programming: Theory and applications","authors":"Alex Lemon, Anthony, Man-Cho So, Yinyu Ye","year":"2016","venue":"Foundations and Trends R in Optimization","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Madani et al., 2014] Ramtin Madani, Ghazal Fazelnia, So- mayeh Sojoudi, and Javad Lavaei. Low-rank solutions of matrix inequalities with applications to polynomial opti- mization and matrix completion problems. In 53rd IEEE Conference on Decision and Control, pages 4328-4335, Dec 2014.","title":"Low-rank solutions of matrix inequalities with applications to polynomial optimization and matrix completion problems","authors":"Ramtin Madani, Ghazal Fazelnia, Somayeh Sojoudi, Javad Lavaei","year":"2014","venue":"53rd IEEE Conference on Decision and Control","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mareček and Takáč, 2017] Jakub Mareček and Martin Takáč. A low-rank coordinate-descent algorithm for semidefinite programming relaxations of optimal power flow. Optimization Methods and Software, 32(4):849-871, 2017.","title":"A low-rank coordinate-descent algorithm for semidefinite programming relaxations of optimal power flow","authors":"Jakub Mareček, Martin Takáč","year":"2017","venue":"Optimization Methods and Software","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mei et al., 2017] Song Mei, Theodor Misiakiewicz, Andrea Montanari, and Roberto Imbuzeiro Oliveira. Solving SDPs for synchronization and MaxCut problems via the Grothendieck inequality. In 2017 Conference on Learning Theory, volume 65 of Proceedings of Machine Learning Research, pages 1476-1515, 07-10 Jul 2017.","title":"Solving SDPs for synchronization and MaxCut problems via the Grothendieck inequality","authors":"Mei","year":"2017","venue":"Proceedings of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nesterov, 1998] Yury Nesterov. Semidefinite relaxation and nonconvex quadratic optimization. Optimization Methods and Software, 9(1-3):141-160, 1998.","title":"Semidefinite relaxation and nonconvex quadratic optimization","authors":"Yury Nesterov","year":"1998","venue":"Optimization Methods and Software","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Parrilo, 2003] Pablo Parrilo. Semidefinite programming re- laxations for semialgebraic problems. Mathematical pro- gramming, page 293-320, 2003.","title":"Pablo Parrilo. Semidefinite programming relaxations for semialgebraic problems","authors":"Parrilo","year":"2003","venue":"Mathematical programming","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pogodin et al., 2017] Roman Pogodin, Mikhail Krechetov, and Yury Maximov. Efficient rank minimization to tighten semidefinite programming for unconstrained binary quadratic optimization. 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pages 1153-1159, 2017.","title":"Efficient rank minimization to tighten semidefinite programming for unconstrained binary quadratic optimization","authors":"Pogodin","year":"2017","venue":"55th Annual Allerton Conference on Communication, Control, and Computing","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Recht et al., 2010] Benjamin Recht, Maryam Fazel, and Pablo A Parrilo. Guaranteed minimum-rank solutions of lin- ear matrix equations via nuclear norm minimization. SIAM Review, 52(3):471-501, 2010.","title":"Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization","authors":"Benjamin Recht, Maryam Fazel, Pablo A Parrilo","year":"2010","venue":"SIAM Review","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shang et al., 2018] Fanhua Shang, James Cheng, Yuanyuan Liu, Zhi-Quan Luo, and Zhouchen Lin. Bilinear factor matrix norm minimization for robust PCA: Algorithms and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(9):2066-2080, 2018.","title":"Bilinear factor matrix norm minimization for robust PCA: Algorithms and applications","authors":"Fanhua Shang, James Cheng, Yuanyuan Liu, Zhi-Quan Luo, Zhouchen Lin","year":"2018","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Srebro et al., 2004] Nathan Srebro, Jason D. M. Rennie, and Tommi S. Jaakkola. Maximum-margin matrix factorization. In Advances in Neural Information Processing Systems, NIPS'04, pages 1329-1336, 2004.","title":"Maximum-margin matrix factorization","authors":"Nathan Srebro, Jason D M Rennie, Tommi S Jaakkola","year":"2004","venue":"Advances in Neural Information Processing Systems, NIPS'04","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sun et al., 2013] Qian Sun, Shuo Xiang, and Jieping. Ye. Ro- bust principal component analysis via capped norms. In Proceedings of the 19th ACM SIGKDD international con- ference on Knowledge discovery and data mining, pages 311-319, 2013.","title":"Robust principal component analysis via capped norms","authors":"Qian Sun, Shuo Xiang, Jieping Ye","year":"2013","venue":"Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Torr, 2003] Philip Torr. Solving Markov random fields using semidefinite programming. In Proceedings of the Ninth International Workshop on Artificial Intelligence and Statis- tics, AISTATS, pages 1-8, 2003.","title":"Solving Markov random fields using semidefinite programming","authors":"Philip Torr","year":"2003","venue":"Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics, AISTATS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wainwright and Jordan, 2008] Martin J. Wainwright and Michael Jordan. Graphical models, exponential families, and variational inference. Foundations and Trends R in Machine Learning, 2008.","title":"Graphical models, exponential families, and variational inference","authors":"Martin J Wainwright, Michael Jordan","year":"2008","venue":"Foundations and Trends R in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhou, 2019] Cunlu Zhou. Entropy, optimization and coding theory. PhD thesis, University of Notre Dame, Notre Dame, Indiana, 2019.","title":"Entropy, optimization and coding theory","authors":"Cunlu Zhou, Zhou","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}