{"type":"ScholarlyArticle","title":"Lifted Hybrid Variational Inference","authors":[{"name":"Yuqiao Chen","slug":"yuqiao-chen","given":"Yuqiao","family":"Chen","affiliation":"The University of Texas at Dallas"},{"name":"Yibo Yang","slug":"yibo-yang","given":"Yibo","family":"Yang","affiliation":"University of California Irvine"},{"name":"Sriraam Natarajan","slug":"sriraam-natarajan","given":"Sriraam","family":"Natarajan","affiliation":"The University of Texas at Dallas"},{"name":"Nicholas Ruozzi","slug":"nicholas-ruozzi","given":"Nicholas","family":"Ruozzi","affiliation":"The University of Texas at Dallas"}],"abstract":"Lifted inference algorithms exploit model symmetry to reduce computational cost in probabilistic inference. However, most existing lifted inference algorithms operate only over discrete domains or continuous domains with restricted potential functions. We investigate two approximate lifted variational approaches that apply to domains with general hybrid potentials, and are expressive enough to capture multi-modality. We demonstrate that the proposed variational methods are highly scalable and can exploit approximate model symmetries even in the presence of a large amount of continuous evidence, outperforming existing message-passing-based approaches in a variety of settings. Additionally, we present a sufficient condition for the Bethe variational approximation to yield a non-trivial estimate over the marginal polytope.","doi":"10.24963/ijcai.2020/585","url":"https://ijcai.org/proceedings/2020/585","pdf":"https://ijcai.org/proceedings/2020/0585.pdf","markdown":"https://ijcai.org/proceedings/2020/585.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/585","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Uncertainty in AI","track":"Main track","number":585,"pages":{"first":4237,"last":4244},"keywords":["Uncertainty in AI: Statistical Relational AI","Uncertainty in AI: Graphical Models","Uncertainty in AI: Approximate Probabilistic Inference"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25226/file_25226.m3u8","length_sec":274,"provider":"hls"}],"references":[{"position":1,"raw":"[Ahmadi et al., 2011] Babak Ahmadi, Kristian Kersting, and Scott Sanner. Multi-evidence lifted message passing, with application to PageRank and the Kalman filter. In Interna- tional Joint Conference on Artificial Intelligence (IJCAI), 2011.","title":"Multi-evidence lifted message passing, with application to PageRank and the Kalman filter","authors":"Babak Ahmadi, Kristian Kersting, Scott Sanner","year":"2011","venue":"International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2011/197"},{"position":2,"raw":"[Bui et al., 2014] Hung Hai Bui, Tuyen N Huynh, and David Sontag. Lifted tree-reweighted variational inference. In Conference on Uncertainty in Artificial Intelligence (UAI), 2014.","title":"Lifted tree-reweighted variational inference","authors":"Hai Hung, Tuyen N Bui, David Huynh, Sontag","year":"2014","venue":"Conference on Uncertainty in Artificial Intelligence (UAI)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2019] Yuqiao Chen, Nicholas Ruozzi, and Sri- raam Natarajan. Lifted message passing for hybrid prob- abilistic inference. In International Joint Conference on Artificial Intelligence (IJCAI), 2019.","title":"Lifted message passing for hybrid probabilistic inference","authors":"Chen","year":"2019","venue":"International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2019/790"},{"position":4,"raw":"[Choi and Amir, 2012] Jaesik Choi and Eyal Amir. Lifted relational variational inference. In Conference on Uncer- tainty in Artificial Intelligence (UAI), 2012.","title":"Lifted relational variational inference","authors":"Amir Choi, Jaesik Choi, Eyal Amir","year":"2012","venue":"Conference on Uncertainty in Artificial Intelligence (UAI)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Choi et al., 2010] Jaesik Choi, Eyal Amir, and David Hill. Lifted inference for relational continuous models. In Con- ference on Uncertainty in Artificial Intelligence (UAI), 2010.","title":"Lifted inference for relational continuous models","authors":"Choi","year":"2010","venue":"Conference on Uncertainty in Artificial Intelligence (UAI)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Choi et al., 2011] Jaesik Choi, Abner Guzman-Rivera, and Eyal Amir. Lifted relational Kalman filtering. In Confer- ence on Artificial Intelligence (AAAI), 2011.","title":"Lifted relational Kalman filtering","authors":"Choi","year":"2011","venue":"Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":"/proceedings/2011/350"},{"position":7,"raw":"[Choi et al., 2015] Jaesik Choi, Eyal Amir, Tianfang Xu, and Albert J. Valocchi. Learning relational Kalman filtering. In Conference on Artificial Intelligence (AAAI), 2015.","title":"Valocchi. Learning relational Kalman filtering","authors":"Choi","year":"2015","venue":"Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cseke and Heskes, 2011] Botond Cseke and Tom Heskes. Properties of Bethe free energies and message passing in Gaussian models. Journal of Artificial Intelligence Re- search, pages 1-24, 2011.","title":"Properties of Bethe free energies and message passing in Gaussian models","authors":"Heskes Cseke, Botond Cseke, Tom Heskes","year":"2011","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gallo and Ihler, 2018] Nicholas Gallo and Alexander Ihler. Lifted generalized dual decomposition. In Conference on Artificial Intelligence (AAAI), 2018.","title":"Lifted generalized dual decomposition","authors":"Ihler Gallo, Nicholas Gallo, Alexander Ihler","year":"2018","venue":"Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gershman et al., 2012] Samuel Gershman, Matt Hoffman, and David Blei. Nonparametric variational inference. In International Conference on Machine Learning (ICML), pages 235-242, 2012.","title":"Nonparametric variational inference","authors":"Gershman","year":"2012","venue":"International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Golub and Welsch, 1969] Gene H. Golub and John H. Welsch. Calculation of Gauss quadrature rules. Mathe- matics of computation, 23(106):221-230, 1969.","title":"Calculation of Gauss quadrature rules","authors":"Gene H Golub, John H Welsch","year":"1969","venue":"Mathematics of computation","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Guo et al., 2019a] Yuanzhen Guo, Hao Xiong, and Nicholas Ruozzi. Marginal inference in continuous Markov random fields using mixtures. In Proceedings of the Conference on Artificial Intelligence (AAAI), volume 33, pages 7834- 7841, 2019.","title":"Marginal inference in continuous Markov random fields using mixtures","authors":"Guo","year":"2019","venue":"Proceedings of the Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Guo et al., 2019b] Yuanzhen Guo, Hao Xiong, Yibo Yang, and Nicholas Ruozzi. One-shot marginal MAP inference in Markov random fields. In Conference on Uncertainty in Artificial Intelligence (UAI), 2019.","title":"One-shot marginal MAP inference in Markov random fields","authors":"Guo","year":"2019","venue":"Conference on Uncertainty in Artificial Intelligence (UAI)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Habeeb et al., 2017] Haroun Habeeb, Ankit Anand, Mausam, and Parag Singla. Coarse-to-fine lifted map inference in computer vision. In International Joint Conference on Artificial Intelligence (IJCAI), 2017.","title":"Coarse-to-fine lifted map inference in computer vision","authors":"Haroun Habeeb, Ankit Anand, Parag Mausam, Singla","year":"2017","venue":"International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2017/641"},{"position":15,"raw":"[Jaakkola and Jordan, 1998] Tommi S. Jaakkola and Michael Jordan. Improving the mean field approximation via the use of mixture distributions. In Learning in graphical models, pages 163-173. Springer, 1998.","title":"Improving the mean field approximation via the use of mixture distributions","authors":"Tommi S Jaakkola, Michael Jordan","year":"1998","venue":"Learning in graphical models","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kersting et al., 2009] Kristian Kersting, Babak Ahmadi, and Sriraam Natarajan. Counting belief propagation. In Conference on Uncertainty in Artificial Intelligence (UAI), 2009.","title":"Counting belief propagation","authors":"Kristian Kersting, Babak Ahmadi, Sriraam Natarajan","year":"2009","venue":"Conference on Uncertainty in Artificial Intelligence (UAI)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Koller and Friedman, 2009] Daphne Koller and Nir Fried- man. Probabilistic graphical models: principles and tech- niques. MIT press, 2009.","title":"Probabilistic graphical models: principles and techniques","authors":"Friedman Koller","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lienart et al., 2015] Thibaut Lienart, Yee Whye Teh, and Arnaud Doucet. Expectation particle belief propagation. In Advances in Neural Information Processing Systems (NeurIPS), 2015.","title":"Expectation particle belief propagation","authors":"Thibaut Lienart, Yee Whye Teh, Arnaud Doucet","year":"2015","venue":"Advances in Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu and Wang, 2016] Qiang Liu and Dilin Wang. Stein vari- ational gradient descent: A general purpose bayesian in- ference algorithm. In Advances in Neural Information Pro- cessing Systems (NeurIPS), 2016.","title":"Stein variational gradient descent: A general purpose bayesian inference algorithm","authors":"Wang ; ] Qiang Liu, Dilin Liu, Wang","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[McKusick, 2003] Vincent L. McKusick. Final report for the special master with certificate of adoption of RRCA groundwater model. U.S. Supreme Court, 2003.","title":"Final report for the special master with certificate of adoption of RRCA groundwater model","authors":"L Vincent, Mckusick","year":"2003","venue":"U.S. Supreme Court","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ruozzi, 2012] Nicholas Ruozzi. The Bethe partition func- tion of log-supermodular graphical models. In Neural In- formation Processing Systems (NeurIPS), 2012.","title":"The Bethe partition function of log-supermodular graphical models","authors":"Nicholas Ruozzi","year":"2012","venue":"Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ruozzi, 2013] Nicholas Ruozzi. Beyond log- supermodularity: Lower bounds and the Bethe partition function. In Proceedings of the Twenty-Ninth Confer- ence Annual Conference on Uncertainty in Artificial Intelligence (UAI), 2013.","title":"Beyond logsupermodularity: Lower bounds and the Bethe partition function","authors":"Nicholas Ruozzi","year":"2013","venue":"Proceedings of the Twenty-Ninth Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Ruozzi, 2017] Nicholas Ruozzi. A lower bound on the par- tition function of attractive graphical models in the con- tinuous case. In Artificial Intelligence and Statistics (AIS- TATS), 2017.","title":"A lower bound on the partition function of attractive graphical models in the continuous case","authors":"Nicholas Ruozzi","year":"2017","venue":"Artificial Intelligence and Statistics (AIS-TATS)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Singla and Domingos, 2008] Parag Singla and Pedro Domingos. Lifted first-order belief propagation. In Twenty-Second AAAI Conference on Artificial Intelligence (AAAI), 2008.","title":"Lifted first-order belief propagation","authors":"Parag Singla, Pedro Domingos","year":"2008","venue":"Twenty-Second AAAI Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang and Domingos, 2008] Jue Wang and Pedro Domin- gos. Hybrid markov logic networks. In Conference on Artificial Intelligence (AAAI), 2008.","title":"Hybrid markov logic networks","authors":"Jue Wang, Pedro Domingos","year":"2008","venue":"Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Welling and Teh, 2001] Max Welling and Yee Whye Teh. Belief optimization for binary networks: A stable alter- native to loopy belief propagation. In Conference on Un- certainty in Artificial Intelligence (UAI), 2001.","title":"Belief optimization for binary networks: A stable alternative to loopy belief propagation","authors":"Teh ; ] Max Welling, Yee Whye Welling, Teh","year":"2001","venue":"Conference on Uncertainty in Artificial Intelligence (UAI)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yedidia et al., 2001] Jonathan S. Yedidia, William T. Free- man, and Yair Weiss. Bethe free energy, Kikuchi approxi- mations, and belief propagation algorithms. Neural Infor- mation Processing Systems (NeurIPS), 2001.","title":"Bethe free energy, Kikuchi approximations, and belief propagation algorithms","authors":"Jonathan S Yedidia, William T Freeman, Yair Weiss","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yedidia et al., 2005] Jonathan S. Yedidia, William T. Free- man, and Yair Weiss. Constructing free-energy approx- imations and generalized belief propagation algorithms. Information Theory, IEEE Transactions on, 51(7):2282 - 2312, July 2005.","title":"Constructing free-energy approximations and generalized belief propagation algorithms","authors":"Jonathan S Yedidia, William T Freeman, Yair Weiss","year":"2005","venue":"IEEE Transactions on","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}