{"type":"ScholarlyArticle","title":"Exchangeability-Aware Sum-Product Networks","authors":[{"name":"Stefan Lüdtke","slug":"stefan-ludtke","given":"Stefan","family":"Lüdtke","affiliation":"Institute for Enterprise Systems , University of Mannheim , Germany"},{"name":"Christian Bartelt","slug":"christian-bartelt","given":"Christian","family":"Bartelt","affiliation":"Institute for Enterprise Systems , University of Mannheim , Germany"},{"name":"Heiner Stuckenschmidt","slug":"heiner-stuckenschmidt","given":"Heiner","family":"Stuckenschmidt","affiliation":"Data and Web Science Group , University of Mannheim , Germany"}],"abstract":"Sum-Product Networks (SPNs) are expressive probabilistic models that provide exact, tractable inference. They achieve this efficiency by making use of local independence. \r\nOn the other hand, mixtures of exchangeable variable models (MEVMs) are a class of tractable probabilistic models that make use of exchangeability of discrete random variables to render inference tractable. Exchangeability, which arises naturally in relational domains, has not been considered for efficient representation and inference in SPNs yet.\r\nThe contribution of this paper is a novel probabilistic model which we call Exchangeability-Aware Sum-Product Networks (XSPNs). It contains both SPNs and MEVMs as special cases, and combines the ability of SPNs to efficiently learn deep probabilistic models with the ability of MEVMs to efficiently handle exchangeable random variables. We introduce a structure learning algorithm for XSPNs and empirically show that they can be more accurate than conventional SPNs when the data contains repeated, interchangeable parts.","doi":"10.24963/ijcai.2022/674","url":"https://ijcai.org/proceedings/2022/674","pdf":"https://ijcai.org/proceedings/2022/0674.pdf","markdown":"https://ijcai.org/proceedings/2022/674.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/674","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Uncertainty in AI","track":"Main Track","number":674,"pages":{"first":4864,"last":4870},"keywords":["Uncertainty in AI: Graphical Models","Uncertainty in AI: Inference","Uncertainty in AI: Tractable Probabilistic Models"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986558","length_sec":89,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984701","length_sec":318,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bergmann et al., 2018] Henning Bergmann, Ste- fanie Bailer, Tamaki Ohmura, Thomas Saalfeld, and Ulrich Sieberer. BTVote Voting Behavior. https://dataverse.harvard.edu/dataverse/btvote, 2018. Accessed: 2022-05-09.","title":"BTVote Voting Behavior","authors":"Henning Bergmann, Stefanie Bailer, Tamaki Ohmura, Thomas Saalfeld, Ulrich Sieberer","year":"2022","venue":null,"doi":null,"url":"https://dataverse.harvard.edu/dataverse/btvote,2018","internal_url":null},{"position":2,"raw":"[De Finetti, 1938] Bruno De Finetti. Sur la condition d'equivalence partielle. 1938. English translation in: Studies in Inductive Logic and Probability 2, 1980.","title":"Sur la condition d'equivalence partielle","authors":"De Finetti","year":"1938","venue":"Studies in Inductive Logic and Probability","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Delalleau and Bengio, 2011] Olivier Delalleau and Yoshua Bengio. Shallow vs. deep sum-product networks. Ad- vances in neural information processing systems, 24:666- 674, 2011.","title":"Shallow vs. deep sum-product networks","authors":"Bengio Delalleau, Olivier Delalleau, Yoshua Bengio","year":"2011","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Diaconis and Freedman, 1980] Persi Diaconis and David Freedman. De finetti's generalizations of exchangeability. Studies in Inductive Logic and Probability II, 1980.","title":"De finetti's generalizations of exchangeability","authors":"Persi Diaconis, David Freedman","year":"1980","venue":"Studies in Inductive Logic and Probability","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gens and Domingos, 2013] Robert Gens and Predro Domingos. Learning the structure of sum-product net- works. In International conference on machine learning, 2013.","title":"Robert Gens and Predro Domingos. Learning the structure of sum-product networks","authors":"Domingos Gens","year":"2013","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Germain et al., 2015] Mathieu Germain, Karol Gregor, Iain Murray, and Hugo Larochelle. Made: Masked autoen- coder for distribution estimation. In International Con- ference on Machine Learning, 2015.","title":"Made: Masked autoencoder for distribution estimation","authors":"Karol Mathieu Germain, Iain Gregor, Hugo Murray, Larochelle","year":"2015","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jaeger and Van den Broeck, 2012] Manfred Jaeger and Guy Van den Broeck. Liftability of probabilistic inference: Up- per and lower bounds. In Proceedings of the 2nd interna- tional workshop on statistical relational AI, 2012.","title":"Liftability of probabilistic inference: Upper and lower bounds","authors":"Van Jaeger, Manfred Den Broeck, Guy Jaeger, Van Den Broeck","year":"2012","venue":"Proceedings of the 2nd international workshop on statistical relational AI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kingma and Welling, 2014] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In Proc. of ICLR, 2014.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2014","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Larochelle and Murray, 2011] Hugo Larochelle and Iain Murray. The neural autoregressive distribution estimator. In Artificial Intelligence and Statistics, pages 29-37, 2011.","title":"The neural autoregressive distribution estimator","authors":"Murray ; Larochelle, Hugo Larochelle, Iain Murray","year":"2011","venue":"Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lewis et al., 2021] Jeffrey B. Lewis, Keith Poole, Howard Rosenthal, Adam Boche, Aaron Rudkin, and Luke Sonnet. Voteview: Congressional roll-call votes database. https: //voteview.com, 2021. Accessed: 2022-05-09.","title":"Voteview: Congressional roll-call votes database","authors":"Jeffrey B Lewis, Keith Poole, Howard Rosenthal, Adam Boche, Aaron Rudkin, Luke Sonnet","year":"2021","venue":null,"doi":null,"url":"https://voteview.com","internal_url":null},{"position":11,"raw":"[Liang et al., 2017] Yitao Liang, Jessa Bekker, and Guy Van den Broeck. Learning the structure of probabilistic sentential decision diagrams. In Uncertainty in Artificial Intelligence, 2017.","title":"Learning the structure of probabilistic sentential decision diagrams","authors":"Liang","year":"2017","venue":"Uncertainty in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Molina et al., 2018] Alejandro Molina, Antonio Vergari, Nicola Di Mauro, Sriraam Natarajan, Floriana Esposito, and Kristian Kersting. Mixed sum-product networks: A deep architecture for hybrid domains. In Thirty-second AAAI conference on artificial intelligence, 2018.","title":"Mixed sum-product networks: A deep architecture for hybrid domains","authors":"Alejandro Molina, Antonio Vergari, Nicola Di Mauro, Sriraam Natarajan, Floriana Esposito, Kristian Kersting","year":"2018","venue":"Thirty-second AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Molina et al., 2019] Alejandro Molina, Antonio Ver- gari, Karl Stelzner, Robert Peharz, Pranav Subramani, Nicola Di Mauro, Pascal Poupart, and Kristian Ker- sting. Spflow: An easy and extensible library for deep probabilistic learning using sum-product network. https://github.com/SPFlow/SPFlow, 2019. Accessed: 2022-05-09.","title":"Spflow: An easy and extensible library for deep probabilistic learning using sum-product network","authors":"Alejandro Molina, Antonio Vergari, Karl Stelzner, Robert Peharz, Pranav Subramani, Nicola Di Mauro, Pascal Poupart, Kristian Kersting","year":"2019","venue":null,"doi":null,"url":"https://github.com/SPFlow/SPFlow","internal_url":null},{"position":14,"raw":"[Nath and Domingos, 2015] Aniruddh Nath and Pedro M Domingos. Learning relational sum-product networks. In Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015.","title":"Learning relational sum-product networks","authors":"Domingos Nath, Aniruddh Nath, Pedro M Domingos","year":"2015","venue":"Twenty-Ninth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Niepert and Domingos, 2014] Mathias Niepert and Pedro Domingos. Exchangeable variable models. In Interna- tional Conference on Machine Learning, 2014.","title":"Exchangeable variable models","authors":"Domingos Niepert, Mathias Niepert, Pedro Domingos","year":"2014","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Niepert and Van den Broeck, 2014] Mathias Niepert and Guy Van den Broeck. Tractability through exchangeabil- ity: A new perspective on efficient probabilistic inference. In Twenty-Eighth AAAI Conference on Artificial Intelli- gence, 2014.","title":"Tractability through exchangeability: A new perspective on efficient probabilistic inference","authors":"Van Niepert, Broeck Den, Mathias Niepert, Guy Van Den Broeck","year":"2014","venue":"Twenty-Eighth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Peharz et al., 2015] Robert Peharz, Sebastian Tschiatschek, Franz Pernkopf, and Pedro Domingos. On theoretical properties of sum-product networks. In Artificial Intelli- gence and Statistics, 2015.","title":"On theoretical properties of sum-product networks","authors":"Robert Peharz, Sebastian Tschiatschek, Franz Pernkopf, Pedro Domingos","year":"2015","venue":"Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Peharz et al., 2020] Robert Peharz, Antonio Vergari, Karl Stelzner, Alejandro Molina, Xiaoting Shao, Martin Trapp, Kristian Kersting, and Zoubin Ghahramani. Random sum- product networks: A simple and effective approach to probabilistic deep learning. In Uncertainty in Artificial In- telligence, 2020.","title":"Random sumproduct networks: A simple and effective approach to probabilistic deep learning","authors":"Robert Peharz, Antonio Vergari, Karl Stelzner, Alejandro Molina, Xiaoting Shao, Martin Trapp, Kristian Kersting, Zoubin Ghahramani","year":"2020","venue":"Uncertainty in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Poon and Domingos, 2011] Hoifung Poon and Pedro Domingos. Sum-product networks: a new deep ar- chitecture. In Uncertainty in Artificial Intelligence, 2011.","title":"Hoifung Poon and Pedro Domingos. Sum-product networks: a new deep architecture","authors":"Domingos Poon","year":"2011","venue":"Uncertainty in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rahman et al., 2014] Tahrima Rahman, Prasanna Kothalkar, and Vibhav Gogate. Cutset networks: A simple, tractable, and scalable approach for improving the accuracy of chow-liu trees. In Joint European con- ference on machine learning and knowledge discovery in databases, pages 630-645. Springer, 2014.","title":"Cutset networks: A simple, tractable, and scalable approach for improving the accuracy of chow-liu trees","authors":"Rahman","year":"2014","venue":"Joint European conference on machine learning and knowledge discovery in databases","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rezende and Mohamed, 2015] Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In International conference on machine learning, pages 1530-1538. PMLR, 2015.","title":"Variational inference with normalizing flows","authors":"Mohamed ; Rezende, Danilo Rezende, Shakir Mohamed","year":"2015","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Richardson and Domingos, 2006] Matthew Richardson and Pedro Domingos. Markov logic networks. Machine learn- ing, 62(1-2):107-136, 2006.","title":"Matthew Richardson and Pedro Domingos. Markov logic networks","authors":"Domingos Richardson","year":"2006","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Schervish, 1995] Mark J Schervish. Theory of statistics. Springer, 1995.","title":"Theory of statistics","authors":"J Mark, Schervish","year":"1995","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Trapp et al., 2019] Martin Trapp, Robert Peharz, Hong Ge, Franz Pernkopf, and Zoubin Ghahramani. Bayesian learn- ing of sum-product networks. In Advances in Neural In- formation Processing Systems, pages 6347-6358, 2019.","title":"Bayesian learning of sum-product networks","authors":"Martin Trapp, Robert Peharz, Hong Ge, Franz Pernkopf, Zoubin Ghahramani","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Vergari et al., 2015] Antonio Vergari, Nicola Di Mauro, and Floriana Esposito. Simplifying, regularizing and strength- ening sum-product network structure learning. In Joint Eu- ropean Conference on Machine Learning and Knowledge Discovery in Databases, pages 343-358. Springer, 2015.","title":"Simplifying, regularizing and strengthening sum-product network structure learning","authors":"Vergari","year":"2015","venue":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yang et al., 2020] Mengjiao Yang, Bo Dai, Hanjun Dai, and Dale Schuurmans. Energy-based processes for exchange- able data. In International Conference on Machine Learn- ing, pages 10681-10692. PMLR, 2020.","title":"Energy-based processes for exchangeable data","authors":"Yang","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}