{"type":"ScholarlyArticle","title":"Locally Consistent Bayesian Network Scores for Multi-Relational Data","authors":[{"name":"Oliver Schulte","slug":"oliver-schulte","given":"Oliver","family":"Schulte","affiliation":"University , Burnaby , Canada"},{"name":"Sajjad Gholami","slug":"sajjad-gholami","given":"Sajjad","family":"Gholami","affiliation":"University , Burnaby , Canada"}],"abstract":"An important task for relational learning is Bayesian network (BN) structure learning. A fundamental component of structure learning is a model selection score that measures how well a model fits a dataset. We describe a new method that upgrades for multi-relational databases, a log-linear BN score designed for single-table i.i.d. data. Chickering and Meek showed that for i.i.d. data, standard BN scores are locally consistent, meaning that their maxima converge to an optimal model, that represents the data generating distribution {\\em and} contains no redundant edges. Our main theorem establishes that if a model selection score is locally consistent for i.i.d. data, then our upgraded gain function is locally consistent for relational data as well. To our knowledge this is the first consistency result for relational structure learning. A novel aspect of our approach is employing a {\\em gain function} that compares two models: a current vs. an alternative BN structure. In contrast, previous approaches employed a score that is a function of a single model only. Empirical evaluation on six benchmark relational databases shows that our gain function is also practically useful: On realistic size data sets, it selects informative BN structures  with a better data fit than those selected by baseline single-model scores.","doi":"10.24963/ijcai.2017/375","url":"https://ijcai.org/proceedings/2017/375","pdf":"https://ijcai.org/proceedings/2017/0375.pdf","markdown":"https://ijcai.org/proceedings/2017/375.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/375","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":375,"pages":{"first":2693,"last":2700},"keywords":["Machine Learning: Relational Learning","Machine Learning: Learning Graphical Models"],"videos":[],"references":[{"position":1,"raw":"[Bouckaert, 1995] R. Bouckaert. Bayesian belief networks: from construction to inference. PhD thesis, U. Utrecht, 1995.","title":"Bayesian belief networks: from construction to inference","authors":"R Bouckaert","year":"1995","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Buchman and Poole, 2015] David Buchman and David Poole. Representing aggregators in relational probabilistic models. In AAAI, pages 3489-3495, 2015.","title":"Representing aggregators in relational probabilistic models","authors":"Poole ; ] Buchman, David David Buchman, Poole","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chickering and Meek, 2002] David Maxwell Chickering and Christopher Meek. Finding optimal Bayesian net- works. In UAI, pages 94-102, 2002.","title":"Finding optimal Bayesian networks","authors":"Chickering, Meek, David Maxwell, Chickering, Christopher Meek","year":"2002","venue":"UAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Domingos and Richardson, 2007] Pedro Domingos and Matthew Richardson. Markov logic: A unifying frame- work for statistical relational learning. In Introduction to Statistical Relational Learning [2007].","title":"Pedro Domingos and Matthew Richardson. Markov logic: A unifying framework for statistical relational learning","authors":"Richardson Domingos","year":"2007","venue":"Introduction to Statistical Relational Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Frank, 1978] O. Frank. Estimation of the number of con- nected components in a graph by using a sampled sub- graph. Scandinavian Journal of Statistics, pages 177-188, 1978.","title":"Estimation of the number of connected components in a graph by using a sampled subgraph","authors":"O Frank","year":"1978","venue":"Scandinavian Journal of Statistics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Friedman et al., 1999] Nir Friedman, Lise Getoor, Daphne Koller, and Avi Pfeffer. Learning probabilistic relational models. In IJCAI, volume 99, pages 1300-1309, 1999.","title":"Learning probabilistic relational models","authors":"Nir Friedman, Lise Getoor, Daphne Koller, Avi Pfeffer","year":"1999","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/1999/185"},{"position":7,"raw":"[Getoor and Taskar, 2007] Lise Getoor and Ben Taskar. In- troduction to Statistical Relational Learning. MIT Press, 2007.","title":"Introduction to Statistical Relational Learning","authors":"Lise Getoor, Ben Taskar","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Getoor et al., 2001a] Lise Getoor, Benjamin Taskar, and Daphne Koller. Selectivity estimation using probabilistic models. ACM SIGMOD Record, 30(2):461-472, 2001.","title":"Selectivity estimation using probabilistic models","authors":"Getoor","year":"2001","venue":"ACM SIGMOD Record","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Getoor et al., 2001b] Lise Getoor Getoor, Nir Friedman, and Benjamin Taskar. Learning probabilistic models of re- lational structure. In ICML, pages 170-177.","title":"Learning probabilistic models of relational structure","authors":"Getoor","year":"2001","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"Morgan Kauf- mann, 2001.","title":null,"authors":"Morgan Kaufmann","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Getoor, 2001] Lise Getoor. Learning Statistical Models From Relational Data. PhD thesis, Department of Com- puter Science, Stanford University, 2001.","title":"Learning Statistical Models From Relational Data","authors":"Lise Getoor","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gholami, 2016] Sajjad Gholami. Upgrading Bayesian net- work scores for multi-relational data. Master's thesis, School of Computing Science, Simon Fraser University, 2016.","title":"Upgrading Bayesian network scores for multi-relational data","authors":"Sajjad Gholami","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Halpern, 1990] Joseph Y. Halpern. An analysis of first-order logics of probability. Artificial Intelligence, 46(3):311- 350, 1990.","title":"An analysis of first-order logics of probability","authors":"Y Joseph, Halpern","year":"1990","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/1989/220"},{"position":14,"raw":"[Kimmig et al., 2014] Angelika Kimmig, Lilyana Mi- halkova, and Lise Getoor. Lifted graphical models: a survey. Machine Learning, pages 1-45, 2014.","title":"Lifted graphical models: a survey","authors":"Angelika Kimmig, Lilyana Mihalkova, Lise Getoor","year":"2014","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Laer and de Raedt, 2001] Wim Van Laer and Luc de Raedt. How to upgrade propositional learners to first-order logic: A case study. In Relational Data Mining. Springer Verlag, 2001.","title":"How to upgrade propositional learners to first-order logic: A case study","authors":"De Laer, ; Raedt, Van Wim, Luc Laer, Raedt De","year":"2001","venue":"Relational Data Mining","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lowd and Domingos, 2007] Daniel Lowd and Pedro Domingos. Efficient weight learning for Markov logic networks. In PKDD, pages 200-211, 2007.","title":"Efficient weight learning for Markov logic networks","authors":"Domingos Lowd, Daniel Lowd, Pedro Domingos","year":"2007","venue":"PKDD","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Motl and Schulte, 2015] Jan Motl and Oliver Schulte. The CTU prague relational learning repository. CoRR, abs/1511.03086, 2015.","title":"The CTU prague relational learning repository","authors":"Schulte ; Motl Motl, Oliver Schulte","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nickel et al., 2016] Maximilian Nickel, Kevin Murphy, Volker Tresp, and Evgeniy Gabrilovich. A review of re- lational machine learning for knowledge graphs. Proceed- ings of the IEEE, 104(1):11-33, 2016.","title":"A review of relational machine learning for knowledge graphs","authors":"Nickel","year":"2016","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Niu et al., 2011] Feng Niu, Christopher Ré, AnHai Doan, and Jude W. Shavlik. Tuffy: Scaling up statistical infer- ence in markov logic networks using an rdbms. PVLDB, 4(6):373-384, 2011.","title":"Tuffy: Scaling up statistical inference in markov logic networks using an rdbms","authors":"Feng Niu, Christopher Ré, Anhai Doan, Jude W Shavlik","year":"2011","venue":"PVLDB","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pearl, 1988] J. Pearl. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, 1988.","title":"Probabilistic Reasoning in Intelligent Systems","authors":"J Pearl","year":"1988","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Poole, 2003] David Poole. First-order probabilistic infer- ence. In IJCAI, 2003.","title":"First-order probabilistic inference","authors":"David Poole","year":"2003","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2003/141"},{"position":22,"raw":"[Quinlan and Cameron-Jones, 1993] J. Ross Quinlan and R. Mike Cameron-Jones. Foil: A midterm report. In ECML, volume 667, pages 3-20. Springer, 1993.","title":"Foil: A midterm report","authors":"J, Ross Quinlan, R, Mike Cameron-Jones","year":"1993","venue":"ECML","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Russell and Norvig, 2010] Stuart Russell and Peter Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall, 2010.","title":"Artificial Intelligence: A Modern Approach","authors":"Stuart Russell, Peter Norvig","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Russell, 2015] Stuart Russell. Unifying logic and probabil- ity. Communications of the ACM, 58(7):88-97, 2015.","title":"Unifying logic and probability","authors":"Stuart Russell","year":"2015","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sakai and Yamanishi, 2013] Yoshiki Sakai and Kenji Ya- manishi. An NML-based model selection criterion for general relational data modeling. In Big Data, pages 421- 429. IEEE, 2013.","title":"An NML-based model selection criterion for general relational data modeling","authors":"Yamanishi Sakai, Yoshiki Sakai, Kenji Yamanishi","year":"2013","venue":"Big Data","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Schulte and Gholami, 2016] Oliver Schulte and Sajjad Gho- lami. Consistent bayesian network scores for multi- relational data. StarAI-IJCAI Workshop, July 2016.","title":"Consistent bayesian network scores for multirelational data","authors":"Gholami Schulte, Oliver Schulte, Sajjad Gholami","year":"2016","venue":"StarAI-IJCAI Workshop","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Schulte and Khosravi, 2012] Oliver Schulte and Hassan Khosravi. Learning graphical models for relational data via lattice search. Machine Learning, 88(3):331-368, 2012.","title":"Learning graphical models for relational data via lattice search","authors":"Khosravi Schulte, Oliver Schulte, Hassan Khosravi","year":"2012","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Schulte et al., 2014] Oliver Schulte, Hassan Khosravi, Arthur Kirkpatrick, Tianxiang Gao, and Yuke Zhu. Modelling relational statistics with Bayes nets. Machine Learning, 94:105-125, 2014.","title":"Modelling relational statistics with Bayes nets","authors":"Oliver Schulte, Hassan Khosravi, Arthur Kirkpatrick, Tianxiang Gao, Yuke Zhu","year":"2014","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Schulte, 2011] Oliver Schulte. A tractable pseudo- likelihood function for Bayes nets applied to relational data. In SIAM SDM, pages 462-473, 2011.","title":"A tractable pseudolikelihood function for Bayes nets applied to relational data","authors":"Oliver Schulte","year":"2011","venue":"SIAM SDM","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Shalizi and Rinaldo, 2013] Cosma Rohilla Shalizi and Alessandro Rinaldo. Consistency under sampling of exponential random graph models. Annals of Statistics, 41(2):508, 2013.","title":"Consistency under sampling of exponential random graph models","authors":"Rohilla Cosma, Alessandro Shalizi, Rinaldo","year":"2013","venue":"Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Van Haaren et al., 2016] Jan Van Haaren, Guy Van den Broeck, Wannes Meert, and Jesse Davis. Lifted generative learning of Markov logic networks. Machine Learning, 103(1):27-55, 2016.","title":"Lifted generative learning of Markov logic networks","authors":"Van Haaren","year":"2016","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wang et al., 2008] Daisy Zhe Wang, Eirinaios Michelakis, Minos Garofalakis, and Joseph M Hellerstein. Bayesstore: managing large, uncertain data repositories with proba- bilistic graphical models. In Proceedings VLDB, pages 340-351. VLDB Endowment, 2008.","title":"Bayesstore: managing large, uncertain data repositories with probabilistic graphical models","authors":"Daisy Zhe, Wang, Eirinaios Michelakis, Minos Garofalakis, Joseph M Hellerstein","year":"2008","venue":"Proceedings VLDB","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Xiang and Neville, 2011] Rongjing Xiang and Jennifer Neville. Relational learning with one network: An asymp- totic analysis. In AISTATS, pages 779-788, 2011.","title":"Rongjing Xiang and Jennifer Neville. Relational learning with one network: An asymptotic analysis","authors":"Neville Xiang","year":"2011","venue":"AISTATS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}