{"type":"ScholarlyArticle","title":"Learning Relational Representations with Auto-encoding Logic Programs","authors":[{"name":"Sebastijan Dumancic","slug":"sebastijan-dumancic","given":"Sebastijan","family":"Dumancic","affiliation":"KU Leuven , Belgium"},{"name":"Tias Guns","slug":"tias-guns","given":"Tias","family":"Guns","affiliation":"VUB , Belgium"},{"name":"Wannes Meert","slug":"wannes-meert","given":"Wannes","family":"Meert","affiliation":"KU Leuven , Belgium"},{"name":"Hendrik Blockeel","slug":"hendrik-blockeel","given":"Hendrik","family":"Blockeel","affiliation":"KU Leuven , Belgium"}],"abstract":"Deep learning methods capable of handling relational data have proliferated over the past years. In contrast to traditional relational learning methods that leverage first-order logic for representing such data, these methods aim at re-representing symbolic relational data in Euclidean space. They offer better scalability, but can only approximate rich relational structures and are less flexible in terms of reasoning. This paper introduces a novel framework for relational representation learning that combines the best of both worlds. This framework, inspired by the auto-encoding principle, uses first-order logic as a data representation language, and the mapping between the the original and latent representation is done by means of logic programs instead of neural networks. We show how learning can be cast as a constraint optimisation problem for which existing solvers can be used. The use of logic as a representation language makes the proposed framework more accurate (as the representation is exact, rather than approximate), more flexible, and more interpretable than deep learning methods. We experimentally show that these latent representations are indeed beneficial in relational learning tasks.","doi":"10.24963/ijcai.2019/842","url":"https://ijcai.org/proceedings/2019/842","pdf":"https://ijcai.org/proceedings/2019/0842.pdf","markdown":"https://ijcai.org/proceedings/2019/842.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/842","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Understanding Intelligence and Human-level AI in the New Machine Learning era","track":"Understanding Intelligence and Human-level AI in the New Machine Learning era","number":842,"pages":{"first":6081,"last":6087},"keywords":["Special Track on Understanding Intelligence and Human-level AI in the New Machine Learning era: Learning knowledge representations (Special Track on Human AI and Machine Learning)","Special Track on Understanding Intelligence and Human-level AI in the New Machine Learning era: Knowledge representations for Learning (Special Track on Human AI and Machine Learning)"],"videos":[],"references":[{"position":1,"raw":"[Ahuja et al., 2002] Ravindra K. Ahuja, Özlem Ergun, James B. Orlin, and Abraham P. Punnen. A survey of very large-scale neighborhood search techniques. Discrete Appl. Math., 123(1-3):75-102, 2002.","title":"A survey of very large-scale neighborhood search techniques","authors":"K Ravindra, Özlem Ahuja, James B Ergun, Abraham P Orlin, Punnen","year":"2002","venue":"Discrete Appl. Math","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Campero et al., 2018] Andres Campero, Aldo Pareja, Tim Klinger, Josh Tenenbaum, and Sebastian Riedel. Logical rule induction and theory learning using neural theorem proving. CoRR, abs/1809.02193, 2018.","title":"Logical rule induction and theory learning using neural theorem proving","authors":"Andres Campero, Aldo Pareja, Tim Klinger, Josh Tenenbaum, Sebastian Riedel","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cropper and Muggleton, 2018] Andrew Cropper and Stephen H. Muggleton. Learning efficient logic programs. Machine Learning, 2018.","title":"Learning efficient logic programs","authors":"Andrew Cropper, Stephen H Muggleton","year":"2018","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Davis and Goadrich, 2006] Jesse Davis and Mark Goadrich. The relationship between precision-recall and roc curves. In ICML, pages 233-240, 2006.","title":"The relationship between precision-recall and roc curves","authors":"Goadrich Davis, Jesse Davis, Mark Goadrich","year":"2006","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Demeester et al., 2016] Thomas Demeester, Tim Rocktäschel, and Sebastian Riedel. Lifted rule in- jection for relation embeddings. In EMNLP, pages 1389-1399, 2016.","title":"Lifted rule injection for relation embeddings","authors":"Thomas Demeester, Tim Rocktäschel, Sebastian Riedel","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ellis et al., 2015] Kevin Ellis, Armando Solar-Lezama, and Joshua B. Tenenbaum. Unsupervised learning by program synthesis. In NIPS, pages 973-981, 2015.","title":"Unsupervised learning by program synthesis","authors":"Kevin Ellis, Armando Solar-Lezama, Joshua B Tenenbaum","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Evans and Grefenstette, 2018] Richard Evans and Edward Grefenstette. Learning explanatory rules from noisy data. J. Artif. Intell. Res., 61:1-64, 2018.","title":"Learning explanatory rules from noisy data","authors":"Grefenstette Evans, Richard Evans, Edward Grefenstette","year":"2018","venue":"J. Artif. Intell. Res","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gay et al., 2015] Steven Gay, Renaud Hartert, Christophe Lecoutre, and Pierre Schaus. Conflict ordering search for scheduling problems. In Principles and Practice of Con- straint Programming, pages 140-148. Springer, 2015.","title":"Conflict ordering search for scheduling problems","authors":"Steven Gay, Renaud Hartert, Christophe Lecoutre, Pierre Schaus","year":"2015","venue":"Principles and Practice of Constraint Programming","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Getoor and Taskar, 2007] Lise Getoor and Ben Taskar. Intro- duction to Statistical Relational Learning (Adaptive Com- putation and Machine Learning). The MIT Press, 2007.","title":"Introduction to Statistical Relational Learning (Adaptive Computation and Machine Learning","authors":"Lise Getoor, Ben Taskar","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Goodfellow et al., 2016] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016.","title":"Deep Learning","authors":"Ian Goodfellow, Yoshua Bengio, Aaron Courville","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gulwani et al., 2017] Sumit Gulwani, Oleksandr Polozov, and Rishabh Singh. Program synthesis. Foundations and Trends R in Programming Languages, 4(1-2):1-119, 2017.","title":"Program synthesis","authors":"Sumit Gulwani, Oleksandr Polozov, Rishabh Singh","year":"2017","venue":"Foundations and Trends R in Programming Languages","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hamilton et al., 2017] William L. Hamilton, Rex Ying, and Jure Leskovec. Representation learning on graphs: Meth- ods and applications. IEEE Data Eng. Bull., 40(3):52-74, 2017.","title":"Representation learning on graphs: Methods and applications","authors":"William L Hamilton, Rex Ying, Jure Leskovec","year":"2017","venue":"IEEE Data Eng. Bull","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hinton and Salakhutdinov, 2006] G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313(5786):504-507, 2006.","title":"Reducing the dimensionality of data with neural networks","authors":"G E Hinton, R R Salakhutdinov","year":"2006","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kazemi and Poole, 2017] Seyed Mehran Kazemi and David Poole. Relnn: A deep neural model for relational learning. In AAAI, 2017.","title":"Relnn: A deep neural model for relational learning","authors":"Seyed Mehran, Kazemi, David Poole","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas N Kipf, Max Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kok and Domingos, 2007] Stanley Kok and Pedro Domin- gos. Statistical predicate invention. In ICML, pages 433- 440, 2007.","title":"Statistical predicate invention","authors":"Domingos ; Kok, Pedro Kok, Domingos","year":"2007","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kok and Domingos, 2010] Stanley Kok and Pedro Domin- gos. Learning markov logic networks using structural mo- tifs. In ICML, pages 551-558, 2010.","title":"Learning markov logic networks using structural motifs","authors":"Domingos Kok, Stanley Kok, Pedro Domingos","year":"2010","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kramer, 1995] Stefan Kramer. Predicate Invention: A Com- prehensive View. Technical report, 1995.","title":"Predicate Invention: A Comprehensive View","authors":"Stefan Kramer","year":"1995","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lake et al., 2015] Brenden M. Lake, Ruslan Salakhutdinov, and Joshua B. Tenenbaum. Human-level concept learn- ing through probabilistic program induction. Science, 350:1332-1338, 2015.","title":"Human-level concept learning through probabilistic program induction","authors":"M Brenden, Ruslan Lake, Joshua B Salakhutdinov, Tenenbaum","year":"2015","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Manhaeve et al., 2018] Robin Manhaeve, Sebastijan Duman- cic, Angelika Kimmig, Thomas Demeester, and Luc De Raedt. Deepproblog: Neural probabilistic logic pro- gramming. In NeurIPS, pages 3749-3759. 2018.","title":"Deepproblog: Neural probabilistic logic programming","authors":"Manhaeve","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mihalkova and Mooney, 2007] Lilyana Mihalkova and Ray- mond J. Mooney. Bottom-up learning of markov logic network structure. In ICML, 2007.","title":"Lilyana Mihalkova and Raymond J. Mooney. Bottom-up learning of markov logic network structure","authors":"Mooney Mihalkova","year":"2007","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Minervini et al., 2017] Pasquale Minervini, Thomas De- meester, Tim Rocktäschel, and Sebastian Riedel. Adver- sarial sets for regularising neural link predictors. In UAI, 2017.","title":"Adversarial sets for regularising neural link predictors","authors":"Pasquale Minervini, Thomas Demeester, Tim Rocktäschel, Sebastian Riedel","year":"2017","venue":"UAI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Nickel et al., 2016] M. Nickel, K. Murphy, V. Tresp, and E. Gabrilovich. A review of relational machine learning for knowledge graphs. Proceedings of the IEEE, 104(1):11-33, 2016.","title":"A review of relational machine learning for knowledge graphs","authors":"M Nickel, K Murphy, V Tresp, E Gabrilovich","year":"2016","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Richardson and Domingos, 2006] Matthew Richardson and Pedro Domingos. Markov logic networks. Mach. Learn., 62(1-2):107-136, 2006.","title":"Markov logic networks","authors":"Matthew Richardson, Pedro Domingos","year":"2006","venue":"Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Rocktäschel and Riedel, 2017] Tim Rocktäschel and Sebas- tian Riedel. End-to-end differentiable proving. In NIPS, pages 3788-3800.","title":"Tim Rocktäschel and Sebastian Riedel. End-to-end differentiable proving","authors":"Riedel Rocktäschel","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"Curran Associates, Inc., 2017.","title":null,"authors":"Curran Associates, Inc","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Rossi et al., 2006] Francesca Rossi, Peter van Beek, and Toby Walsh. Handbook of Constraint Programming (Foun- dations of Artificial Intelligence). Elsevier Science Inc., 2006.","title":"Handbook of Constraint Programming (Foundations of Artificial Intelligence","authors":"Francesca Rossi, Peter Van Beek, Toby Walsh","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Sourek et al., 2016] Gustav Sourek, Suresh Manandhar, Filip Zelezny, Steven Schockaert, and Ondrej Kuzelka. Learning Predictive Categories Using Lifted Relational Neural Networks, volume 10326 of LNAI. Springer Inter- national Publishing, 2016.","title":"Learning Predictive Categories Using Lifted Relational Neural Networks","authors":"Sourek","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Trouillon et al., 2019] Théo Trouillon, Eric Gaussier, Christopher R. Dance, and Guillaume Bouchard. On inductive abilities of latent factor models for relational learning. Journal of Artificial Intelligence Research, 64, 2019.","title":"On inductive abilities of latent factor models for relational learning","authors":"Eric Théo Trouillon, Christopher R Gaussier, Guillaume Dance, Bouchard","year":"2019","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Van Haaren et al., 2016] Jan Van Haaren, Guy Van den Broeck, Wannes Meert, and Jesse Davis. Lifted generative learning of markov logic networks. Machine Learning, 103(1):27-55, 2016.","title":"Lifted generative learning of markov logic networks","authors":"Van Haaren","year":"2016","venue":"Machine Learning","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}