{"type":"ScholarlyArticle","title":"Learning First-Order Rules with Differentiable Logic Program Semantics","authors":[{"name":"Kun Gao","slug":"kun-gao","given":"Kun","family":"Gao","affiliation":"Key Laboratory of High Confidence Software Technologies (MOE) , School of Computer Science , Peking University"},{"name":"Katsumi Inoue","slug":"katsumi-inoue","given":"Katsumi","family":"Inoue","affiliation":"National Institute of Informatics"},{"name":"Yongzhi Cao","slug":"yongzhi-cao","given":"Yongzhi","family":"Cao","affiliation":"Key Laboratory of High Confidence Software Technologies (MOE) , School of Computer Science , Peking University"},{"name":"Hanpin Wang","slug":"hanpin-wang","given":"Hanpin","family":"Wang","affiliation":"School of Computer Science and Cyber Engineering , Guangzhou University; Key Laboratory of High Confidence Software Technologies (MOE) , School of Computer Science , Peking University"}],"abstract":"Learning first-order logic programs (LPs) from relational facts which yields intuitive insights into the data is a challenging topic in neuro-symbolic research. We introduce a novel differentiable inductive logic programming (ILP) model, called differentiable first-order rule learner (DFOL), which finds the correct LPs from relational facts by searching for the interpretable matrix representations of LPs. These interpretable matrices are deemed as trainable tensors in neural networks (NNs). The NNs are devised according to the differentiable semantics of LPs. Specifically, we first adopt a novel propositionalization method that transfers facts to NN-readable vector pairs representing interpretation pairs. We replace the immediate consequence operator with NN constraint functions consisting of algebraic operations and a sigmoid-like activation function. We map the symbolic forward-chained format of LPs into NN constraint functions consisting of operations between subsymbolic vector representations of atoms. By applying gradient descent, the trained well parameters of NNs can be decoded into precise symbolic LPs in forward-chained logic format. We demonstrate that DFOL can perform on several standard ILP datasets, knowledge bases, and probabilistic relation facts and outperform several well-known differentiable ILP models. Experimental results indicate that DFOL is a precise, robust, scalable, and computationally cheap differentiable ILP model.","doi":"10.24963/ijcai.2022/417","url":"https://ijcai.org/proceedings/2022/417","pdf":"https://ijcai.org/proceedings/2022/0417.pdf","markdown":"https://ijcai.org/proceedings/2022/417.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/417","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":417,"pages":{"first":3008,"last":3014},"keywords":["Machine Learning: Relational Learning","Machine Learning: Neuro-Symbolic Methods","Knowledge Representation and Reasoning: Learning and reasoning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987083","length_sec":86,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985226","length_sec":718,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In NeurIPS, pages 2787-2795, 2013.","title":"Translating embeddings for modeling multi-relational data","authors":"Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, Oksana Yakhnenko","year":"2013","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bouchard et al., 2015] Guillaume Bouchard, Sameer Singh, and Théo Trouillon. On approximate reasoning capabili- ties of low-rank vector spaces. In AAAI, 2015.","title":"On approximate reasoning capabilities of low-rank vector spaces","authors":"Guillaume Bouchard, Sameer Singh, Théo Trouillon","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cropper et al., 2020] Andrew Cropper, Sebastijan Duman- cic, and Stephen H. Muggleton. Turning 30: New ideas in inductive logic programming. In IJCAI, pages 4833-4839, 2020.","title":"Turning 30: New ideas in inductive logic programming","authors":"Andrew Cropper, Sebastijan Dumancic, Stephen H Muggleton","year":"2020","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/673"},{"position":4,"raw":"['Avila Garcez and Zaverucha, 1999] Artur S. d'Avila Garcez and Gerson Zaverucha. The connectionist induc- tive learning and logic programming system. Appl. Intell., 11(1):59-77, 1999.","title":"The connectionist inductive learning and logic programming system","authors":"Artur S, Avila Garcez, Gerson Zaverucha","year":"1999","venue":"Appl. Intell","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"['Avila Garcez et al., 2001] Artur S. d'Avila Garcez, Krysia Broda, and Dov M. Gabbay. Symbolic knowl- edge extraction from trained neural networks: A sound approach. Artif. Intell., 125(1-2):155-207, 2001.","title":"Symbolic knowledge extraction from trained neural networks: A sound approach","authors":"Artur S Garcez, Krysia Broda, Dov M Gabbay","year":"2001","venue":"Artif. Intell","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dong et al., 2019] Honghua Dong, Jiayuan Mao, Tian Lin, Chong Wang, Lihong Li, and Denny Zhou. Neural logic machines. In ICLR, 2019.","title":"Neural logic machines","authors":"Honghua Dong, Jiayuan Mao, Tian Lin, Chong Wang, Lihong Li, Denny Zhou","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Evans and Grefenstette, 2018] Richard Evans and Edward Grefenstette. Learning explanatory rules from noisy data. J. Artif. Intell. Res., 61:1-64, 2018.","title":"Learning explanatory rules from noisy data","authors":"Grefenstette Evans, Richard Evans, Edward Grefenstette","year":"2018","venue":"J. Artif. Intell. Res","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Evans et al., 2021] Richard Evans, José Hernández-Orallo, Johannes Welbl, Pushmeet Kohli, and Marek J. Sergot. Making sense of sensory input. Artif. Intell., 293:103438, 2021.","title":"Making sense of sensory input","authors":"Richard Evans, José Hernández-Orallo, Johannes Welbl, Pushmeet Kohli, Marek J Sergot","year":"2021","venue":"Artif. Intell","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Franc ¸a et al., 2014] Manoel V M Franc ¸a, Gerson Za- verucha, and Artur S d'Avila Garcez. Fast relational learn- ing using bottom clause propositionalization with artificial neural networks. Mach. Learn., 94(1):81-104, 2014.","title":"Fast relational learning using bottom clause propositionalization with artificial neural networks","authors":"V M Franc Manoel, Gerson ¸a, Artur S D'avila Zaverucha, Garcez","year":"2014","venue":"Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gao et al., 2021] Kun Gao, Hanpin Wang, Yongzhi Cao, and Katsumi Inoue. Learning from interpretation tran- sition using differentiable logic programming semantics. Mach. Learn., 2021.","title":"Learning from interpretation transition using differentiable logic programming semantics","authors":"Kun Gao, Hanpin Wang, Yongzhi Cao, Katsumi Inoue","year":"2021","venue":"Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hájek, 1998] Petr Hájek. Metamathematics of fuzzy logic, volume 4 of Trends in Logic. Kluwer, 1998.","title":"Metamathematics of fuzzy logic","authors":"Petr Hájek, Hájek","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hohenecker and Lukasiewicz, 2020] Patrick Hohenecker and Thomas Lukasiewicz. Ontology reasoning with deep neural networks. J. Artif. Intell. Res., 68:503-540, 2020.","title":"Ontology reasoning with deep neural networks","authors":"Lukasiewicz Hohenecker, Patrick Hohenecker, Thomas Lukasiewicz","year":"2020","venue":"J. Artif. Intell. Res","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Inoue et al., 2014] Katsumi Inoue, Tony Ribeiro, and Chi- aki Sakama. Learning from interpretation transition. Mach. Learn., 94(1):51-79, 2014.","title":"Learning from interpretation transition","authors":"Katsumi Inoue, Tony Ribeiro, Chiaki Sakama","year":"2014","venue":"Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kaminski et al., 2018] Tobias Kaminski, Thomas Eiter, and Katsumi Inoue. Exploiting answer set programming with external sources for meta-interpretive learning. Theory Pract. Log. Program., 18(3-4):571-588, 2018.","title":"Exploiting answer set programming with external sources for meta-interpretive learning","authors":"Kaminski","year":"2018","venue":"Theory Pract. Log. Program","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kaur et al., 2019] Navdeep Kaur, Gautam Kunapuli, Saket Joshi, Kristian Kersting, and Sriraam Natarajan. Neural networks for relational data. In ILP, volume 11770, pages 62-71, 2019.","title":"Neural networks for relational data","authors":"Navdeep Kaur, Gautam Kunapuli, Saket Joshi, Kristian Kersting, Sriraam Natarajan","year":"2019","venue":"ILP","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kok and Domingos, 2007] Stanley Kok and Pedro M. Domingos. Statistical predicate invention. In ICML, vol- ume 227, pages 433-440, 2007.","title":"Statistical predicate invention","authors":"Domingos Kok, Stanley Kok, Pedro M Domingos","year":"2007","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kramer et al., 2001] Stefan Kramer, Nada Lavrač, and Pe- ter Flach. Propositionalization approaches to relational data mining, pages 262-291. Springer, Berlin: Heidel- berg, 2001.","title":"Propositionalization approaches to relational data mining","authors":"Stefan Kramer, Nada Lavrač, Peter Flach","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lehmann et al., 2010] Jens Lehmann, Sebastian Bader, and Pascal Hitzler. Extracting reduced logic programs from artificial neural networks. Appl. Intell., 32(3):249-266, 2010.","title":"Extracting reduced logic programs from artificial neural networks","authors":"Lehmann","year":"2010","venue":"Appl. Intell","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Muggleton et al., 2012] Stephen Muggleton, Luc De Raedt, David Poole, Ivan Bratko, Peter Flach, Katsumi Inoue, and Ashwin Srinivasan. ILP turns 20. Mach. Learn., 86(1):3- 23, 2012.","title":"ILP turns 20","authors":"Stephen Muggleton, Luc De Raedt, David Poole, Ivan Bratko, Peter Flach, Katsumi Inoue, Ashwin Srinivasan","year":"2012","venue":"Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Muggleton et al., 2015] Stephen H Muggleton, Dianhuan Lin, and Alireza Tamaddoni-Nezhad. Meta-interpretive learning of higher-order dyadic datalog: Predicate inven- tion revisited. Mach. Learn., 100(1):49-73, 2015.","title":"Meta-interpretive learning of higher-order dyadic datalog: Predicate invention revisited","authors":"Dianhuan Stephen H Muggleton, Alireza Lin, Tamaddoni-Nezhad","year":"2015","venue":"Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Muggleton, 1991] Stephen Muggleton. Inductive logic pro- gramming. New Gener. Comput., 8(4):295-318, 1991.","title":"Inductive logic programming","authors":"Stephen Muggleton","year":"1991","venue":"New Gener. Comput","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Qu et al., 2021] Meng Qu, Junkun Chen, Louis-Pascal Xhonneux, Yoshua Bengio, and Jian Tang. RNNLogic: Learning logic rules for reasoning on knowledge graphs. In ICLR, 2021.","title":"RNNLogic: Learning logic rules for reasoning on knowledge graphs","authors":"Meng Qu, Junkun Chen, Louis-Pascal Xhonneux, Yoshua Bengio, Jian Tang","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Rocktäschel and Riedel, 2017] Tim Rocktäschel and Sebas- tian Riedel. End-to-end differentiable proving. In NeurIPS, pages 3788-3800, 2017.","title":"Tim Rocktäschel and Sebastian Riedel. End-to-end differentiable proving","authors":"Riedel Rocktäschel","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sakama et al., 2021] Chiaki Sakama, Katsumi Inoue, and Taisuke Sato. Logic programming in tensor spaces. Ann. Math. Artif. Intell., 89(12):1133-1153, 2021.","title":"Logic programming in tensor spaces","authors":"Chiaki Sakama, Katsumi Inoue, Taisuke Sato","year":"2021","venue":"Ann. Math. Artif. Intell","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Serafini and d'Avila Garcez, 2016] Luciano Serafini and Artur S. d'Avila Garcez. Logic tensor networks: deep learning and logical reasoning from data and knowledge. In NeSy@HLAI, volume 1768, 2016.","title":"Logic tensor networks: deep learning and logical reasoning from data and knowledge","authors":"D'avila Serafini, Luciano Garcez, Artur S Serafini, Garcez","year":"2016","venue":"NeSy@HLAI","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Sourek et al., 2018] Gustav Sourek, Vojtech Aschenbren- ner, Filip Zelezný, Steven Schockaert, and Ondrej Kuzelka. Lifted relational neural networks: Efficient learning of latent relational structures. J. Artif. Intell. Res., 62:69-100, 2018.","title":"Lifted relational neural networks: Efficient learning of latent relational structures","authors":"Sourek","year":"2018","venue":"J. Artif. Intell. Res","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Teru et al., 2020] Komal Teru, Etienne Denis, and Will Hamilton. Inductive relation prediction by subgraph rea- soning. In ICML, pages 9448-9457, 2020.","title":"Inductive relation prediction by subgraph reasoning","authors":"Komal Teru, Etienne Denis, Will Hamilton","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Van Emden and Kowalski, 1976] M. H. Van Emden and R. A. Kowalski. The semantics of predicate logic as a pro- gramming language. J. ACM, 23(4):733-742, 1976.","title":"The semantics of predicate logic as a programming language","authors":"M H Van Emden, R A Kowalski","year":"1976","venue":"J. ACM","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yang et al., 2017] Fan Yang, Zhilin Yang, and William W. Cohen. Differentiable learning of logical rules for knowl- edge base reasoning. In NeurIPS, pages 2319-2328, 2017.","title":"Differentiable learning of logical rules for knowledge base reasoning","authors":"Fan Yang, Zhilin Yang, William W Cohen","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}