{"type":"ScholarlyArticle","title":"Learning Explanatory Rules from Noisy Data (Extended Abstract)","authors":[{"name":"Richard Evans","slug":"richard-evans","given":"Richard","family":"Evans","affiliation":""},{"name":"Edward Grefenstette","slug":"edward-grefenstette","given":"Edward","family":"Grefenstette","affiliation":""}],"abstract":"Artificial Neural Networks are powerful function\r\napproximators capable of modelling solutions to\r\na wide variety of problems, both supervised and\r\nunsupervised. As their size and expressivity increases,\r\nso too does the variance of the model,\r\nyielding a nearly ubiquitous overfitting problem.\r\nAlthough mitigated by a variety of model regularisation\r\nmethods, the common cure is to seek\r\nlarge amounts of training data—which is not necessarily\r\neasily obtained—that sufficiently approximates\r\nthe data distribution of the domain we wish\r\nto test on. In contrast, logic programming methods\r\nsuch as Inductive Logic Programming offer an extremely\r\ndata-efficient process by which models can\r\nbe trained to reason on symbolic domains. However,\r\nthese methods are unable to deal with the variety\r\nof domains neural networks can be applied to:\r\nthey are not robust to noise in or mislabelling of inputs,\r\nand perhaps more importantly, cannot be applied\r\nto non-symbolic domains where the data is\r\nambiguous, such as operating on raw pixels. In this\r\npaper, we propose a Differentiable Inductive Logic\r\nframework (∂ILP), which can not only solve tasks\r\nwhich traditional ILP systems are suited for, but\r\nshows a robustness to noise and error in the training\r\ndata which ILP cannot cope with. Furthermore,\r\nas it is trained by backpropagation against a likelihood\r\nobjective, it can be hybridised by connecting\r\nit with neural networks over ambiguous data in\r\norder to be applied to domains which ILP cannot\r\naddress, while providing data efficiency and generalisation\r\nbeyond what neural networks on their own\r\ncan achieve.","doi":"10.24963/ijcai.2018/792","url":"https://ijcai.org/proceedings/2018/792","pdf":"https://ijcai.org/proceedings/2018/0792.pdf","markdown":"https://ijcai.org/proceedings/2018/792.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/792","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Journal track","track":"Journal track","number":792,"pages":{"first":5598,"last":5602},"keywords":["Machine Learning: Interpretability","Machine Learning: Neural Networks","Machine Learning: Relational Learning"],"videos":[],"references":[{"position":1,"raw":"[Abadi et al., 2016] Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al. Tensorflow: Large-scale machine learn- ing on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467, 2016.","title":"Tensorflow: Large-scale machine learning on heterogeneous distributed systems","authors":"Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Andrychowicz and Kurach, 2016] Marcin Andrychow- icz and Karol Kurach. Learning efficient algorithms with hierarchical attentive memory. arXiv preprint arXiv:1602.03218, 2016.","title":"Learning efficient algorithms with hierarchical attentive memory","authors":"Marcin Andrychowicz, Karol Kurach","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bader et al., 2008] Sebastian Bader, Pascal Hitzler, and Steffen Hölldobler. Connectionist model generation: A first-order approach. Neurocomputing, 71(13):2420-2432, 2008.","title":"Connectionist model generation: A first-order approach","authors":"Sebastian Bader, Pascal Hitzler, Steffen Hölldobler","year":"2008","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Corapi et al., 2010] Domenico Corapi, Alessandra Russo, and Emil Lupu. Inductive logic programming as abductive search. In LIPIcs-Leibniz International Proceedings in In- formatics, volume 7. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2010.","title":"Inductive logic programming as abductive search","authors":"Domenico Corapi, Alessandra Russo, Emil Lupu","year":"2010","venue":"LIPIcs-Leibniz International Proceedings in Informatics","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Corapi et al., 2011] Domenico Corapi, Alessandra Russo, and Emil Lupu. Inductive logic programming in answer set programming. In International Conference on Induc- tive Logic Programming, pages 91-97. Springer, 2011.","title":"Inductive logic programming in answer set programming","authors":"Domenico Corapi, Alessandra Russo, Emil Lupu","year":"2011","venue":"International Conference on Inductive Logic Programming","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cropper and Muggleton, 2016] Andrew Cropper and Stephen H Muggleton. Learning higher-order logic pro- grams through abstraction and invention. In Proceedings of the 25th International Joint Conference Artificial Intelligence (IJCAI 2016). IJCAI, 2016.","title":"Learning higher-order logic programs through abstraction and invention","authors":"Andrew Cropper, Stephen H Muggleton","year":"2016","venue":"Proceedings of the 25th International Joint Conference Artificial Intelligence (IJCAI 2016). IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/204"},{"position":7,"raw":"[Cropper et al., 2015] Andrew Cropper, Alireza Tamaddoni- Nezhad, and Stephen H Muggleton. Meta-interpretive learning of data transformation programs. In International Conference on Inductive Logic Programming, pages 46- 59. Springer, 2015.","title":"Meta-interpretive learning of data transformation programs","authors":"Andrew Cropper, Alireza Tamaddoni-Nezhad, Stephen H Muggleton","year":"2015","venue":"International Conference on Inductive Logic Programming","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[De Raedt and Kersting, 2008] Luc De Raedt and Kristian Kersting. Probabilistic inductive logic programming. In Probabilistic Inductive Logic Programming, pages 1-27. Springer, 2008.","title":"Probabilistic inductive logic programming","authors":"De Raedt, Kersting ; Luc De, Raedt, Kristian Kersting","year":"2008","venue":"Probabilistic Inductive Logic Programming","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[De Raedt et al., 2016] Luc De Raedt, Kristian Kersting, Sri- raam Natarajan, and David Poole. Statistical relational ar- tificial intelligence: Logic, probability, and computation. Synthesis Lectures on Artificial Intelligence and Machine Learning, 10(2):1-189, 2016.","title":"Statistical relational artificial intelligence: Logic, probability, and computation","authors":"De Raedt","year":"2016","venue":"Synthesis Lectures on Artificial Intelligence and Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Evans and Grefenstette, 2018] Richard Evans and Edward Grefenstette. Learning explanatory rules from noisy data. Journal of Artificial Intelligence Research, 61:1-64, 2018.","title":"Learning explanatory rules from noisy data","authors":"Grefenstette Evans, Richard Evans, Edward Grefenstette","year":"2018","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Feser et al., 2015] John K Feser, Swarat Chaudhuri, and Isil Dillig. Synthesizing data structure transformations from input-output examples. In ACM SIGPLAN Notices, vol- ume 50, pages 229-239. ACM, 2015.","title":"Synthesizing data structure transformations from input-output examples","authors":"Swarat John K Feser, Isil Chaudhuri, Dillig","year":"2015","venue":"ACM SIGPLAN Notices","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Garcez et al., 2012] Artur S d'Avila Garcez, Krysia Broda, and Dov M Gabbay. Neural-symbolic learning systems: foundations and applications. Springer Science & Busi- ness Media, 2012.","title":"Neural-symbolic learning systems: foundations and applications","authors":"Artur S D'avila, Krysia Garcez, Dov M Broda, Gabbay","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Garcez et al., 2014] Artur S d'Avila Garcez, Dov M Gab- bay, and Luis C Lamb. A neural cognitive model of argu- mentation with application to legal inference and decision making. Journal of Applied Logic, 12(2):109-127, 2014.","title":"A neural cognitive model of argumentation with application to legal inference and decision making","authors":"Garcez","year":"2014","venue":"Journal of Applied Logic","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Garcez et al., 2015] Garcez, Besold, de Raedt, Foldiak, Hit- zler, Icard, Kuhnberger, Lamb, Miikkulainen, and Sil- ver. Neural-symbolic learning and reasoning: contribu- tions and challenges. Proceedings of the AAAI Spring Symposium on Knowledge Representation and Reasoning: Integrating Symbolic and Neural Approaches, Stanford, 2015.","title":"Neural-symbolic learning and reasoning: contributions and challenges","authors":"Besold Garcez, De Raedt, Foldiak, Hitzler, Icard, Kuhnberger, Miikkulainen Lamb, Silver","year":"2015","venue":"Proceedings of the AAAI Spring Symposium on Knowledge Representation and Reasoning: Integrating Symbolic and Neural Approaches","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Graves et al., 2014] Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines. arXiv preprint arXiv:1410.5401, 2014.","title":"Neural turing machines","authors":"Alex Graves, Greg Wayne, Ivo Danihelka","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Graves et al., 2016] Alex Graves, Greg Wayne, Mal- colm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwińska, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, et al. Hybrid computing using a neural network with dynamic external memory. Nature, 538(7626):471-476, 2016.","title":"Hybrid computing using a neural network with dynamic external memory","authors":"Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwińska, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou","year":"2016","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Hölldobler et al., 1999] Steffen Hölldobler, Yvonne Kalinke, and Hans-Peter Störr. Approximating the semantics of logic programs by recurrent neural networks. Applied Intelligence, 11(1):45-58, 1999.","title":"Approximating the semantics of logic programs by recurrent neural networks","authors":"Steffen Hölldobler, Yvonne Kalinke, Hans-Peter Störr","year":"1999","venue":"Applied Intelligence","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kaiser, 2015] Lukasz Kaiser. Neural gpus learn algorithms. arXiv preprint arXiv:1511.08228, 2015.","title":"Neural gpus learn algorithms","authors":"Lukasz Kaiser","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Law et al., 2014] Mark Law, Alessandra Russo, and Krysia Broda. Inductive learning of answer set programs. In European Workshop on Logics in Artificial Intelligence, pages 311-325. Springer, 2014.","title":"Inductive learning of answer set programs","authors":"Mark Law, Alessandra Russo, Krysia Broda","year":"2014","venue":"European Workshop on Logics in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Muggleton et al., 2014] Stephen H Muggleton, Dianhuan Lin, Niels Pahlavi, and Alireza Tamaddoni-Nezhad. Meta- interpretive learning: application to grammatical infer- ence. Machine Learning, 94(1):25-49, 2014.","title":"Metainterpretive learning: application to grammatical inference","authors":"Dianhuan Stephen H Muggleton, Niels Lin, Alireza Pahlavi, Tamaddoni-Nezhad","year":"2014","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Muggleton et al., 2015] Stephen H Muggleton, Dianhuan Lin, and Alireza Tamaddoni-Nezhad. Meta-interpretive learning of higher-order dyadic datalog: Predicate inven- tion revisited. Machine Learning, 100(1):49-73, 2015.","title":"Meta-interpretive learning of higher-order dyadic datalog: Predicate invention revisited","authors":"Dianhuan Stephen H Muggleton, Alireza Lin, Tamaddoni-Nezhad","year":"2015","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Neelakantan et al., 2015] Arvind Neelakantan, Quoc V Le, and Ilya Sutskever. Neural programmer: Inducing la- tent programs with gradient descent. arXiv preprint arXiv:1511.04834, 2015.","title":"Neural programmer: Inducing latent programs with gradient descent","authors":"Arvind Neelakantan, Ilya Quoc V Le, Sutskever","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Reed and de Freitas, 2015] Scott Reed and Nando de Fre- itas. Neural programmer-interpreters. arXiv preprint arXiv:1511.06279, 2015.","title":"Scott Reed and Nando de Freitas","authors":"Reed, De Freitas","year":"2015","venue":"Neural programmer-interpreters","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Rocktäschel and Riedel, 2017] Tim Rocktäschel and Sebas- tian Riedel. End-to-end differentiable proving. In Ad- vances in Neural Information Processing Systems, pages 3791-3803, 2017.","title":"Tim Rocktäschel and Sebastian Riedel. End-to-end differentiable proving","authors":"Riedel Rocktäschel","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Serafini and Garcez, 2016] Luciano Serafini and Artur Garcez. Logic tensor networks: Deep learning and logical reasoning from data and knowledge. arXiv preprint arXiv:1606.04422, 2016.","title":"Logic tensor networks: Deep learning and logical reasoning from data and knowledge","authors":"Garcez Serafini, Luciano Serafini, Artur Garcez","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}