{"type":"ScholarlyArticle","title":"Learning Big Logical Rules by Joining Small Rules","authors":[{"name":"Céline Hocquette","slug":"celine-hocquette","given":"Céline","family":"Hocquette","affiliation":"University of Oxford"},{"name":"Andreas Niskanen","slug":"andreas-niskanen","given":"Andreas","family":"Niskanen","affiliation":"University of Helsinki"},{"name":"Rolf Morel","slug":"rolf-morel","given":"Rolf","family":"Morel","affiliation":"University of Oxford"},{"name":"Matti Järvisalo","slug":"matti-jarvisalo","given":"Matti","family":"Järvisalo","affiliation":"University of Helsinki"},{"name":"Andrew Cropper","slug":"andrew-cropper","given":"Andrew","family":"Cropper","affiliation":"University of Oxford"}],"abstract":"A major challenge in inductive logic programming is learning big rules. To address this challenge, we introduce an approach where we join small rules to learn big rules. We implement our approach in a constraint-driven system and use constraint solvers to efficiently join rules. Our experiments on many domains, including game playing and drug design, show that our approach can (i) learn rules with more than 100 literals, and (ii) drastically outperform existing approaches in terms of predictive accuracies.","doi":"10.24963/ijcai.2024/380","url":"https://ijcai.org/proceedings/2024/380","pdf":"https://ijcai.org/proceedings/2024/0380.pdf","markdown":"https://ijcai.org/proceedings/2024/380.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/380","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Knowledge Representation and Reasoning","track":"Main Track","number":380,"pages":{"first":3430,"last":3438},"keywords":["Machine Learning: ML: Symbolic methods","Knowledge Representation and Reasoning: KRR: Logic programming"],"videos":[],"references":[{"position":1,"raw":"[Bacchus et al., 2021] Fahiem Bacchus, Matti Järvisalo, and Ruben Martins. Maximum satisfiabiliy. In Armin Biere, Marijn Heule, Hans van Maaren, and Toby Walsh, editors, Handbook of Satisfiability -Second Edition, volume 336 of Frontiers in Artificial Intelligence and Applications, pages 929-991. IOS Press, 2021.","title":"Fahiem Bacchus, Matti Järvisalo, and Ruben Martins. Maximum satisfiabiliy","authors":"Bacchus","year":"2021","venue":"Frontiers in Artificial Intelligence and Applications","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bembenek et al., 2023] Aaron Bembenek, Michael Green- berg, and Stephen Chong. From SMT to ASP: Solver-based approaches to solving datalog synthesis-as-rule-selection problems. Proc. ACM Program. Lang., 7(POPL), jan 2023.","title":"From SMT to ASP: Solver-based approaches to solving datalog synthesis-as-rule-selection problems","authors":"Bembenek","year":"2023","venue":"Proc. ACM Program. Lang","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Biere et al., 2021] Armin Biere, Marijn Heule, Hans van Maaren, and Toby Walsh, editors. Handbook of Satisfi- ability -Second Edition, volume 336 of FAIA. IOS Press, 2021.","title":"Handbook of Satisfiability -Second Edition","authors":"Biere","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Biere et al., 2023] Armin Biere, Mathias Fleury, and Flo- rian Pollitt. CaDiCaL, vivinst, IsaSAT, Gimsatul, Kissat, and TabularaSAT entering the SAT Competition 2023. In Proc. SAT Competition 2023: Solver, Benchmark and Proof Checker Descriptions, volume B-2023-1 of Department of Computer Science Report Series B, pages 14-15. University of Helsinki, 2023.","title":"Gimsatul, Kissat, and TabularaSAT entering the SAT Competition","authors":"Biere","year":"2023","venue":"Proc. SAT Competition 2023: Solver, Benchmark and Proof Checker Descriptions","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Blockeel and De Raedt, 1998] Hendrik Blockeel and Luc De Raedt. Top-down induction of first-order logical deci- sion trees. Artificial intelligence, 101(1-2):285-297, 1998.","title":"Top-down induction of first-order logical decision trees","authors":"De Blockeel, Hendrik Raedt, Luc Blockeel, Raedt De","year":"1998","venue":"Artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bramley et al., 2018] Neil Bramley, Anselm Rothe, Josh Tenenbaum, Fei Xu, and Todd M. Gureckis. Grounding compositional hypothesis generation in specific instances. In Proceedings of the 40th Annual Meeting of the Cognitive Science Society, CogSci 2018, 2018.","title":"Grounding compositional hypothesis generation in specific instances","authors":"Neil Bramley, Anselm Rothe, Josh Tenenbaum, Fei Xu, Todd M Gureckis","year":"2018","venue":"Proceedings of the 40th Annual Meeting of the Cognitive Science Society","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Bunel et al., 2018] Rudy Bunel, Matthew Hausknecht, Jacob Devlin, Rishabh Singh, and Pushmeet Kohli. Leveraging grammar and reinforcement learning for neural program synthesis. arXiv preprint arXiv:1805.04276, 2018.","title":"Leveraging grammar and reinforcement learning for neural program synthesis","authors":"Rudy Bunel, Matthew Hausknecht, Jacob Devlin, Rishabh Singh, Pushmeet Kohli","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Castillo and Wrobel, 2002] Lourdes Pena Castillo and Ste- fan Wrobel. Macro-operators in multirelational learning: a search-space reduction technique. In European Conference on Machine Learning, pages 357-368. Springer, 2002.","title":"Lourdes Pena Castillo and Stefan Wrobel. Macro-operators in multirelational learning: a search-space reduction technique","authors":"Wrobel Castillo","year":"2002","venue":"European Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Chollet, 2019] Franc ¸ois Chollet. On the measure of intelli- gence. CoRR, 2019.","title":"Franc ¸ois Chollet. On the measure of intelligence","authors":"Chollet","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Corapi et al., 2011] Domenico Corapi, Alessandra Russo, and Emil Lupu. Inductive logic programming in answer set programming. In Inductive Logic Programming -21st International Conference, pages 91-97, 2011.","title":"Inductive logic programming in answer set programming","authors":"Domenico Corapi, Alessandra Russo, Emil Lupu","year":"2011","venue":"Inductive Logic Programming -21st International Conference","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Costa et al., 2003] Vítor Santos Costa, Ashwin Srinivasan, Rui Camacho, Hendrik Blockeel, Bart Demoen, Gerda Janssens, Jan Struyf, Henk Vandecasteele, and Wim Van Laer. Query transformations for improving the efficiency of ILP systems. Journal of Machine Learning Research, pages 465-491, 2003.","title":"Query transformations for improving the efficiency of ILP systems","authors":"Santos Vítor, Ashwin Costa, Rui Srinivasan, Hendrik Camacho, Bart Blockeel, Gerda Demoen, Jan Janssens, Henk Struyf, Wim Vandecasteele, Van Laer","year":"2003","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Cropper and Dumančić, 2020] Andrew Cropper and Sebasti- jan Dumančić. Learning large logic programs by going beyond entailment. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020, pages 2073-2079, 2020.","title":"Learning large logic programs by going beyond entailment","authors":"Andrew Cropper, Sebastijan Dumančić","year":"2020","venue":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020","doi":null,"url":null,"internal_url":"/proceedings/2020/287"},{"position":13,"raw":"[Cropper and Hocquette, 2023] Andrew Cropper and Céline Hocquette. Learning logic programs by combining pro- grams. In ECAI 2023 -26th European Conference on Artificial Intelligence, volume 372, pages 501-508. IOS Press, 2023.","title":"Learning logic programs by combining programs","authors":"Andrew Cropper, Céline Hocquette","year":"2023","venue":"ECAI 2023 -26th European Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Cropper and Morel, 2021] Andrew Cropper and Rolf Morel. Learning programs by learning from failures. Machine Learning, 110(4):801-856, 2021.","title":"Learning programs by learning from failures","authors":"Morel Cropper, Andrew Cropper, Rolf Morel","year":"2021","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Cropper et al., 2020] Andrew Cropper, Richard Evans, and Mark Law. Inductive general game playing. Machine Learning, 109(7):1393-1434, 2020.","title":"Inductive general game playing","authors":"Andrew Cropper, Richard Evans, Mark Law","year":"2020","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Cropper et al., 2022] Andrew Cropper, Sebastijan Du- mančić, Richard Evans, and Stephen H. Muggleton. In- ductive logic programming at 30. Machine Learning, 111(1):147-172, 2022.","title":"Inductive logic programming at 30","authors":"Andrew Cropper, Sebastijan Dumančić, Richard Evans, Stephen H Muggleton","year":"2022","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Devlin et al., 2017] Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel-rahman Mohamed, and Pushmeet Kohli. Robustfill: Neural program learning under noisy i/o. In International conference on machine learning, pages 990-998. PMLR, 2017.","title":"Robustfill: Neural program learning under noisy i/o","authors":"Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel-Rahman Mohamed, Pushmeet Kohli","year":"2017","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Evans and Grefenstette, 2018] Richard Evans and Edward Grefenstette. Learning explanatory rules from noisy data. Journal of Artificial Intelligence Research, pages 1-64, 2018.","title":"Learning explanatory rules from noisy data","authors":"Grefenstette Evans, Richard Evans, Edward Grefenstette","year":"2018","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Evans et al., 2021] Richard Evans, José Hernández-Orallo, Johannes Welbl, Pushmeet Kohli, and Marek J. Sergot. Making sense of sensory input. Artificial Intelligence, page 103438, 2021.","title":"Making sense of sensory input","authors":"Richard Evans, José Hernández-Orallo, Johannes Welbl, Pushmeet Kohli, Marek J Sergot","year":"2021","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ferilli, 2016] Stefano Ferilli. Predicate invention-based spe- cialization in inductive logic programming. Journal of Intelligent Information Systems, 47(1):33-55, 2016.","title":"Predicate invention-based specialization in inductive logic programming","authors":"Stefano Ferilli","year":"2016","venue":"Journal of Intelligent Information Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Finn et al., 1998] Paul Finn, Stephen Muggleton, David Page, and Ashwin Srinivasan. Pharmacophore discovery using the inductive logic programming system progol. Ma- chine Learning, 30(2):241-270, 1998.","title":"Pharmacophore discovery using the inductive logic programming system progol","authors":"Paul Finn, Stephen Muggleton, David Page, Ashwin Srinivasan","year":"1998","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Galárraga et al., 2015] Luis Galárraga, Christina Teflioudi, Katja Hose, and Fabian M. Suchanek. Fast rule mining in ontological knowledge bases with AMIE+. VLDB J., 24(6):707-730, 2015.","title":"Fast rule mining in ontological knowledge bases with AMIE+","authors":"Luis Galárraga, Christina Teflioudi, Katja Hose, Fabian M Suchanek","year":"2015","venue":"VLDB J","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Genesereth and Björnsson, 2013] Michael Genesereth and Yngvi Björnsson. The international general game play- ing competition. AI Magazine, 34(2):107-107, 2013.","title":"The international general game playing competition","authors":"Björnsson Genesereth, Michael Genesereth, Yngvi Björnsson","year":"2013","venue":"AI Magazine","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Hocquette et al., 2024] Céline Hocquette, Andreas Niska- nen, Matti Järvisalo, and Andrew Cropper. Learning MDL logic programs from noisy data. In AAAI, 2024.","title":"Learning MDL logic programs from noisy data","authors":"Hocquette","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Ignatiev et al., 2018] Alexey Ignatiev, Antonio Morgado, and Joao Marques-Silva. PySAT: A Python toolkit for prototyping with SAT oracles. In International Conference on Theory and Applications of Satisfiability Testing, pages 428-437. Springer, 2018.","title":"PySAT: A Python toolkit for prototyping with SAT oracles","authors":"Alexey Ignatiev, Antonio Morgado, Joao Marques-Silva","year":"2018","venue":"International Conference on Theory and Applications of Satisfiability Testing","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Kaminski et al., 2019] Tobias Kaminski, Thomas Eiter, and Katsumi Inoue. Meta-interpretive learning using hex- programs. In Proceedings of the Twenty-Eighth Interna- tional Joint Conference on Artificial Intelligence, IJCAI 2019, pages 6186-6190, 2019.","title":"Meta-interpretive learning using hexprograms","authors":"Kaminski","year":"2019","venue":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Law et al., 2014] Mark Law, Alessandra Russo, and Krysia Broda. Inductive learning of answer set programs. In Logics in Artificial Intelligence -14th European Conference, JELIA 2014, pages 311-325, 2014.","title":"Inductive learning of answer set programs","authors":"Mark Law, Alessandra Russo, Krysia Broda","year":"2014","venue":"Logics in Artificial Intelligence -14th European Conference, JELIA 2014","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Liffiton and Sakallah, 2008] Mark H. Liffiton and Karem A. Sakallah. Algorithms for computing minimal unsatisfiable subsets of constraints. J. Autom. Reason., 40(1):1-33, 2008.","title":"Algorithms for computing minimal unsatisfiable subsets of constraints","authors":"H Mark, Karem A Liffiton, Sakallah","year":"2008","venue":"J. Autom. Reason","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Lloyd, 2012] John W Lloyd. Foundations of logic program- ming. Springer Science & Business Media, 2012.","title":"Foundations of logic programming","authors":"John W, Lloyd","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Manthey et al., 2014] Norbert Manthey, Tobias Philipp, and Peter Steinke. A more compact translation of pseudo- Boolean constraints into CNF such that generalized arc consistency is maintained. In Carsten Lutz and Michael Thielscher, editors, KI 2014, volume 8736 of LNCS, pages 123-134. Springer, 2014.","title":"A more compact translation of pseudo-Boolean constraints into CNF such that generalized arc consistency is maintained","authors":"Manthey","year":"2014","venue":"LNCS","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Mihalkova et al., 2007] Lilyana Mihalkova, Tuyen N. Huynh, and Raymond J. Mooney. Mapping and revis- ing markov logic networks for transfer learning. In Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence, July 22-26, 2007, Vancouver, British Columbia, Canada, pages 608-614. AAAI Press, 2007.","title":"Mapping and revising markov logic networks for transfer learning","authors":"Lilyana Mihalkova, N Tuyen, Raymond J Huynh, Mooney","year":"2007","venue":"Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Muggleton, 1991] Stephen Muggleton. Inductive logic pro- gramming. New Generation Computing, 8(4):295-318, 1991.","title":"Inductive logic programming","authors":"Stephen Muggleton","year":"1991","venue":"New Generation Computing","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Muggleton, 1995] Stephen H. Muggleton. Inverse en- tailment and Progol. New Generation Computing, 13(3&4):245-286, 1995.","title":"Inverse entailment and Progol","authors":"H Stephen, Muggleton","year":"1995","venue":"New Generation Computing","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Piotrów, 2020] Marek Piotrów. UWrMaxSat: Efficient solver for MaxSAT and pseudo-Boolean problems. In 2020 IEEE 32nd International Conference on Tools with Artifi- cial Intelligence (ICTAI), pages 132-136, 2020.","title":"UWrMaxSat: Efficient solver for MaxSAT and pseudo-Boolean problems","authors":"Marek Piotrów","year":"2020","venue":"IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI)","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Quinlan, 1990] J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, pages 239-266, 1990.","title":"Learning logical definitions from relations","authors":"J, Ross Quinlan","year":"1990","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Raghothaman et al., 2020] Mukund Raghothaman, Jonathan Mendelson, David Zhao, Mayur Naik, and Bernhard Scholz. Provenance-guided synthesis of datalog programs. Proc. ACM Program. Lang., 4(POPL):62:1-62:27, 2020.","title":"Provenance-guided synthesis of datalog programs","authors":"Mukund Raghothaman, Jonathan Mendelson, David Zhao, Mayur Naik, Bernhard Scholz","year":"2020","venue":"Proc. ACM Program. Lang","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Shapiro, 1983] E.Y. Shapiro. Algorithmic program debug- ging. 1983.","title":"Algorithmic program debugging","authors":"E Y Shapiro","year":"1983","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Shi et al., 2022] Kensen Shi, Hanjun Dai, Kevin Ellis, and Charles Sutton. Crossbeam: Learning to search in bottom- up program synthesis. In International Conference on Learning Representations, 2022.","title":"Crossbeam: Learning to search in bottomup program synthesis","authors":"Kensen Shi, Hanjun Dai, Kevin Ellis, Charles Sutton","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Si et al., 2019] Xujie Si, Mukund Raghothaman, Kihong Heo, and Mayur Naik. Synthesizing datalog programs using numerical relaxation. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, pages 6117-6124, 2019.","title":"Synthesizing datalog programs using numerical relaxation","authors":"Si","year":"2019","venue":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019","doi":null,"url":null,"internal_url":"/proceedings/2019/847"},{"position":40,"raw":"[Silverstein and Pazzani, 1991] Glenn Silverstein and Michael J Pazzani. Relational clichés: Constraining constructive induction during relational learning. In Machine Learning Proceedings 1991, pages 203-207. Elsevier, 1991.","title":"Relational clichés: Constraining constructive induction during relational learning","authors":"Glenn Silverstein, Michael J Pazzani","year":"1991","venue":"Machine Learning Proceedings 1991","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Srinivasan, 2001] Ashwin Srinivasan. The ALEPH manual. Machine Learning at the Computing Laboratory, Oxford University, 2001.","title":"The ALEPH manual","authors":"Ashwin Srinivasan","year":"2001","venue":"Machine Learning at the Computing Laboratory","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Xu et al., 2023] Yudong Xu, Wenhao Li, Pashootan Vaezipoor, Scott Sanner, and Elias B Khalil. LLMs and the abstraction and reasoning corpus: Successes, failures, and the importance of object-based representations. arXiv preprint arXiv:2305.18354, 2023.","title":"LLMs and the abstraction and reasoning corpus: Successes, failures, and the importance of object-based representations","authors":"Yudong Xu, Wenhao Li, Pashootan Vaezipoor, Scott Sanner, Elias B Khalil","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}