{"type":"ScholarlyArticle","title":"Complete Bottom-Up Predicate Invention in Meta-Interpretive Learning","authors":[{"name":"Céline Hocquette","slug":"celine-hocquette","given":"Céline","family":"Hocquette","affiliation":"Department of Computing , Imperial College London , London , UK"},{"name":"Stephen H. Muggleton","slug":"stephen-h-muggleton","given":"Stephen H.","family":"Muggleton","affiliation":"Department of Computing , Imperial College London , London , UK"}],"abstract":"Predicate Invention in Meta-Interpretive Learning (MIL) is generally based on a top-down approach, and the search for a consistent hypothesis is carried out starting from the positive examples as goals. We consider augmenting top-down MIL systems with a bottom-up step during which the background knowledge is generalised with an extension of the immediate consequence operator for second-order logic programs. This new method provides a way to perform extensive predicate invention useful for feature discovery. We demonstrate this method is complete with respect to a fragment of dyadic datalog. We theoretically prove this method reduces the number of clauses to be learned for the top-down learner, which in turn can reduce the sample complexity. We formalise an equivalence relation for predicates which is used to eliminate redundant predicates. Our experimental results suggest pairing the state-of-the-art MIL system Metagol with an initial bottom-up step can significantly improve learning performance.","doi":"10.24963/ijcai.2020/320","url":"https://ijcai.org/proceedings/2020/320","pdf":"https://ijcai.org/proceedings/2020/0320.pdf","markdown":"https://ijcai.org/proceedings/2020/320.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/320","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":320,"pages":{"first":2312,"last":2318},"keywords":["Machine Learning: Relational Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25526/file_25526.m3u8","length_sec":285,"provider":"hls"}],"references":[{"position":1,"raw":"[Bratko, 1978] Ivan Bratko. Proving correctness of strategies in the AL1 assertional language. Information Processing Letters, 7(5):223 -230, 1978.","title":"Proving correctness of strategies in the AL1 assertional language","authors":"Ivan Bratko, Bratko","year":"1978","venue":"Information Processing Letters","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Corapi et al., 2011] Domenico Corapi, Alessandra Russo, and Emil Lupu. Inductive logic programming in answer set programming. In ILP, volume 7207 of Lecture Notes in Computer Science, pages 91-97. Springer, 2011.","title":"Inductive logic programming in answer set programming","authors":"Domenico Corapi, Alessandra Russo, Emil Lupu","year":"2011","venue":"ILP","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cropper and Muggleton, 2016a] Andrew Cropper and Stephen H. Muggleton. Learning Higher-Order Logic Programs through Abstraction and Invention. In Subbarao Kambhampati, editor, Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9-15 July 2016, pages 1418-1424. IJCAI/AAAI Press, 2016.","title":"Learning Higher-Order Logic Programs through Abstraction and Invention","authors":"Muggleton Cropper, Andrew Cropper, Stephen H Muggleton","year":"2016","venue":"Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016","doi":null,"url":null,"internal_url":"/proceedings/2016/204"},{"position":4,"raw":"[Cropper and Muggleton, 2016b] Andrew Cropper and Stephen H. Muggleton. Metagol system. https://github.com/metagol/metagol, 2016.","title":"Metagol system","authors":"Muggleton Cropper, Andrew Cropper, Stephen H Muggleton","year":"2016","venue":null,"doi":null,"url":"https://github.com/metagol/metagol,2016","internal_url":null},{"position":5,"raw":"[Cropper, 2019] Andrew Cropper. Playgol: Learning Pro- grams Through Play. In Sarit Kraus, editor, Proceedings of the Twenty-Eighth International Joint Conference on Arti- ficial Intelligence, IJCAI 2019, Macao, China, August 10- 16, 2019, pages 6074-6080. ijcai.org, 2019.","title":"Playgol: Learning Programs Through Play","authors":"Andrew Cropper","year":"2019","venue":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019","doi":null,"url":null,"internal_url":"/proceedings/2019/841"},{"position":6,"raw":"[Fensel and Wiese, 1993] Dieter Fensel and Markus Wiese. Refinement of rule sets with jojo. In Pavel B. Brazdil, edi- tor, Machine Learning: ECML-93, pages 378-383, Berlin, Heidelberg, 1993. Springer Berlin Heidelberg.","title":"Refinement of rule sets with jojo","authors":"Wiese Fensel, Dieter Fensel, Markus Wiese","year":"1993","venue":"editor, Machine Learning: ECML-93","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gulwani, 2011] Sumit Gulwani. Automating string pro- cessing in spreadsheets using input-output examples. In Proceedings of the 38th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL '11, pages 317-330, New York, NY, USA, 2011. ACM.","title":"Automating string processing in spreadsheets using input-output examples","authors":"Sumit Gulwani","year":"2011","venue":"Proceedings of the 38th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL '11","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Law et al., 2014] Mark Law, Alessandra Russo, and Krysia Broda. Inductive learning of answer set programs. In E. Fermé and J. Leite, editors, Logics in Artificial Intelli- gence, pages 311-325, Cham, 2014. Springer International Publishing.","title":"Inductive learning of answer set programs","authors":"Mark Law, Alessandra Russo, Krysia Broda","year":"2014","venue":"Logics in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lin et al., 2014] Dianhuan Lin, Eyal Dechter, Kevin Ellis, Joshua B. Tenenbaum, and Stephen H. Muggleton. Bias reformulation for one-shot function induction. In Proceed- ings of the 23rd European Conference on Artificial Intelli- gence (ECAI 2014), pages 525-530, 2014.","title":"Bias reformulation for one-shot function induction","authors":"Dianhuan Lin, Eyal Dechter, Kevin Ellis, Joshua B Tenenbaum, Stephen H Muggleton","year":"2014","venue":"Proceedings of the 23rd European Conference on Artificial Intelligence (ECAI 2014)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lloyd, 1984] John W. Lloyd. Foundations of Logic Pro- gramming, 1st Edition. Springer, 1984.","title":"Foundations of Logic Programming","authors":"John W Lloyd","year":"1984","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Maher, 1988] Michael J. Maher. Chapter 16 -equivalences of logic programs. pages 627 -658, 1988.","title":"Chapter 16 -equivalences of logic programs","authors":"J Michael, Maher","year":"1988","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Mitchell, 1982] Tom M. Mitchell. Generalization as search. Artificial Intelligence, 18(2):203 -226, 1982.","title":"Generalization as search","authors":"Tom M Mitchell","year":"1982","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Muggleton and Buntine, 1988] Stephen H. Muggleton and Wray Buntine. Machine invention of first-order predi- cates by inverting resolution. In John Laird, editor, Ma- chine Learning Proceedings 1988, pages 339 -352.","title":"Machine invention of first-order predicates by inverting resolution","authors":"Stephen H Muggleton, Wray Buntine","year":"1988","venue":"Machine Learning Proceedings","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"Mor- gan Kaufmann, San Francisco (CA), 1988.","title":null,"authors":"Morgan Kaufmann","year":"1988","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Muggleton and Lin, 2013] Stephen H. Muggleton and Di- anhuan Lin. Meta-interpretive learning of higher-order dyadic datalog: Predicate invention revisited. In Proceed- ings of the 23rd International Joint Conference Artificial Intelligence, pages 1551-1557, 2013.","title":"Meta-interpretive learning of higher-order dyadic datalog: Predicate invention revisited","authors":"Stephen H Muggleton, Dianhuan Lin","year":"2013","venue":"Proceedings of the 23rd International Joint Conference Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2013/231"},{"position":16,"raw":"[Muggleton et al., 2012] Stephen Muggleton, Luc De Raedt, David Poole, Ivan Bratko, Peter A. Flach, Katsumi Inoue, and Ashwin Srinivasan. ILP turns 20. Machine Learning, 86(1):3-23, Jan 2012.","title":"ILP turns 20","authors":"Stephen Muggleton, Luc De Raedt, David Poole, Ivan Bratko, Peter A Flach, Katsumi Inoue, Ashwin Srinivasan","year":"2012","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Muggleton et al., 2014] Stephen H. Muggleton, Dianhuan Lin, Niels Pahlavi, and Alireza Tamaddoni-Nezhad. Meta- interpretive learning: application to grammatical infer- ence. Machine Learning 94, pages 25-49, 2014.","title":"Metainterpretive learning: application to grammatical inference","authors":"H Stephen, Dianhuan Muggleton, Niels Lin, Alireza Pahlavi, Tamaddoni-Nezhad","year":"2014","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Muggleton, 1991] Stephen H. Muggleton. Inductive logic programming. New Generation Computing, 8(4):295-318, Feb 1991.","title":"Inductive logic programming","authors":"H Stephen, Muggleton","year":"1991","venue":"New Generation Computing","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Muggleton, 1995] Stephen H. Muggleton. Inverse En- tailment and Progol. New Generation Comput., 13(3&4):245-286, 1995.","title":"Inverse Entailment and Progol","authors":"H Stephen, Muggleton","year":"1995","venue":"New Generation Comput","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Quinlan, 1983] J. Ross Quinlan. Learning Efficient Classi- fication Procedures and Their Application to Chess End Games, pages 463-482. Springer Berlin Heidelberg, Berlin, Heidelberg, 1983.","title":"Learning Efficient Classification Procedures and Their Application to Chess End Games","authors":"J, Ross Quinlan","year":"1983","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rabinovich et al., 1996] Savely Rabinovich, Gregory Berkolaiko, and Shlomo Havlin. Solving nonlinear recur- sions. Journal of Mathematical Physics, 37:5828-5836, November 1996.","title":"Solving nonlinear recursions","authors":"Savely Rabinovich, Gregory Berkolaiko, Shlomo Havlin","year":"1996","venue":"Journal of Mathematical Physics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shapiro and Niblett, 1982] Alen D. Shapiro and Tim Niblett. Automatic induction of classification rules for a chess endgame. In M.R.B. Clarke, editor, Advances in Computer Chess, volume 3, pages 73-91. Pergammon, Oxford, 1982.","title":"Automatic induction of classification rules for a chess endgame","authors":"Alen D Shapiro, Tim Niblett","year":"1982","venue":"Advances in Computer Chess","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Stahl, 1993] Irene Stahl. Predicate invention in ILP - an overview. In P. B. Brazdil, editor, Machine Learn- ing: ECML-93, pages 311-322, Berlin, Heidelberg, 1993. Springer Berlin Heidelberg.","title":"Predicate invention in ILPan overview","authors":"Irene Stahl","year":"1993","venue":"Machine Learning: ECML-93","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Stahl, 1995] Irene Stahl. The appropriateness of predicate invention as bias shift operation in ILP. Machine Learning, 20(1):95-117, Jul 1995.","title":"The appropriateness of predicate invention as bias shift operation in ILP","authors":"Irene Stahl, Stahl","year":"1995","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Van Emden and Kowalski, 1976] Maarten H. Van Emden and Robert A. Kowalski. The semantics of predicate logic as a programming language. J. ACM, 23(4):733-742, Oc- tober 1976.","title":"The semantics of predicate logic as a programming language","authors":"Van Emden, H Kowalski ; ] Maarten, Robert A Van Emden, Kowalski","year":"1976","venue":"J. ACM","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zelle et al., 1994] John M. Zelle, Raymond J. Mooney, and Joshua B. Konvisser. Combining top-down and bottom-up techniques in inductive logic programming. In William W. Cohen and Haym Hirsh, editors, Machine Learning Pro- ceedings 1994, pages 343 -351.","title":"Combining top-down and bottom-up techniques in inductive logic programming","authors":"John M Zelle, Raymond J Mooney, Joshua B Konvisser","year":"1994","venue":"Machine Learning Proceedings","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"Morgan Kaufmann, San Francisco (CA), 1994.","title":null,"authors":"Morgan Kaufmann","year":"1994","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}