{"type":"ScholarlyArticle","title":"An Ensemble Approach for Automated Theorem Proving Based on Efficient Name Invariant Graph Neural Representations","authors":[{"name":"Achille Fokoue","slug":"achille-fokoue","given":"Achille","family":"Fokoue","affiliation":"IBM Research"},{"name":"Ibrahim Abdelaziz","slug":"ibrahim-abdelaziz","given":"Ibrahim","family":"Abdelaziz","affiliation":"IBM Research"},{"name":"Maxwell Crouse","slug":"maxwell-crouse","given":"Maxwell","family":"Crouse","affiliation":"IBM Research"},{"name":"Shajith Ikbal","slug":"shajith-ikbal","given":"Shajith","family":"Ikbal","affiliation":"IBM Research"},{"name":"Akihiro Kishimoto","slug":"akihiro-kishimoto","given":"Akihiro","family":"Kishimoto","affiliation":"IBM Research"},{"name":"Guilherme Lima","slug":"guilherme-lima","given":"Guilherme","family":"Lima","affiliation":"IBM Research"},{"name":"Ndivhuwo Makondo","slug":"ndivhuwo-makondo","given":"Ndivhuwo","family":"Makondo","affiliation":"IBM Research"},{"name":"Radu Marinescu","slug":"radu-marinescu","given":"Radu","family":"Marinescu","affiliation":"IBM Research"}],"abstract":"Using reinforcement learning for automated theorem proving has recently received much attention. Current approaches use representations of logical statements that often rely on the names used in these statements and, as a result, the models are generally not transferable from one domain to another. The size of these representations and whether to include the whole theory or part of it are other important decisions that affect the performance of these approaches as well as their runtime efficiency. In this paper, we present NIAGRA; an ensemble Name InvAriant Graph RepresentAtion. NIAGRA addresses this problem by using 1) improved Graph Neural Networks for learning name-invariant formula representations that is tailored for their unique characteristics and 2) an efficient ensemble approach for automated theorem proving. Our experimental evaluation shows state-of-the-art performance on multiple datasets from different domains with improvements up to 10% compared to the best learning-based approaches. Furthermore, transfer learning experiments show that our approach significantly outperforms other learning-based approaches by up to 28%.","doi":"10.24963/ijcai.2023/359","url":"https://ijcai.org/proceedings/2023/359","pdf":"https://ijcai.org/proceedings/2023/0359.pdf","markdown":"https://ijcai.org/proceedings/2023/359.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/359","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Knowledge Representation and Reasoning","track":"Main Track","number":359,"pages":{"first":3221,"last":3229},"keywords":["Machine Learning: ML: Applications","Knowledge Representation and Reasoning: KRR: Automated reasoning and theorem proving","Machine Learning: ML: Representation learning"],"videos":[],"references":[{"position":1,"raw":"[Abdelaziz et al., 2022] Ibrahim Abdelaziz, Maxwell Crouse, Bassem Makni, Vernon Austel, Cristina Cornelio, Shajith Ikbal, Pavan Kapanipathi, Ndivhuwo Makondo, Kavitha Srinivas, Michael Witbrock, et al. Learning to guide a saturation-based theorem prover. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 2022.","title":"Learning to guide a saturation-based theorem prover","authors":"Ibrahim Abdelaziz, Maxwell Crouse, Bassem Makni, Vernon Austel, Cristina Cornelio, Shajith Ikbal, Pavan Kapanipathi, Ndivhuwo Makondo, Kavitha Srinivas, Michael Witbrock","year":"2022","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Andrychowicz et al., 2017] Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, OpenAI Pieter Abbeel, and Wojciech Zaremba. Hindsight expe- rience replay. Advances in neural information processing systems, 30, 2017.","title":"Hindsight experience replay","authors":"Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob Mcgrew, Josh Tobin, Openai, Pieter Abbeel, Wojciech Zaremba","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Aygün et al., 2022] Eser Aygün, Ankit Anand, Laurent Orseau, Xavier Glorot, Stephen M Mcaleer, Vlad Firoiu, Lei M Zhang, Doina Precup, and Shibl Mourad. Prov- ing theorems using incremental learning and hindsight ex- perience replay. In International Conference on Machine Learning, pages 1198-1210. PMLR, 2022.","title":"Proving theorems using incremental learning and hindsight experience replay","authors":"Ankit Eser Aygün, Laurent Anand, Xavier Orseau, Stephen M Glorot, Vlad Mcaleer, Firoiu, M Lei, Doina Zhang, Shibl Precup, Mourad","year":"2022","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bansal et al., 2019] Kshitij Bansal, Sarah Loos, Markus Rabe, Christian Szegedy, and Stewart Wilcox. Holist: An environment for machine learning of higher order logic theorem proving. In International Conference on Machine Learning, pages 454-463. PMLR, 2019.","title":"Holist: An environment for machine learning of higher order logic theorem proving","authors":"Kshitij Bansal, Sarah Loos, Markus Rabe, Christian Szegedy, Stewart Wilcox","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bronstein et al., 2017] Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18-42, 2017.","title":"Geometric deep learning: going beyond euclidean data","authors":"Joan Michael M Bronstein, Yann Bruna, Arthur Lecun, Pierre Szlam, Vandergheynst","year":"2017","venue":"IEEE Signal Processing Magazine","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Crouse et al., 2019] Maxwell Crouse, Ibrahim Abdelaziz, Cristina Cornelio, Veronika Thost, Lingfei Wu, Kenneth Forbus, and Achille Fokoue. Improving graph neural net- work representations of logical formulae with subgraph pooling. arXiv preprint arXiv:1911.06904, 2019.","title":"Improving graph neural network representations of logical formulae with subgraph pooling","authors":"Maxwell Crouse, Ibrahim Abdelaziz, Cristina Cornelio, Veronika Thost, Lingfei Wu, Kenneth Forbus, Achille Fokoue","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Crouse et al., 2021] Maxwell Crouse, Ibrahim Abdelaziz, Bassem Makni, Spencer Whitehead, Cristina Cornelio, Pavan Kapanipathi, Kavitha Srinivas, Veronika Thost, Michael Witbrock, and Achille Fokoue. A deep reinforce- ment learning approach to first-order logic theorem prov- ing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 6279-6287, 2021.","title":"A deep reinforcement learning approach to first-order logic theorem proving","authors":"Maxwell Crouse, Ibrahim Abdelaziz, Bassem Makni, Spencer Whitehead, Cristina Cornelio, Pavan Kapanipathi, Kavitha Srinivas, Veronika Thost, Michael Witbrock, Achille Fokoue","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Garland and Lynch, 2000] Stephen J Garland and Nancy A Lynch. Using I/O automata for developing distributed sys- tems. Foundations of component-based systems, 13(285- 312):5-2, 2000.","title":"Using I/O automata for developing distributed systems","authors":"J Stephen, Nancy A Garland, Lynch","year":"2000","venue":"Foundations of component-based systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Grabowski et al., 2010] Adam Grabowski, Artur Kornilow- icz, and Adam Naumowicz. Mizar in a nutshell. Journal of Formalized Reasoning, 3(2):153-245, 2010.","title":"Mizar in a nutshell","authors":"Adam Grabowski, Artur Kornilowicz, Adam Naumowicz","year":"2010","venue":"Journal of Formalized Reasoning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hamadi et al., 2010] Youssef Hamadi, Said Jabbour, and Lakhdar Sais. ManySAT: a parallel SAT solver. Jour- nal on Satisfiability, Boolean Modeling and Computation, 6(4):245-262, 2010.","title":"ManySAT: a parallel SAT solver. Journal on Satisfiability","authors":"Youssef Hamadi, Said Jabbour, Lakhdar Sais","year":"2010","venue":"Boolean Modeling and Computation","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hawblitzel et al., 2015] Chris Hawblitzel, Jon Howell, Manos Kapritsos, Jacob R Lorch, Bryan Parno, Michael L Roberts, Srinath Setty, and Brian Zill. Ironfleet: proving practical distributed systems correct. In Proceedings of the 25th Symposium on Operating Systems Principles, pages 1-17, 2015.","title":"Ironfleet: proving practical distributed systems correct","authors":"Chris Hawblitzel, Jon Howell, Manos Kapritsos, Bryan Jacob R Lorch, Parno, Srinath Michael L Roberts, Brian Setty, Zill","year":"2015","venue":"Proceedings of the 25th Symposium on Operating Systems Principles","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huberman et al., 1997] Bernardo A Huberman, Rajan M Lukose, and Tad Hogg. An economics approach to hard computational problems. Science, 275(5296):51-54, 1997.","title":"An economics approach to hard computational problems","authors":"Rajan M Bernardo A Huberman, Tad Lukose, Hogg","year":"1997","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jakubuv et al., 2020] Jan Jakubuv, Karel Chvalovský, Miroslav Olsák, Bartosz Piotrowski, Martin Suda, and Josef Urban. Enigma anonymous: Symbol-independent inference guiding machine (system description). In International Joint Conference on Automated Reasoning, pages 448-463. Springer, 2020.","title":"Enigma anonymous: Symbol-independent inference guiding machine (system description)","authors":"Jakubuv","year":"2020","venue":"International Joint Conference on Automated Reasoning","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kaliszyk et al., 2018] Cezary Kaliszyk, Josef Urban, Hen- ryk Michalewski, and Miroslav Olšák. Reinforcement learning of theorem proving. In Advances in Neural In- formation Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, pages 8836-8847, 2018.","title":"Reinforcement learning of theorem proving","authors":"Cezary Kaliszyk, Josef Urban, Henryk Michalewski, Miroslav Olšák","year":"2018","venue":"Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Katz et al., 2018] Michael Katz, Shirin Sohrabi, Horst Samulowitz, and Silvan Sievers. Delfi: Online planner se- lection for cost-optimal planning. In Ninth International Planning Competition (IPC-9), pages 55-62, 2018.","title":"Delfi: Online planner selection for cost-optimal planning","authors":"Michael Katz, Shirin Sohrabi, Horst Samulowitz, Silvan Sievers","year":"2018","venue":"Ninth International Planning Competition (IPC-9)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kovács and Voronkov, 2013] Laura Kovács and Andrei Voronkov. First-order theorem proving and vampire. In International Conference on Computer Aided Verification, pages 1-35. Springer, 2013.","title":"First-order theorem proving and vampire","authors":"Laura Kovács, Andrei Voronkov","year":"2013","venue":"International Conference on Computer Aided Verification","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Loos et al., 2017] Sarah Loos, Geoffrey Irving, Christian Szegedy, and Cezary Kaliszyk. Deep network guided proof search. In Proceedings of the 21st International Conference on Logic for Programming, Artificial Intelli- gence and Reasoning, pages 85-105, 2017.","title":"Deep network guided proof search","authors":"Sarah Loos, Geoffrey Irving, Christian Szegedy, Cezary Kaliszyk","year":"2017","venue":"Proceedings of the 21st International Conference on Logic for Programming, Artificial Intelligence and Reasoning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lucas, 1993] Peter Lucas. The representation of medical reasoning models in resolution-based theorem provers. Ar- tificial Intelligence in Medicine, 5(5):395-414, 1993.","title":"The representation of medical reasoning models in resolution-based theorem provers","authors":"Peter Lucas","year":"1993","venue":"Artificial Intelligence in Medicine","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Masci et al., 2014] Paolo Masci, Yi Zhang, Paul Jones, Paul Curzon, and Harold Thimbleby. Formal verification of medical device user interfaces using PVS. In International Conference on Fundamental Approaches to Software En- gineering, pages 200-214. Springer, 2014.","title":"Formal verification of medical device user interfaces using PVS","authors":"Paolo Masci, Yi Zhang, Paul Jones, Paul Curzon, Harold Thimbleby","year":"2014","venue":"International Conference on Fundamental Approaches to Software Engineering","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Olšák et al., 2019] Miroslav Olšák, Cezary Kaliszyk, and Josef Urban. Property invariant embedding for automated reasoning. In 24th European Conference on Artificial In- telligence, 2019.","title":"Property invariant embedding for automated reasoning","authors":"Olšák","year":"2019","venue":"24th European Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Paliwal et al., 2020] Aditya Paliwal, Sarah Loos, Markus Rabe, Kshitij Bansal, and Christian Szegedy. Graph repre- sentations for higher-order logic and theorem proving. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 34, pages 2967-2974, 2020.","title":"Graph representations for higher-order logic and theorem proving","authors":"Aditya Paliwal, Sarah Loos, Markus Rabe, Kshitij Bansal, Christian Szegedy","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Rawson and Reger, 2020] Michael Rawson and Giles Reger. Directed graph networks for logical reasoning. In Workshop on Practical Aspects of Automated Reasoning, 2020.","title":"Directed graph networks for logical reasoning","authors":"Michael Rawson, Giles Reger","year":"2020","venue":"Workshop on Practical Aspects of Automated Reasoning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"Rawson and Reger, EasyChair 2020]","title":"EasyChair","authors":"Reger Rawson","year":"2020","venue":"EasyChair","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"Michael Rawson and Giles Reger. Directed graph networks for logical entail- ment. EasyChair Preprint no. 2185, EasyChair, 2020.","title":"Directed graph networks for logical entailment","authors":"Michael Rawson, Giles Reger","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Robinson, 1965] John Alan Robinson. A machine-oriented logic based on the resolution principle. Journal of the ACM (JACM), 12(1):23-41, 1965.","title":"A machine-oriented logic based on the resolution principle","authors":"John Alan, Robinson","year":"1965","venue":"Journal of the ACM (JACM)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Schulz, 2002] Stephan Schulz. E-a brainiac theorem prover. AI Communications, 15(2, 3):111-126, 2002.","title":"E-a brainiac theorem prover","authors":"Stephan Schulz","year":"2002","venue":"AI Communications","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Schulz, 2017] Stephan Schulz. We know (nearly) nothing! but can we learn? In 1st International Workshop on Au- tomated Reasoning: Challenges, Applications, Directions, Exemplary Achievements, 2017.","title":"We know (nearly) nothing! but can we learn?","authors":"Stephan Schulz","year":"2017","venue":"1st International Workshop on Automated Reasoning: Challenges, Applications, Directions, Exemplary Achievements","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Suda, 2021] Martin Suda. Vampire with a brain is a good ITP hammer. In International Symposium on Frontiers of Combining Systems, pages 192-209. Springer, 2021.","title":"Vampire with a brain is a good ITP hammer","authors":"Martin Suda","year":"2021","venue":"International Symposium on Frontiers of Combining Systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Valenzano et al., 2010] Richard Valenzano, Nathan Sturte- vant, Jonathan Schaeffer, Karen Buro, and Akihiro Kishi- moto. Simultaneously searching with multiple settings: An alternative to parameter tuning for suboptimal single- agent search algorithms. In Proceedings of the 20th Inter- national Conference on Automated Planning and Schedul- ing, pages 177-184, 2010.","title":"Simultaneously searching with multiple settings: An alternative to parameter tuning for suboptimal singleagent search algorithms","authors":"Richard Valenzano, Nathan Sturtevant, Jonathan Schaeffer, Karen Buro, Akihiro Kishimoto","year":"2010","venue":"Proceedings of the 20th International Conference on Automated Planning and Scheduling","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang et al., 2017] Mingzhe Wang, Yihe Tang, Jian Wang, and Ji Deng. Premise selection for theorem proving by deep graph embedding. In Proceedings of the 31st Con- ference on Neural Information Processing Systems, pages 2783-2793, 2017.","title":"Premise selection for theorem proving by deep graph embedding","authors":"Wang","year":"2017","venue":"Proceedings of the 31st Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zombori et al., 2020] Zsolt Zombori, Josef Urban, and Chad E Brown. Prolog technology reinforcement learning prover: (system description). In Automated Reasoning: 10th International Joint Conference, IJCAR 2020, Paris, France, July 1-4, 2020, Proceedings, Part II, pages 489- 507. Springer, 2020.","title":"Prolog technology reinforcement learning prover: (system description)","authors":"Zsolt Zombori, Josef Urban, Chad E Brown","year":"2020","venue":"Automated Reasoning: 10th International Joint Conference, IJCAR 2020","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}