{"type":"ScholarlyArticle","title":"Neuro-Symbolic Learning of Answer Set Programs from Raw Data","authors":[{"name":"Daniel Cunnington","slug":"daniel-cunnington","given":"Daniel","family":"Cunnington","affiliation":"IBM Research Europe; Imperial College London"},{"name":"Mark Law","slug":"mark-law","given":"Mark","family":"Law","affiliation":"ILASP Limited"},{"name":"Jorge Lobo","slug":"jorge-lobo","given":"Jorge","family":"Lobo","affiliation":"ICREA-Universitat Pompeu Fabra"},{"name":"Alessandra Russo","slug":"alessandra-russo","given":"Alessandra","family":"Russo","affiliation":"Imperial College London"}],"abstract":"One of the ultimate goals of Artificial Intelligence is to assist humans in complex decision making. A promising direction for achieving this goal is Neuro-Symbolic AI, which aims to combine the interpretability of symbolic techniques with the ability of deep learning to learn from raw data. However, most current approaches require manually engineered symbolic knowledge, and where end-to-end training is considered, such approaches are either restricted to learning definite programs, or are restricted to training binary neural networks. In this paper, we introduce Neuro-Symbolic Inductive Learner (NSIL), an approach that trains a general neural network to extract latent concepts from raw data, whilst learning symbolic knowledge that maps latent concepts to target labels. The novelty of our approach is a method for biasing the learning of symbolic knowledge, based on the in-training performance of both neural and symbolic components. We evaluate NSIL on three problem domains of different complexity, including an NP-complete problem. Our results demonstrate that NSIL learns expressive knowledge, solves computationally complex problems, and achieves state-of-the-art performance in terms of accuracy and data efficiency. Code and technical appendix: https://github.com/DanCunnington/NSIL","doi":"10.24963/ijcai.2023/399","url":"https://ijcai.org/proceedings/2023/399","pdf":"https://ijcai.org/proceedings/2023/0399.pdf","markdown":"https://ijcai.org/proceedings/2023/399.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/399","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":399,"pages":{"first":3586,"last":3596},"keywords":["Machine Learning: ML: Neuro-symbolic methods","Knowledge Representation and Reasoning: KRR: Learning and reasoning","Knowledge Representation and Reasoning: KRR: Logic programming"],"videos":[],"references":[{"position":1,"raw":"[An et al., 2020] Sanghyeon An, Minjun Lee, Sanglee Park, Heerin Yang, and Jungmin So. An ensemble of simple convolutional neural network models for mnist digit recog- nition. arXiv preprint arXiv:2008.10400, 2020.","title":"An ensemble of simple convolutional neural network models for mnist digit recognition","authors":"An","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aspis et al., 2022] Yaniv Aspis, Krysia Broda, Jorge Lobo, and Alessandra Russo. Embed2sym-scalable neuro- symbolic reasoning via clustered embeddings. In Pro- ceedings of the International Conference on Principles of Knowledge Representation and Reasoning, volume 19, pages 421-431, 2022.","title":"Embed2sym-scalable neurosymbolic reasoning via clustered embeddings","authors":"Yaniv Aspis, Krysia Broda, Jorge Lobo, Alessandra Russo","year":"2022","venue":"Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Badreddine et al., 2022] Samy Badreddine, Artur d'Avila Garcez, Luciano Serafini, and Michael Spranger. Logic tensor networks. Artificial Intelligence, 303:103649, 2022.","title":"Logic tensor networks","authors":"Samy Badreddine, Artur D'avila, Luciano Garcez, Michael Serafini, Spranger","year":"2022","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Besold et al., 2022] Tarek R Besold, Artur d'Avila Garcez, Sebastian Bader, Howard Bowman, Luis C Lamb, Leo de Penning, BV Illuminoo, Hoifung Poon, and COPPE Gerson Zaverucha. Neural-symbolic learning and reason- ing: A survey and interpretation. Neuro-Symbolic Artificial Intelligence: The State of the Art, 342:1, 2022.","title":"Neural-symbolic learning and reasoning: A survey and interpretation","authors":"Artur D'avila Tarek R Besold, Sebastian Garcez, Howard Bader, Luis C Bowman, Leo Lamb, De Penning, Hoifung Illuminoo, Poon, Coppe Gerson, Zaverucha","year":"2022","venue":"Neuro-Symbolic Artificial Intelligence: The State of the Art","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Corapi et al., 2010] Domenico Corapi, Alessandra Russo, and Emil Lupu. Inductive logic programming as abductive search. In Technical communications of the 26th inter- national conference on logic programming, pages 54-63.","title":"Inductive logic programming as abductive search","authors":"Domenico Corapi, Alessandra Russo, Emil Lupu","year":null,"venue":"Technical communications of the 26th international conference on logic programming","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cunnington et al., 2023] Daniel Cunnington, Mark Law, Jorge Lobo, and Alessandra Russo. Ffnsl: Feed-forward neural-symbolic learner. Machine Learning, pages 1-55, 2023.","title":"Ffnsl: Feed-forward neural-symbolic learner","authors":"Mark Daniel Cunnington, Jorge Law, Alessandra Lobo, Russo","year":"2023","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dai and Muggleton, 2021] Wang-Zhou Dai and Stephen Muggleton. Abductive knowledge induction from raw data. In Zhi-Hua Zhou, editor, Proceedings of the Thirtieth Inter- national Joint Conference on Artificial Intelligence, IJCAI- 21, pages 1845-1851. International Joint Conferences on Artificial Intelligence Organization, 8 2021. Main Track.","title":"Abductive knowledge induction from raw data","authors":"Wang-Zhou Dai, Stephen Muggleton","year":"2021","venue":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21","doi":null,"url":null,"internal_url":"/proceedings/2021/254"},{"position":8,"raw":"[Dai et al., 2019] Wang-Zhou Dai, Qiuling Xu, Yang Yu, and Zhi-Hua Zhou. Bridging machine learning and logical reasoning by abductive learning. Advances in Neural Infor- mation Processing Systems, 32, 2019.","title":"Bridging machine learning and logical reasoning by abductive learning","authors":"Wang-Zhou Dai, Qiuling Xu, Yang Yu, Zhi-Hua Zhou","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Daniele et al., 2022] Alessandro Daniele, Tommaso Cam- pari, Sagar Malhotra, and Luciano Serafini. Deep symbolic learning: Discovering symbols and rules from perceptions. arXiv preprint arXiv:2208.11561, 2022.","title":"Deep symbolic learning: Discovering symbols and rules from perceptions","authors":"Alessandro Daniele, Tommaso Campari, Sagar Malhotra, Luciano Serafini","year":"2022","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2023/400"},{"position":10,"raw":"[Dantsin et al., 2001] Evgeny Dantsin, Thomas Eiter, Georg Gottlob, and Andrei Voronkov. Complexity and expressive power of logic programming. ACM Computing Surveys (CSUR), 33(3):374-425, 2001.","title":"Complexity and expressive power of logic programming","authors":"Evgeny Dantsin, Thomas Eiter, Georg Gottlob, Andrei Voronkov","year":"2001","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Dasaratha et al., 2023] Sridhar Dasaratha, Sai Akhil Pu- ranam, Karmvir Singh Phogat, Sunil Reddy Tiyyagura, and Nigel P. Duffy. Deeppsl: End-to-end perception and reasoning, 2023.","title":"Deeppsl: End-to-end perception and reasoning","authors":"Sridhar Dasaratha, Akhil Sai, Karmvir Puranam, Sunil Singh Phogat, Nigel P Reddy Tiyyagura, Duffy","year":"2023","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2023/401"},{"position":12,"raw":"['Avila Garcez et al., 2019] Artur d'Avila Garcez, Marco Gori, Luís C. Lamb, Luciano Serafini, Michael Spranger, and Son N. Tran. Neural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning. FLAP, 6(4):611-632, 2019.","title":"Neural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning","authors":"Avila Garcez","year":"2019","venue":"FLAP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[De Raedt et al., 2020] Luc De Raedt, Sebastijan Dumančić, Robin Manhaeve, and Giuseppe Marra. From statistical relational to neuro-symbolic artificial intelligence. In Chris- tian Bessiere, editor, Proceedings of the Twenty-Ninth Inter- national Joint Conference on Artificial Intelligence, IJCAI- 20, pages 4943-4950. International Joint Conferences on Artificial Intelligence Organization, 7 2020. Survey track.","title":"From statistical relational to neuro-symbolic artificial intelligence","authors":"De Raedt","year":"2020","venue":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20","doi":null,"url":null,"internal_url":"/proceedings/2020/688"},{"position":14,"raw":"[Evans and Grefenstette, 2018] Richard Evans and Edward Grefenstette. Learning explanatory rules from noisy data. Journal of Artificial Intelligence Research, 61:1-64, 2018.","title":"Learning explanatory rules from noisy data","authors":"Grefenstette Evans, Richard Evans, Edward Grefenstette","year":"2018","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Evans et al., 2021] Richard Evans, Matko Bošnjak, Lars Buesing, Kevin Ellis, David Pfau, Pushmeet Kohli, and Marek Sergot. Making sense of raw input. Artificial Intelli- gence, 299:103521, 2021.","title":"Making sense of raw input","authors":"Richard Evans, Matko Bošnjak, Lars Buesing, Kevin Ellis, David Pfau, Pushmeet Kohli, Marek Sergot","year":"2021","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ferreira et al., 2022] Joao Ferreira, Manuel de Sousa Ribeiro, Ricardo Gonc ¸alves, and Joao Leite. Looking inside the black-box: Logic-based explanations for neural networks. In Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning, volume 19, pages 432-442, 2022.","title":"Looking inside the black-box: Logic-based explanations for neural networks","authors":"Joao Ferreira, Manuel De Sousa Ribeiro, Ricardo Gonc, Joao Leite","year":"2022","venue":"Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Gelfond and Kahl, 2014] Michael Gelfond and Yulia Kahl. Knowledge representation, reasoning, and the design of intelligent agents: The answer-set programming approach. Cambridge University Press, Cambridge, UK, 2014.","title":"Knowledge representation, reasoning, and the design of intelligent agents: The answer-set programming approach","authors":"Kahl Gelfond, Michael Gelfond, Yulia Kahl","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Han et al., 2021] Zhongyi Han, Benzheng Wei, Xiaoming Xi, Bo Chen, Yilong Yin, and Shuo Li. Unifying neural learning and symbolic reasoning for spinal medical report generation. Medical Image Analysis, 67:101872, 2021.","title":"Unifying neural learning and symbolic reasoning for spinal medical report generation","authors":"Zhongyi Han, Benzheng Wei, Xiaoming Xi, Bo Chen, Yilong Yin, Shuo Li","year":"2021","venue":"Medical Image Analysis","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Karp, 1972] Richard M Karp. Reducibility among combina- torial problems. In Complexity of computer computations, pages 85-103. Springer, 1972.","title":"Reducibility among combinatorial problems","authors":"M Richard, Karp","year":"1972","venue":"Complexity of computer computations","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Koh et al., 2020] Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International Conference on Machine Learning, pages 5338-5348, 2020.","title":"Concept bottleneck models","authors":"Pang Wei Koh, Thao Nguyen, Siang Yew, Stephen Tang, Emma Mussmann, Been Pierson, Percy Kim, Liang","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Law et al., 2019] Mark Law, Alessandra Russo, and Krysia Broda. Logic-based learning of answer set programs. In Reasoning Web. Explainable Artificial Intelligence -15th International Summer School 2019, Bolzano, Italy, Septem- ber 20-24, 2019, Tutorial Lectures, pages 196-231, 2019.","title":"Logic-based learning of answer set programs","authors":"Mark Law, Alessandra Russo, Krysia Broda","year":"2019","venue":"Reasoning Web. Explainable Artificial Intelligence -15th International Summer School 2019","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Law et al., 2020] Mark Law, Alessandra Russo, Elisa Bertino, Krysia Broda, and Jorge Lobo. Fastlas: scal- able inductive logic programming incorporating domain- specific optimisation criteria. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 2877-2885, 2020.","title":"Fastlas: scalable inductive logic programming incorporating domainspecific optimisation criteria","authors":"Mark Law, Alessandra Russo, Elisa Bertino, Krysia Broda, Jorge Lobo","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"Mark Law. Inductive learning of answer set programs. PhD thesis, Imperial College London, 2018.","title":"Inductive learning of answer set programs","authors":"Mark Law","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Lyn Paul et al., 2019] Celeste Lyn Paul, Leslie M Blaha, Corey K Fallon, Cleotilde Gonzalez, and Robert S Gutzwiller. Opportunities and challenges for human- machine teaming in cybersecurity operations. In Proceed- ings of the human factors and ergonomics society annual meeting, volume 63, pages 442-446. SAGE Publications Sage CA: Los Angeles, CA, 2019.","title":"Opportunities and challenges for humanmachine teaming in cybersecurity operations","authors":"Celeste, Lyn Paul, Leslie M Blaha, Cleotilde Corey K Fallon, Robert S Gonzalez, Gutzwiller","year":"2019","venue":"Proceedings of the human factors and ergonomics society annual meeting","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Manhaeve et al., 2018] Robin Manhaeve, Sebastijan Duman- cic, Angelika Kimmig, Thomas Demeester, and Luc De Raedt. Deepproblog: Neural probabilistic logic pro- gramming. In Advances in Neural Information Processing Systems, pages 3749-3759, 2018.","title":"Deepproblog: Neural probabilistic logic programming","authors":"Manhaeve","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Muggleton and De Raedt, 1994] Stephen Muggleton and Luc De Raedt. Inductive logic programming: Theory and methods. The Journal of Logic Programming, 19:629-679, 1994.","title":"Inductive logic programming: Theory and methods","authors":"De Muggleton, ; Raedt, Luc Stephen Muggleton, Raedt De","year":"1994","venue":"The Journal of Logic Programming","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Muggleton et al., 2015] Stephen H Muggleton, Dianhuan Lin, and Alireza Tamaddoni-Nezhad. Meta-interpretive learning of higher-order dyadic datalog: Predicate inven- tion revisited. Machine Learning, 100(1):49-73, 2015.","title":"Meta-interpretive learning of higher-order dyadic datalog: Predicate invention revisited","authors":"Dianhuan Stephen H Muggleton, Alireza Lin, Tamaddoni-Nezhad","year":"2015","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Payani and Fekri, 2019] Ali Payani and Faramarz Fekri. In- ductive logic programming via differentiable deep neural logic networks. CoRR, abs/1906.03523, 2019.","title":"Inductive logic programming via differentiable deep neural logic networks","authors":"Fekri Payani, Ali Payani, Faramarz Fekri","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Riegel et al., 2020] Ryan Riegel, Alexander G. Gray, Fran- cois P. S. Luus, Naweed Khan, Ndivhuwo Makondo, Is- mail Yunus Akhalwaya, Haifeng Qian, Ronald Fagin, Francisco Barahona, Udit Sharma, Shajith Ikbal, Hima Karanam, Sumit Neelam, Ankita Likhyani, and San- tosh K. Srivastava. Logical neural networks. CoRR, abs/2006.13155, 2020.","title":"Logical neural networks","authors":"Ryan Riegel, Alexander G Gray, P S Francois, Naweed Luus, Ndivhuwo Khan, Ismail Makondo, Haifeng Yunus Akhalwaya, Ronald Qian, Francisco Fagin, Udit Barahona, Shajith Sharma, Hima Ikbal, Sumit Karanam, Ankita Neelam, Santosh K Likhyani, Srivastava","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Sen et al., 2022] Prithviraj Sen, Breno WSR de Carvalho, Ryan Riegel, and Alexander Gray. Neuro-symbolic induc- tive logic programming with logical neural networks. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 36, pages 8212-8219, 2022.","title":"Neuro-symbolic inductive logic programming with logical neural networks","authors":"Prithviraj Sen, Ryan Breno Wsr De Carvalho, Alexander Riegel, Gray","year":"2022","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Shindo et al., 2021] Hikaru Shindo, Masaaki Nishino, and Akihiro Yamamoto. Differentiable inductive logic pro- gramming for structured examples. Proceedings of the AAAI Conference on Artificial Intelligence, 35(6):5034- 5041, May 2021.","title":"Differentiable inductive logic programming for structured examples","authors":"Hikaru Shindo, Masaaki Nishino, Akihiro Yamamoto","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Sittón et al., 2019] Inés Sittón, Ricardo S Alonso, Elena Hernández-Nieves, Sara Rodríguez-Gonzalez, and Alberto Rivas. Neuro-symbolic hybrid systems for industry 4.0: A systematic mapping study. In International Conference on Knowledge Management in Organizations, pages 455-465. Springer, 2019.","title":"Neuro-symbolic hybrid systems for industry 4.0: A systematic mapping study","authors":"Inés Sittón, Ricardo S Alonso, Elena Hernández-Nieves, Sara Rodríguez-Gonzalez, Alberto Rivas","year":"2019","venue":"International Conference on Knowledge Management in Organizations","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Xu et al., 2018] Jingyi Xu, Zilu Zhang, Tal Friedman, Yitao Liang, and Guy Broeck. A semantic loss function for deep learning with symbolic knowledge. In International conference on machine learning, pages 5502-5511. PMLR, 2018.","title":"A semantic loss function for deep learning with symbolic knowledge","authors":"Xu","year":"2018","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yang et al., 2020] Zhun Yang, Adam Ishay, and Joohyung Lee. Neurasp: Embracing neural networks into answer set programming. In Christian Bessiere, editor, Proceedings of the Twenty-Ninth International Joint Conference on Artifi- cial Intelligence, IJCAI-20, pages 1755-1762. International Joint Conferences on Artificial Intelligence Organization, 7 2020.","title":"Neurasp: Embracing neural networks into answer set programming","authors":"Zhun Yang, Adam Ishay, Joohyung Lee","year":"2020","venue":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20","doi":null,"url":null,"internal_url":"/proceedings/2020/243"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}