{"type":"ScholarlyArticle","title":"Neuro-Symbolic Class Expression Learning","authors":[{"name":"Caglar Demir","slug":"caglar-demir","given":"Caglar","family":"Demir","affiliation":"Data Science Research Group , Paderborn University"},{"name":"Axel-Cyrille Ngonga Ngomo","slug":"axel-cyrille-ngonga-ngomo","given":"Axel-Cyrille","family":"Ngonga Ngomo","affiliation":"Data Science Research Group , Paderborn University"}],"abstract":"Models computed using deep learning have been effectively applied to tackle various problems in many disciplines. Yet, the predictions of these models are often at most post-hoc and locally explainable. \r\nIn contrast, class expressions in description logics are ante-hoc and globally explainable. Although state-of-the-art symbolic machine learning approaches are being successfully applied to learn class expressions, their application at large scale has been hindered by their impractical runtimes. Arguably, the reliance on myopic heuristic functions contributes to this limitation. We propose a novel neuro-symbolic class expression learning model, DRILL, to mitigate this limitation. By learning non-myopic heuristic functions with deep Q-learning, DRILL efficiently steers the standard search procedure in a quasi-ordered search space towards goal states. Our extensive experiments on 4 benchmark datasets and 390 learning problems suggest that DRILL converges to goal states at least 2.7 times faster than state-of-the-art models on all learning problems. The results of our statistical significance test confirms that DRILL converges to goal states significantly faster (p-value <1%) than state-of-the-art models on all benchmark datasets. We provide an open-source implementation of DRILL, including pre-trained models, training and evaluation scripts.","doi":"10.24963/ijcai.2023/403","url":"https://ijcai.org/proceedings/2023/403","pdf":"https://ijcai.org/proceedings/2023/0403.pdf","markdown":"https://ijcai.org/proceedings/2023/403.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/403","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":403,"pages":{"first":3624,"last":3632},"keywords":["Knowledge Representation and Reasoning: KRR: Description logics and ontologies","Knowledge Representation and Reasoning: KRR: Learning and reasoning","Machine Learning: ML: Neuro-symbolic methods","Machine Learning: ML: Deep reinforcement learning"],"videos":[],"references":[{"position":1,"raw":"[Badea and Nienhuys-Cheng, 2000] Liviu Badea and Shan Hwei Nienhuys-Cheng. A refinement operator for description logics. In James Cussens and Alan Frisch, editors, Inductive Logic Programming, pages 40-59, Berlin, Heidelberg, 2000. Springer Berlin Heidelberg.","title":"A refinement operator for description logics","authors":"Liviu Badea, Shan Hwei Nienhuys-Cheng","year":"2000","venue":"Inductive Logic Programming","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bin et al., 2016] Simon Bin, Lorenz Bühmann, Jens Lehmann, and Axel-Cyrille Ngonga Ngomo. Towards sparql-based induction for large-scale rdf data sets. In Proceedings of the Twenty-second European Conference on Artificial Intelligence, pages 1551-1552. IOS Press, 2016.","title":"Towards sparql-based induction for large-scale rdf data sets","authors":"Simon Bin, Jens Lorenz Bühmann, Axel-Cyrille Ngonga Lehmann, Ngomo","year":"2016","venue":"Proceedings of the Twenty-second European Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bizer et al., 2022] Christian Bizer, Robert Meusel, Anna Primpeli, and Alexander Brinkmann. Web data commons -rdfa, microdata, embedded json-ld, and microformats data sets -october 2022. https://webdatacommons.org/ structureddata/2022-12/stats/stats.html, 2022. Accessed: 2023-05-15.","title":"Web data commons -rdfa, microdata, embedded json-ld, and microformats data sets -october 2022","authors":"Bizer","year":"2022","venue":null,"doi":null,"url":"https://webdatacommons.org/structureddata/2022-12/stats/stats.html,2022","internal_url":null},{"position":4,"raw":"[Bühmann et al., 2018] Lorenz Bühmann, Jens Lehmann, Patrick Westphal, and Simon Bin. Dl-learner structured machine learning on semantic web data. In Companion Proceedings of the The Web Conference 2018, WWW '18, pages 467-471, Republic and Canton of Geneva, Switzer- land, 2018. International World Wide Web Conferences Steering Committee.","title":"Dl-learner structured machine learning on semantic web data","authors":"Jens Lorenz Bühmann, Patrick Lehmann, Simon Westphal, Bin","year":"2018","venue":"Companion Proceedings of the The Web Conference 2018, WWW '18","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Demir and Ngomo, 2021] Caglar Demir and Axel- Cyrille Ngonga Ngomo. Convolutional complex knowledge graph embeddings. In European Semantic Web Conference, pages 409-424. Springer, 2021.","title":"Convolutional complex knowledge graph embeddings","authors":"Caglar Demir, Axel-Cyrille Ngonga Ngomo","year":"2021","venue":"European Semantic Web Conference","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"['Amato, 2020] Claudia d'Amato. Machine learning for the semantic web: Lessons learnt and next research directions. Semantic Web, 11(1):195-203, 2020.","title":"Machine learning for the semantic web: Lessons learnt and next research directions","authors":"Claudia Amato, Amato","year":"2020","venue":"Semantic Web","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Edwards et al., 2020] Ashley D. Edwards, Himanshu Sahni, Rosanne Liu, Jane Hung, Ankit Jain, Rui Wang, Adrien Ecoffet, Thomas Miconi, Charles Isbell, and Jason Yosinski. Estimating q(s,s') with deep deterministic dynamics gradi- ents. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 2825-2835. PMLR, 2020.","title":"Estimating q(s,s') with deep deterministic dynamics gradients","authors":"Ashley D Edwards, Himanshu Sahni, Rosanne Liu, Jane Hung, Ankit Jain, Rui Wang, Adrien Ecoffet, Thomas Miconi, Charles Isbell, Jason Yosinski","year":"2020","venue":"Proceedings of the 37th International Conference on Machine Learning, ICML 2020","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fanizzi et al., 2018] Nicola Fanizzi, Giuseppe Rizzo, Clau- dia d'Amato, and Floriana Esposito. Dlfoil: Class expres- sion learning revisited. In European Knowledge Acquisition Workshop, pages 98-113. Springer, 2018.","title":"Dlfoil: Class expression learning revisited","authors":"Nicola Fanizzi, Giuseppe Rizzo, Claudia Amato, Floriana Esposito","year":"2018","venue":"European Knowledge Acquisition Workshop","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hitzler et al., 2009] Pascal Hitzler, Markus Krotzsch, and Sebastian Rudolph. Foundations of semantic web technolo- gies. 2009.","title":"Foundations of semantic web technologies","authors":"Pascal Hitzler, Markus Krotzsch, Sebastian Rudolph","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hitzler et al., 2020] Pascal Hitzler, Federico Bianchi, Monireh Ebrahimi, and Md Kamruzzaman Sarker. Neural- symbolic integration and the semantic web. Semantic Web, 11(1):3-11, 2020.","title":"Neuralsymbolic integration and the semantic web","authors":"Hitzler","year":"2020","venue":"Semantic Web","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hogan et al., 2021] Aidan Hogan, Eva Blomqvist, Michael Cochez, Claudia d'Amato, Gerard de Melo, Claudio Gutier- rez, Sabrina Kirrane, José Emilio Labra Gayo, Roberto Navigli, Sebastian Neumaier, et al. Knowledge graphs. ACM Computing Surveys (CSUR), 54(4):1-37, 2021.","title":"Knowledge graphs","authors":"Aidan Hogan, Eva Blomqvist, Michael Cochez, Claudia Amato, Gerard De Melo, Claudio Gutierrez, Sabrina Kirrane, José Emilio Labra, Roberto Gayo, Sebastian Navigli, Neumaier","year":"2021","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Holzinger et al., 2019] Andreas Holzinger, Georg Langs, Helmut Denk, Kurt Zatloukal, and Heimo Müller. Causabil- ity and explainability of artificial intelligence in medicine. Wiley Interdisciplinary Reviews: Data Mining and Knowl- edge Discovery, 9(4):e1312, 2019.","title":"Causability and explainability of artificial intelligence in medicine","authors":"Holzinger","year":"2019","venue":"Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Horrocks et al., 2006] Ian Horrocks, Oliver Kutz, and Ulrike Sattler. The even more irresistible sroiq. Kr, 6:57-67, 2006.","title":"The even more irresistible sroiq","authors":"Ian Horrocks, Oliver Kutz, Ulrike Sattler","year":"2006","venue":"Kr","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Iannone et al., 2007] Luigi Iannone, Ignazio Palmisano, and Nicola Fanizzi. An algorithm based on counterfactuals for concept learning in the semantic web. Applied Intelligence, 26(2):139-159, 2007.","title":"An algorithm based on counterfactuals for concept learning in the semantic web","authors":"Iannone","year":"2007","venue":"Applied Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jumper et al., 2021] John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583-589, 2021.","title":"Highly accurate protein structure prediction with alphafold","authors":"John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko","year":"2021","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kulmanov et al., 2018] Maxat Kulmanov, Mohammed Asif Khan, and Robert Hoehndorf. Deepgo: predicting pro- tein functions from sequence and interactions using a deep ontology-aware classifier. Bioinformatics, 34(4):660-668, 2018.","title":"Maxat Kulmanov, Mohammed Asif Khan, and Robert Hoehndorf. Deepgo: predicting protein functions from sequence and interactions using a deep ontology-aware classifier","authors":"Kulmanov","year":"2018","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lehmann and Hitzler, 2010] Jens Lehmann and Pascal Hit- zler. Concept learning in description logics using refine- ment operators. Machine Learning, 78(1-2):203, 2010.","title":"Jens Lehmann and Pascal Hitzler. Concept learning in description logics using refinement operators","authors":"Hitzler Lehmann","year":"2010","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lehmann et al., 2011] Jens Lehmann, Sören Auer, Lorenz Bühmann, and Sebastian Tramp. Class expression learn- ing for ontology engineering. Journal of Web Semantics, 9(1):71-81, 2011.","title":"Class expression learning for ontology engineering","authors":"Lehmann","year":"2011","venue":"Journal of Web Semantics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lehmann, 2009] Jens Lehmann. Dl-learner: learning con- cepts in description logics. The Journal of Machine Learn- ing Research, 10:2639-2642, 2009.","title":"Dl-learner: learning concepts in description logics","authors":"Jens Lehmann, Lehmann","year":"2009","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lehmann, 2010] Jens Lehmann. Learning OWL class ex- pressions, volume 22. IOS Press, 2010.","title":"Learning OWL class expressions","authors":"Jens Lehmann, Lehmann","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Li et al., 2017] Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. Hyperband: A novel bandit-based approach to hyperparameter opti- mization. The Journal of Machine Learning Research, 18(1):6765-6816, 2017.","title":"Hyperband: A novel bandit-based approach to hyperparameter optimization","authors":"Lisha Li, Kevin Jamieson, Giulia Desalvo, Afshin Rostamizadeh, Ameet Talwalkar","year":"2017","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Mnih et al., 2015] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Belle- mare, Alex Graves, Martin Riedmiller, Andreas K Fidje- land, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. nature, 518(7540):529-533, 2015.","title":"Human-level control through deep reinforcement learning","authors":"Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski","year":"2015","venue":"nature","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Oramas et al., 2016] Sergio Oramas, Vito Claudio Ostuni, Tommaso Di Noia, Xavier Serra, and Eugenio Di Sciascio. Sound and music recommendation with knowledge graphs. ACM Transactions on Intelligent Systems and Technology (TIST), 8(2):1-21, 2016.","title":"Sound and music recommendation with knowledge graphs","authors":"Sergio Oramas, Claudio Vito, Tommaso Ostuni, Xavier Di Noia, Eugenio Di Serra, Sciascio","year":"2016","venue":"ACM Transactions on Intelligent Systems and Technology (TIST)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Riedmiller, 2005] Martin Riedmiller. Neural fitted q iteration-first experiences with a data efficient neural re- inforcement learning method. In European conference on machine learning, pages 317-328. Springer, 2005.","title":"Neural fitted q iteration-first experiences with a data efficient neural reinforcement learning method","authors":"Martin Riedmiller","year":"2005","venue":"European conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Samek and Müller, 2019] Wojciech Samek and Klaus- Robert Müller. Towards explainable artificial intelligence. In Explainable AI: interpreting, explaining and visualizing deep learning, pages 5-22. Springer, 2019.","title":"Towards explainable artificial intelligence","authors":"Müller ; Samek, Klaus-Robert Müller","year":"2019","venue":"Explainable AI: interpreting, explaining and visualizing deep learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sarker and Hitzler, 2019] Md Kamruzzaman Sarker and Pas- cal Hitzler. Efficient concept induction for description logics. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 3036-3043, 2019. [Statista, 2023] Statista. Number of internet and so- cial media users worldwide as of january 2023 (in billions). https://www.statista.com/statistics/617136/ digital-population-worldwide/, 2023. Accessed: 2023-05- 15.","title":"Efficient concept induction for description logics","authors":"Kamruzzaman Md, Pascal Sarker, Hitzler","year":"2019","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":"https://www.statista.com/statistics/617136/digital-population-worldwide/,2023","internal_url":null},{"position":27,"raw":"[Sutton and Barto, 2018] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.","title":"Reinforcement learning: An introduction","authors":"S Richard, Andrew G Sutton, Barto","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Tran et al., 2017] An C. Tran, Jens Dietrich, Hans W. Gues- gen, and Stephen Marsland. Parallel symmetric class ex- pression learning. J. Mach. Learn. Res., 18:64:1-64:34, 2017.","title":"Parallel symmetric class expression learning","authors":"Tran","year":"2017","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}