{"type":"ScholarlyArticle","title":"Neuro-Symbolic Verification of Deep Neural Networks","authors":[{"name":"Xuan Xie","slug":"xuan-xie","given":"Xuan","family":"Xie","affiliation":"Max Planck Institute for Software Systems , Kaiserslautern , Germany"},{"name":"Kristian Kersting","slug":"kristian-kersting","given":"Kristian","family":"Kersting","affiliation":"Computer Science Department and Centre for Cognitive Science , TU Darmstadt , Germany; Hessian Center for AI (hessian.AI) , Germany"},{"name":"Daniel Neider","slug":"daniel-neider","given":"Daniel","family":"Neider","affiliation":"Max Planck Institute for Software Systems , Kaiserslautern , Germany; Carl von Ossietzky University of Oldenburg , Germany"}],"abstract":"Formal verification has emerged as a powerful approach to ensure the safety and reliability of deep neural networks. However, current verification tools are limited to only a handful of properties that can be expressed as first-order constraints over the inputs and output of a network. While adversarial robustness and fairness fall under this category, many real-world properties (e.g., \"an autonomous vehicle has to stop in front of a stop sign\") remain outside the scope of existing verification technology. To mitigate this severe practical restriction, we introduce a novel framework for verifying neural networks, named neuro-symbolic verification. The key idea is to use neural networks as part of the otherwise logical specification, enabling the verification of a wide variety of complex, real-world properties, including the one above. A defining feature of our framework is that it can be implemented on top of existing verification infrastructure for neural networks, making it easily accessible to researchers and practitioners.","doi":"10.24963/ijcai.2022/503","url":"https://ijcai.org/proceedings/2022/503","pdf":"https://ijcai.org/proceedings/2022/0503.pdf","markdown":"https://ijcai.org/proceedings/2022/503.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/503","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":503,"pages":{"first":3622,"last":3628},"keywords":["Multidisciplinary Topics and Applications: Validation and Verification","Constraint Satisfaction and Optimization: Satisfiabilty","Machine Learning: Neuro-Symbolic Methods"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986886","length_sec":85,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985029","length_sec":339,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Albarghouthi, 2021] Aws Albarghouthi. Introduction to Neu- ral Network Verification. verifieddeeplearning.com, 2021.","title":"Aws Albarghouthi. Introduction to Neural Network Verification","authors":"Albarghouthi","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arabshahi et al., 2021] Forough Arabshahi, Jennifer Lee, Mikayla Gawarecki, Kathryn Mazaitis, Amos Azaria, and Tom M. Mitchell. Conversational neuro-symbolic common- sense reasoning. In AAAI, pages 4902-4911, 2021.","title":"Conversational neuro-symbolic commonsense reasoning","authors":"Forough Arabshahi, Jennifer Lee, Mikayla Gawarecki, Kathryn Mazaitis, Amos Azaria, Tom M Mitchell","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Batten et al., 2021] Ben Batten, Panagiotis Kouvaros, Alessio Lomuscio, and Yang Zheng. Efficient neural net- work verification via layer-based semidefinite relaxations and linear cuts. In IJCAI, pages 2184-2190, 2021.","title":"Efficient neural network verification via layer-based semidefinite relaxations and linear cuts","authors":"Ben Batten, Panagiotis Kouvaros, Alessio Lomuscio, Yang Zheng","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/301"},{"position":4,"raw":"[Cousot and Cousot, 1977] Patrick Cousot and Radhia Cousot. Abstract interpretation: A unified lattice model for static analysis of programs by construction or approx. of fixpoints. In POPL, pages 238-252, 1977.","title":"Abstract interpretation: A unified lattice model for static analysis of programs by construction or approx. of fixpoints","authors":"Cousot Cousot, Patrick Cousot, Radhia Cousot","year":"1977","venue":"POPL","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"['Avila Garcez and Lamb, 2020] Artur d'Avila Garcez and Luís C. Lamb. Neurosymbolic AI: the 3rd wave. CoRR, abs/2012.05876, 2020.","title":"Neurosymbolic AI: the 3rd wave","authors":"Avila Garcez, Lamb ; Artur D'avila, Luís C Garcez, Lamb","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"['Avila Garcez et al., 2009] Artur S. d'Avila Garcez, Luís C. Lamb, and Dov M. Gabbay. Neural-Symbolic Cognitive Reasoning. Cognitive Technologies. Springer, 2009.","title":"Neural-Symbolic Cognitive Reasoning","authors":"Artur S Garcez, Luís C Lamb, Dov M Gabbay","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"['Avila Garcez et al., 2019] Artur S. d'Avila Garcez, Marco Gori, Luís C. Lamb, Luciano Serafini, Michael Spranger, and Son N. Tran. Neural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning. FLAP, 6(4):611-632, 2019.","title":"Neural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning","authors":"Artur S Garcez, Marco Gori, Luís C Lamb, Luciano Serafini, Michael Spranger, Son N Tran","year":"2019","venue":"FLAP","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[de Moura and Bjørner, 2008] Leonardo Mendonc ¸a de Moura and Nikolaj Bjørner. Z3: an efficient SMT solver. In TACAS, volume 4963 of LNCS, pages 337-340, 2008.","title":"Leonardo Mendonc ¸a de Moura and Nikolaj Bjørner. Z3: an efficient SMT solver","authors":"De Moura, Bjørner","year":"2008","venue":"TACAS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[De Raedt et al., 2020] Luc De Raedt, Sebastijan Dumancic, Robin Manhaeve, and Giuseppe Marra. From statistical relational to neuro-symbolic artificial intelligence. In IJCAI, pages 4943-4950, 2020.","title":"From statistical relational to neuro-symbolic artificial intelligence","authors":"De Raedt","year":"2020","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/688"},{"position":10,"raw":"[Ehlers, 2017] Rüdiger Ehlers. Formal verification of piece- wise linear feed-forward neural networks. In ATVA, volume 10482 of LNCS, pages 269-286, 2017.","title":"Formal verification of piecewise linear feed-forward neural networks","authors":"Rüdiger Ehlers","year":"2017","venue":"ATVA","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gehr et al., 2018] Timon Gehr, Matthew Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri, and Martin T. Vechev. AI2: safety and robustness certification of neural networks with abstract interpretation. In SP, pages 3-18, 2018.","title":"Vechev. AI2: safety and robustness certification of neural networks with abstract interpretation","authors":"Timon Gehr, Matthew Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri, Martin T","year":"2018","venue":"SP","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gowal et al., 2019] Sven Gowal, Krishnamurthy Dvijotham, Robert Stanforth, Timothy A. Mann, and Pushmeet Kohli. A dual approach to verify and train deep networks. In IJCAI, pages 6156-6160, 2019.","title":"A dual approach to verify and train deep networks","authors":"Sven Gowal, Krishnamurthy Dvijotham, Robert Stanforth, Timothy A Mann, Pushmeet Kohli","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/854"},{"position":13,"raw":"[Henriksen and Lomuscio, 2021] Patrick Henriksen and Alessio Lomuscio. DEEPSPLIT: an efficient splitting method for neural network verification via indirect effect analysis. In IJCAI, pages 2549-2555, 2021.","title":"DEEPSPLIT: an efficient splitting method for neural network verification via indirect effect analysis","authors":"Lomuscio Henriksen, Patrick Henriksen, Alessio Lomuscio","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/351"},{"position":14,"raw":"[Katz et al., 2019] Guy Katz, Derek A. Huang, Duligur Ibel- ing, Kyle Julian, Christopher Lazarus, Rachel Lim, Parth Shah, Shantanu Thakoor, Haoze Wu, Aleksandar Zeljic, David L. Dill, Mykel J. Kochenderfer, and Clark W. Barrett. The marabou framework for verification and analysis of deep neural networks. In CAV, volume 11561 of LNCS, pages 443-452, 2019.","title":"The marabou framework for verification and analysis of deep neural networks","authors":"Guy Katz, Derek A Huang, Duligur Ibeling, Kyle Julian, Christopher Lazarus, Rachel Lim, Parth Shah, Shantanu Thakoor, Haoze Wu, Aleksandar Zeljic, David L Dill, J Mykel, Clark W Kochenderfer, Barrett","year":"2019","venue":"CAV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[LeCun et al., 2010] Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs. Available: http:// yann.lecun.com/ exdb/ mnist, 2, 2010.","title":"Mnist handwritten digit database","authors":"Yann Lecun, Corinna Cortes, Burges","year":"2010","venue":null,"doi":null,"url":"http://yann.lecun.com/exdb/mnist,2","internal_url":null},{"position":16,"raw":"[McGregor, 2021] Sean McGregor. Preventing repeated real world AI failures by cataloging incidents: The AI incident database. In AAAI, pages 15458-15463, 2021.","title":"Preventing repeated real world AI failures by cataloging incidents: The AI incident database","authors":"Sean Mcgregor","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sakurada and Yairi, 2014] Mayu Sakurada and Takehisa Yairi. Anomaly detection using autoencoders with non- linear dimensionality reduction. In MLSDA, page 4, 2014.","title":"Anomaly detection using autoencoders with nonlinear dimensionality reduction","authors":"Yairi ; ] Mayu Sakurada, Takehisa Sakurada, Yairi","year":"2014","venue":"MLSDA","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Seshia et al., 2018] Sanjit A. Seshia, Ankush Desai, Tom- maso Dreossi, Daniel J. Fremont, Shromona Ghosh, Edward Kim, Sumukh Shivakumar, Marcell Vazquez- Chanlatte, and Xiangyu Yue. Formal specification for deep neural networks. In ATVA, volume 11138 of LNCS, pages 20-34, 2018.","title":"Formal specification for deep neural networks","authors":"A Sanjit, Ankush Seshia, Tommaso Desai, Daniel J Dreossi, Shromona Fremont, Edward Ghosh, Sumukh Kim, Marcell Shivakumar, Xiangyu Vazquez-Chanlatte, Yue","year":"2018","venue":"ATVA","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Singh et al., 2018] Gagandeep Singh, Timon Gehr, Matthew Mirman, Markus Püschel, and Martin T. Vechev. Fast and effective robustness certification. In NeurIPS, pages 10825- 10836, 2018.","title":"Fast and effective robustness certification","authors":"Gagandeep Singh, Timon Gehr, Matthew Mirman, Markus Püschel, Martin T Vechev","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Stallkamp et al., 2011] Johannes Stallkamp, Marc Schlips- ing, Jan Salmen, and Christian Igel. The german traffic sign recognition benchmark: A multi-class classification competition. In IJCNN, pages 1453-1460, 2011.","title":"The german traffic sign recognition benchmark: A multi-class classification competition","authors":"Johannes Stallkamp, Marc Schlipsing, Jan Salmen, Christian Igel","year":"2011","venue":"IJCNN","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Stammer et al., 2021] Wolfgang Stammer, Patrick Schramowski, and Kristian Kersting. Right for the right concept: Revising neuro-symbolic concepts by interacting with their explanations. In CVPR, pages 3619-3629, 2021.","title":"Right for the right concept: Revising neuro-symbolic concepts by interacting with their explanations","authors":"Stammer","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sun et al., 2018] Youcheng Sun, Min Wu, Wenjie Ruan, Xi- aowei Huang, Marta Kwiatkowska, and Daniel Kroening. Concolic testing for deep neural networks. In ASE, pages 109-119, 2018.","title":"Concolic testing for deep neural networks","authors":"Sun","year":"2018","venue":"ASE","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Szegedy et al., 2014] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Good- fellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, 2014.","title":"Intriguing properties of neural networks","authors":"Szegedy","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xie et al., 2022] Xuan Xie, Kristian Kersting, and Daniel Neider. Neuro-symbolic verification of deep neural net- works. CoRR, abs/2203.00938, 2022.","title":"Neuro-symbolic verification of deep neural networks","authors":"Xuan Xie, Kristian Kersting, Daniel Neider","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yang et al., 2021] Pengfei Yang, Jianlin Li, Jiangchao Liu, Cheng-Chao Huang, Renjue Li, Liqian Chen, Xiaowei Huang, and Lijun Zhang. Enhancing robustness verifi- cation for deep neural networks via symbolic propagation. Formal Aspects Comput., 33(3):407-435, 2021.","title":"Enhancing robustness verification for deep neural networks via symbolic propagation","authors":"Pengfei Yang, Jianlin Li, Jiangchao Liu, Cheng-Chao Huang, Renjue Li, Liqian Chen, Xiaowei Huang, Lijun Zhang","year":"2021","venue":"Formal Aspects Comput","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yi et al., 2018] Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli, and Josh Tenenbaum. Neural- symbolic VQA: disentangling reasoning from vision and language understanding. In NeurIPS, pages 1039-1050, 2018.","title":"Neuralsymbolic VQA: disentangling reasoning from vision and language understanding","authors":"Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli, Josh Tenenbaum","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}