{"type":"ScholarlyArticle","title":"Towards Formal Verification of Neuro-symbolic Multi-agent Systems","authors":[{"name":"Panagiotis Kouvaros","slug":"panagiotis-kouvaros","given":"Panagiotis","family":"Kouvaros","affiliation":"Imperial College London , London , UK"}],"abstract":"This paper outlines some of the key methods we developed towards the formal verification of multi- agent systems, covering both symbolic and connectionist systems. It discusses logic-based methods for the verification of unbounded multi-agent systems (i.e., systems composed of an arbitrary number of homogeneous agents, e.g., robot swarms), optimisation approaches for establishing the robustness of neural network models, and methods for analysing properties of neuro-symbolic multi-agent systems.","doi":"10.24963/ijcai.2023/800","url":"https://ijcai.org/proceedings/2023/800","pdf":"https://ijcai.org/proceedings/2023/0800.pdf","markdown":"https://ijcai.org/proceedings/2023/800.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/800","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Early Career","track":"Early Career","number":800,"pages":{"first":7014,"last":7019},"keywords":["EC: Trustworthy AI","EC: Multiagent Systems"],"videos":[],"references":[{"position":1,"raw":"[Akintunde et al., 2020a] M. Akintunde, E. Botoeva, P. Kou- varos, and A. Lomuscio. Formal verification of neural agents in non-deterministic environments. In Proceed- ings of the 19th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS20), page To Ap- pear. ACM, 2020.","title":"Formal verification of neural agents in non-deterministic environments","authors":"Akintunde","year":"2020","venue":"Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS20), page To Appear","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Akintunde et al., 2020b] M. Akintunde, E. Botoeva, P. Kou- varos, and A. Lomuscio. Verifying strategic abilities of neural-symbolic multi-agent systems. In Proceedings of the International Conference on Principles of Knowl- edge Representation and Reasoning (KR20), pages 22-32. AAAI Press, 2020.","title":"Verifying strategic abilities of neural-symbolic multi-agent systems","authors":"Akintunde","year":"2020","venue":"Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning (KR20)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Akintunde et al., 2022] M. Akintunde, E. Botoeva, P. Kou- varos, and A. Lomuscio. Formal verification of neural agents in non-deterministic environments. Journal of Au- tonomous Agents and Multi-Agent Systems, 36(1), 2022.","title":"Formal verification of neural agents in non-deterministic environments","authors":"M Akintunde, E Botoeva, P Kouvaros, A Lomuscio","year":"2022","venue":"Journal of Autonomous Agents and Multi-Agent Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Alur et al., 1998] R. Alur, T. Henzinger, and O. Kupferman. Alternating-time temporal logic. In Proceedings of the In- ternational Symposium Compositionality: The Significant Difference (COMPOS97), volume 1536 of Lecture Notes in Computer Science, pages 23-60. Springer, 1998.","title":"Alternating-time temporal logic","authors":"R Alur, T Henzinger, O Kupferman","year":"1998","venue":"Proceedings of the International Symposium Compositionality: The Significant Difference (COMPOS97)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Aminof et al., 2014] B. Aminof, T. Kotek, S. Rubin, F. Spegni, and H. Veith. Parameterized model checking of rendezvous systems. In Proceedings of the 25th Interna- tional Conference on Concurrency Theory (CONCUR14), pages 109-124. Springer, 2014.","title":"Parameterized model checking of rendezvous systems","authors":"B Aminof, T Kotek, S Rubin, F Spegni, H Veith","year":"2014","venue":"Proceedings of the 25th International Conference on Concurrency Theory (CONCUR14)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Apt and Kozen, 1986] K. Apt and D. C. Kozen. Limits for automatic verification of finite-state concurrent systems. Information Processing Letters, 22(6):307-309, 1986.","title":"Limits for automatic verification of finite-state concurrent systems","authors":"K Apt, D C Kozen","year":"1986","venue":"Information Processing Letters","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Bastani et al., 2016] O. Bastani, Y. Ioannou, L. Lampropou- los, D. Vytiniotis, A. V. Nori, and A. Criminisi. Measuring neural net robustness with constraints. In Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS16), pages 2613-2621, 2016.","title":"Measuring neural net robustness with constraints","authors":"O Bastani, Y Ioannou, L Lampropoulos, D Vytiniotis, A V Nori, A Criminisi","year":"2016","venue":"Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS16)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Batten et al., 2021] B. Batten, P. Kouvaros, A. Lomuscio, and Y. Zheng. Efficient neural network verification via layer-based semidefinite relaxations and linear cuts. In Proceedings of the 30th International Joint Conference on Artificial Intelligence (IJCAI21), pages 2184-2190. ij- cai.org, 2021.","title":"Efficient neural network verification via layer-based semidefinite relaxations and linear cuts","authors":"B Batten, P Kouvaros, A Lomuscio, Y Zheng","year":"2021","venue":"Proceedings of the 30th International Joint Conference on Artificial Intelligence (IJCAI21)","doi":null,"url":null,"internal_url":"/proceedings/2021/301"},{"position":9,"raw":"[Belardinelli et al., 2017] F. Belardinelli, P. Kouvaros, and A. Lomuscio. Parameterised verification of data-aware multi-agent systems. In Proceedings of the 24th Inter- national Joint Conference on Artificial Intelligence (IJ- CAI17), pages 98-104. AAAI Press, 2017.","title":"Parameterised verification of data-aware multi-agent systems","authors":"F Belardinelli, P Kouvaros, A Lomuscio","year":"2017","venue":"Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJ-CAI17)","doi":null,"url":null,"internal_url":"/proceedings/2017/15"},{"position":10,"raw":"[Botoeva et al., 2020] E. Botoeva, P. Kouvaros, J. Kronqvist, A. Lomuscio, and R. Misener. Efficient verification of neural networks via dependency analysis. In Proceed- ings of the 34th AAAI Conference on Artificial Intellifence (AAAI20), 2020.","title":"Efficient verification of neural networks via dependency analysis","authors":"E Botoeva, P Kouvaros, J Kronqvist, A Lomuscio, R Misener","year":"2020","venue":"Proceedings of the 34th AAAI Conference on Artificial Intellifence (AAAI20)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Boureanu et al., 2016] I. Boureanu, P. Kouvaros, and A. Lo- muscio. Verifying security properties in unbounded multi- agent systems. In Proceedings of the 15th International Conference on Autonomous Agents and Multi-Agent Sys- tems (AAMAS16), pages 1209-1218. IFAAMAS Press, 2016.","title":"Verifying security properties in unbounded multiagent systems","authors":"Boureanu","year":"2016","venue":"Proceedings of the 15th International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS16)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Brix et al., 2023] Christopher Brix, Mark Niklas Müller, Stanley Bak, Taylor T Johnson, and Changliu Liu. First three years of the international verification of neu- ral networks competition (vnn-comp). arXiv preprint arXiv:2301.05815, 2023.","title":"First three years of the international verification of neural networks competition","authors":"Christopher Brix, Mark Niklas Müller, Stanley Bak, Taylor T Johnson, Changliu Liu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Bunel et al., 2018] R. Bunel, I. Turkaslan, P. Torr, P. Kohli, and P. Mudigonda. A unified view of piecewise linear neu- ral network verification. In Proceedings of the 31st An- nual Conference on Neural Information Processing Sys- tems (NeurIPS18), pages 4790-4799, 2018.","title":"A unified view of piecewise linear neural network verification","authors":"R Bunel, I Turkaslan, P Torr, P Kohli, P Mudigonda","year":"2018","venue":"Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NeurIPS18)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Clarke et al., 1999] E.M. Clarke, O. Grumberg, and D.A. Peled. Model Checking. The MIT Press, 1999.","title":"Model Checking","authors":"E M Clarke, O Grumberg, D A Peled","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Clarke et al., 2001] E. Clarke, A. Biere, R. Raimi, and Y. Zhu. Bounded model checking using satisfiability solv- ing. Formal Methods in System Design, 19(1):7-34, 2001.","title":"Bounded model checking using satisfiability solving","authors":"E Clarke, A Biere, R Raimi, Y Zhu","year":"2001","venue":"Formal Methods in System Design","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Doshi-Velez and Kim, 2017] F. Doshi-Velez and B. Kim. Towards a rigorous science of interpretable machine learn- ing. arXiv preprint arXiv: 1702.08608, 2017.","title":"Towards a rigorous science of interpretable machine learning","authors":"Doshi-Velez, ; ] F Kim, B Doshi-Velez, Kim","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Emerson and Kahlon, 2000] E. Emerson and V. Kahlon. Reducing model checking of the many to the few. In Proceedings of the 17th International Conference on Au- tomated Deduction (CADE00), volume 1831 of Lecture Notes in Computer Science, pages 236-254. Springer, 2000.","title":"Reducing model checking of the many to the few","authors":"E Emerson, V Kahlon","year":"2000","venue":"Proceedings of the 17th International Conference on Automated Deduction (CADE00)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Emerson and Namjoshi, 1995] E. Emerson and K. Namjoshi. Reasoning about rings. In Proceed- ings of the 22nd Annual Sigact-Aigplan on Principles of Programming Languages (POPL95), pages 85-94.","title":"Reasoning about rings","authors":"E Emerson, K Namjoshi","year":null,"venue":"Proceedings of the 22nd Annual Sigact-Aigplan on Principles of Programming Languages (POPL95)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Fagin et al., 1995a] R. Fagin, J. Y. Halpern, Y. Moses, and M. Y. Vardi. Reasoning about Knowledge. MIT Press, Cambridge, 1995.","title":"Reasoning about Knowledge","authors":"Fagin","year":"1995","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Fagin et al., 1995b] R. Fagin, J. Y. Halpern, and M. Vardi. A nonstandard approach to the logical omniscience problem. Artificial Intelligence, 79, 1995.","title":"A nonstandard approach to the logical omniscience problem","authors":"Fagin","year":"1995","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Gammie and van der Meyden, 2004] P. Gammie and R. van der Meyden. MCK: Model checking the logic of knowledge. In Proceedings of 16th International Conference on Computer Aided Verification (CAV04), volume 3114 of Lecture Notes in Computer Science, pages 479-483. Springer, 2004.","title":"MCK: Model checking the logic of knowledge","authors":"P Gammie, R Van Der Meyden","year":"2004","venue":"Proceedings of 16th International Conference on Computer Aided Verification (CAV04)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Hashemi et al., 2021] V. Hashemi, P. Kouvaros, and A. Lo- muscio. Osip: Tightened bound propagation for the ver- ification of relu neural networks. In Proceedings of the 19th International Conference on Software Engineering and Formal Methods (SEFM21), pages 463-480. IEEE Computer Society, 2021.","title":"Osip: Tightened bound propagation for the verification of relu neural networks","authors":"V Hashemi, P Kouvaros, A Lomuscio","year":"2021","venue":"Proceedings of the 19th International Conference on Software Engineering and Formal Methods (SEFM21)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Katz et al., 2017] G. Katz, C. W. Barrett, D. L. Dill, K. Ju- lian, and M. J. Kochenderfer. Reluplex: An efficient SMT solver for verifying deep neural networks. In Proceedings of the 29th International Conference on Computer Aided Verification (CAV17), pages 97-117, 2017.","title":"Reluplex: An efficient SMT solver for verifying deep neural networks","authors":"G Katz, C W Barrett, D L Dill, K Julian, M J Kochenderfer","year":"2017","venue":"Proceedings of the 29th International Conference on Computer Aided Verification (CAV17)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Kouvaros and Lomuscio, 2013a] P. Kouvaros and A. Lo- muscio. Automatic verification of parametrised inter- leaved multi-agent systems. In Proceedings of the 12th In- ternational Conference on Autonomous Agents and Multi- Agent systems (AAMAS13), pages 861-868. IFAAMAS Press, 2013.","title":"Automatic verification of parametrised interleaved multi-agent systems","authors":"Lomuscio Kouvaros, P Kouvaros, A Lomuscio","year":"2013","venue":"Proceedings of the 12th International Conference on Autonomous Agents and Multi-Agent systems (AAMAS13)","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Kouvaros and Lomuscio, 2013b] P. Kouvaros and A. Lo- muscio. A cutoff technique for the verification of param- eterised interpreted systems with parameterised environ- ments. In Proceedings of the 23rd International Joint Con- ference on Artificial Intelligence (IJCAI13), pages 2013- 2019. AAAI Press, 2013.","title":"A cutoff technique for the verification of parameterised interpreted systems with parameterised environments","authors":"Lomuscio Kouvaros, P Kouvaros, A Lomuscio","year":"2013","venue":"Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI13)","doi":null,"url":null,"internal_url":"/proceedings/2013/297"},{"position":26,"raw":"[Kouvaros and Lomuscio, 2015a] P. Kouvaros and A. Lo- muscio. A counter abstraction technique for the verifica- tion of robot swarms. In Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI15), pages 2081-2088. AAAI Press, 2015.","title":"A counter abstraction technique for the verification of robot swarms","authors":"Lomuscio Kouvaros, P Kouvaros, A Lomuscio","year":"2015","venue":"Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI15)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Kouvaros and Lomuscio, 2015b] P. Kouvaros and A. Lo- muscio. Verifying emergent properties of swarms. In Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI15), pages 1083-1089. AAAI Press, 2015.","title":"Verifying emergent properties of swarms","authors":"Lomuscio Kouvaros, P Kouvaros, A Lomuscio","year":"2015","venue":"Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI15)","doi":null,"url":null,"internal_url":"/proceedings/2015/157"},{"position":28,"raw":"[Kouvaros and Lomuscio, 2016a] P. Kouvaros and A. Lo- muscio. Formal verification of opinion formation in swarms. In Proceedings of the 15th International Con- ference on Autonomous Agents and Multi-Agent systems (AAMAS16), pages 1200-1209. IFAAMAS Press, 2016.","title":"Formal verification of opinion formation in swarms","authors":"Lomuscio Kouvaros, P Kouvaros, A Lomuscio","year":"2016","venue":"Proceedings of the 15th International Conference on Autonomous Agents and Multi-Agent systems (AAMAS16)","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Kouvaros and Lomuscio, 2016b] P. Kouvaros and A. Lo- muscio. Parameterised model checking for alternating- time temporal logic. In Proceedings of the 22nd Eu- ropean Conference on Artificial Intelligence (ECAI16), pages 1230-1238. IOS Press, 2016.","title":"Parameterised model checking for alternatingtime temporal logic","authors":"Lomuscio Kouvaros, P Kouvaros, A Lomuscio","year":"2016","venue":"Proceedings of the 22nd European Conference on Artificial Intelligence (ECAI16)","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Kouvaros and Lomuscio, 2016c] P. Kouvaros and A. Lo- muscio. Parameterised verification for multi-agent sys- tems. Artificial Intelligence, 234:152-189, 2016.","title":"Parameterised verification for multi-agent systems","authors":"Lomuscio Kouvaros, P Kouvaros, A Lomuscio","year":"2016","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Kouvaros and Lomuscio, 2017a] P. Kouvaros and A. Lo- muscio. Parameterised verification of infinite state multi- agent systems via predicate abstraction. In Proceedings of the 24th International Joint Conference on Artificial Intel- ligence (IJCAI17), pages 3013-3020. AAAI Press, 2017.","title":"Parameterised verification of infinite state multiagent systems via predicate abstraction","authors":"Lomuscio Kouvaros, P Kouvaros, A Lomuscio","year":"2017","venue":"Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI17)","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Kouvaros and Lomuscio, 2017b] P. Kouvaros and A. Lo- muscio. Verifying fault-tolerance in parameterised multi- agent systems. In Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI17), pages 288-294. AAAI Press, 2017.","title":"Verifying fault-tolerance in parameterised multiagent systems","authors":"Lomuscio Kouvaros, P Kouvaros, A Lomuscio","year":"2017","venue":"Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI17)","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Kouvaros and Lomuscio, 2021] P. Kouvaros and A. Lomus- cio. Towards scalable complete verification of relu neural networks via dependency-based branching. In Proceed- ings of the 30th International Joint Conference on Arti- ficial Intelligence (IJCAI21), pages 2643-2650. ijcai.org, 2021.","title":"Towards scalable complete verification of relu neural networks via dependency-based branching","authors":"P Kouvaros, A Lomuscio","year":"2021","venue":"Proceedings of the 30th International Joint Conference on Artificial Intelligence (IJCAI21)","doi":null,"url":null,"internal_url":"/proceedings/2021/364"},{"position":34,"raw":"[Kouvaros et al., 2018] P. Kouvaros, A. Lomuscio, and E. Pirovano. Symbolic synthesis of fault-tolerance ratios in parameterised multi-agent systems. In Proceedings of the 27th International Joint Conference on Artificial In- telligence and 23rd European Conference on Artificial In- telligence (IJCAI-ECAI18), pages 324-330. AAAI Press, 2018.","title":"Symbolic synthesis of fault-tolerance ratios in parameterised multi-agent systems","authors":"P Kouvaros, A Lomuscio, E Pirovano","year":"2018","venue":"Proceedings of the 27th International Joint Conference on Artificial Intelligence and 23rd European Conference on Artificial Intelligence (IJCAI-ECAI18)","doi":null,"url":null,"internal_url":"/proceedings/2018/45"},{"position":35,"raw":"[Kouvaros et al., 2019] P. Kouvaros, A. Lomuscio, E. Pirovano, and H. Punchihewa. Formal verifica- tion of open multi-agent systems. In Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS19), pages 179-187.","title":"Formal verification of open multi-agent systems","authors":"P Kouvaros, A Lomuscio, E Pirovano, H Punchihewa","year":null,"venue":"Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS19)","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Kouvaros et al., 2021] P. Kouvaros, T. Kyono, F. Leo- fante, A. Lomuscio, D. Margineantu, D. Osipychev, and Y. Zheng. Formal analysis of neural network-based sys- tems in the aircraft domain. In Proceedings of the 24th In- ternational Symposium on Formal Methods (FM21), vol- ume 13047 of Lecture Notes in Computer Science, pages 730-740. Springer, 2021.","title":"Formal analysis of neural network-based systems in the aircraft domain","authors":"P Kouvaros, T Kyono, F Leofante, A Lomuscio, D Margineantu, D Osipychev, Y Zheng","year":"2021","venue":"Proceedings of the 24th International Symposium on Formal Methods (FM21)","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Kouvaros et al., 2023] P. Kouvaros, F. Leofante, C. Chung, B. Edwards, D. Margineantu, and A. Lomuscio. Verifica- tion of semantic key point detection for aircraft pose esti- mation. In Proceedings of the 20th International Confer- ence on Principles of Knowledge Representation and Rea- soning (KR23), 2023. To Appear. [Lomuscio and Michaliszyn, 2015]","title":"Verification of semantic key point detection for aircraft pose estimation","authors":"P Kouvaros, F Leofante, C Chung, B Edwards, D Margineantu, A Lomuscio","year":"2015","venue":"Proceedings of the 20th International Conference on Principles of Knowledge Representation and Reasoning (KR23)","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"A. Lomuscio and J. Michaliszyn. Verifying multi-agent systems by model checking three-valued abstractions. In Proceedings of the 14th International Conference on Autonomous Agents and Multiagent Systems (AAMAS15), pages 189-198, 2015.","title":"Verifying multi-agent systems by model checking three-valued abstractions","authors":"A Lomuscio, J Michaliszyn","year":"2015","venue":"Proceedings of the 14th International Conference on Autonomous Agents and Multiagent Systems (AAMAS15)","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Lomuscio and Raimondi, 2006] A. Lomuscio and F. Rai- mondi. Model checking knowledge, strategies, and games in multi-agent systems. In Proceedings of the 5th Interna- tional Joint Conference on Autonomous agents and Multi- Agent Systems (AAMAS06), pages 161-168. ACM Press, 2006.","title":"Model checking knowledge, strategies, and games in multi-agent systems","authors":"A Lomuscio, F","year":"2006","venue":"Proceedings of the 5th International Joint Conference on Autonomous agents and Multi-Agent Systems (AAMAS06)","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Penczek and Lomuscio, 2003] W. Penczek and A. Lomus- cio. Verifying epistemic properties of multi-agent systems via bounded model checking. Fundamenta Informaticae, 55(2):167-185, 2003.","title":"Verifying epistemic properties of multi-agent systems via bounded model checking","authors":"W Penczek, A Lomuscio","year":"2003","venue":"Fundamenta Informaticae","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Raghunathan et al., 2018] A. Raghunathan, J. Steinhardt, and P. S. Liang. Semidefinite relaxations for certifying robustness to adversarial examples. In Proceedings of the 32th International Conference on Neural Information Pro- cessing Systems (NIPS18), pages 10900-10910, 2018.","title":"Semidefinite relaxations for certifying robustness to adversarial examples","authors":"A Raghunathan, J Steinhardt, P S Liang","year":"2018","venue":"Proceedings of the 32th International Conference on Neural Information Processing Systems (NIPS18)","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Singh et al., 2019] G. Singh, T. Gehr, M. Püschel, and P. Vechev. An abstract domain for certifying neural networks. In Proceedings of the 46th ACM SIGPLAN Symposium on Principles of Programming Languages (POPL19), volume 3, pages 1-30. ACM, 2019.","title":"An abstract domain for certifying neural networks","authors":"G Singh, T Gehr, M Püschel, P Vechev","year":"2019","venue":"Proceedings of the 46th ACM SIGPLAN Symposium on Principles of Programming Languages (POPL19)","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Szegedy et al., 2014] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. In- triguing properties of neural networks. In Proceedings of the 2nd International Conference on Learning Represen- tations (ICLR14), 2014.","title":"Intriguing properties of neural networks","authors":"C Szegedy, W Zaremba, I Sutskever, J Bruna, D Erhan, I Goodfellow, R Fergus","year":"2014","venue":"Proceedings of the 2nd International Conference on Learning Representations (ICLR14)","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Wang et al., 2021] S. Wang, H. Zhang, K. Xu, X. Lin, S. Jana, C. Hsieh, and Z.Kolter. Beta-crown: Efficient bound propagation with per-neuron split constraints for neural network robustness verification. In Procceddings of the 34th International Conference on Neural Information Processing Systems (NIPS21), pages 29909-29921, 2021.","title":"Beta-crown: Efficient bound propagation with per-neuron split constraints for neural network robustness verification","authors":"S Wang, H Zhang, K Xu, X Lin, S Jana, C Hsieh, Z Kolter","year":"2021","venue":"Procceddings of the 34th International Conference on Neural Information Processing Systems (NIPS21)","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Wong and Kolter, 2018] E. Wong and J. Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In Proceedings of the 35th Interna- tional Conference on Machine Learning (ICML18), pages 5286-5295, 2018.","title":"Provable defenses against adversarial examples via the convex outer adversarial polytope","authors":"E Wong, J Kolter","year":"2018","venue":"Proceedings of the 35th International Conference on Machine Learning (ICML18)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}