{"type":"ScholarlyArticle","title":"Statistical Guarantees for the Robustness of Bayesian Neural Networks","authors":[{"name":"Luca Cardelli","slug":"luca-cardelli","given":"Luca","family":"Cardelli","affiliation":"University of Oxford"},{"name":"Marta Kwiatkowska","slug":"marta-kwiatkowska","given":"Marta","family":"Kwiatkowska","affiliation":"University of Oxford"},{"name":"Luca Laurenti","slug":"luca-laurenti","given":"Luca","family":"Laurenti","affiliation":"University of Oxford"},{"name":"Nicola Paoletti","slug":"nicola-paoletti","given":"Nicola","family":"Paoletti","affiliation":"Royal Holloway University of London"},{"name":"Andrea Patane","slug":"andrea-patane","given":"Andrea","family":"Patane","affiliation":"University of Oxford"},{"name":"Matthew Wicker","slug":"matthew-wicker","given":"Matthew","family":"Wicker","affiliation":"University of Oxford"}],"abstract":"We introduce a probabilistic robustness measure for Bayesian Neural Networks (BNNs), defined as the probability that, given a test point, there exists a point within a bounded set such that the BNN prediction differs between the two. Such a measure can be used, for instance, to quantify the probability of the existence of adversarial examples. Building on statistical verification techniques for probabilistic models, we develop a framework that allows us to estimate probabilistic robustness for a BNN with statistical guarantees, i.e., with a priori error and confidence bounds. We provide experimental comparison for several approximate BNN inference techniques on image classification tasks associated to MNIST and a two-class subset of the GTSRB dataset. Our results enable quantification of uncertainty of BNN predictions in adversarial settings.","doi":"10.24963/ijcai.2019/789","url":"https://ijcai.org/proceedings/2019/789","pdf":"https://ijcai.org/proceedings/2019/0789.pdf","markdown":"https://ijcai.org/proceedings/2019/789.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/789","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Uncertainty in AI","track":"Main track","number":789,"pages":{"first":5693,"last":5700},"keywords":["Uncertainty in AI: Uncertainty in AI","Uncertainty in AI: Bayesian Networks","Machine Learning: Adversarial Machine Learning","Computer Vision: Statistical Methods and Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Abate et al., 2008] Alessandro Abate, Maria Prandini, John Lygeros, and Shankar Sastry. Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems. Automatica, 44(11):2724-2734, 2008.","title":"Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems","authors":"Alessandro Abate, Maria Prandini, John Lygeros, Shankar Sastry","year":"2008","venue":"Automatica","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bartlett et al., 2017] Peter L Bartlett, Dylan J Foster, and Matus J Telgarsky. Spectrally-normalized margin bounds for neural networks. In NIPS, pages 6240-6249, 2017.","title":"Spectrally-normalized margin bounds for neural networks","authors":"Dylan J Peter L Bartlett, Matus J Foster, Telgarsky","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Biggio and Roli, 2018] Battista Biggio and Fabio Roli. Wild patterns: Ten years after the rise of adversarial machine learning. Pattern Recognition, 84:317-331, 2018.","title":"Wild patterns: Ten years after the rise of adversarial machine learning","authors":"Roli Biggio, Battista Biggio, Fabio Roli","year":"2018","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Blundell et al., 2015] Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks. arXiv preprint arXiv:1505.05424, 2015.","title":"Weight uncertainty in neural networks","authors":"Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, Daan Wierstra","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bortolussi et al., 2016] Luca Bortolussi, Luca Cardelli, Marta Kwiatkowska, and Luca Laurenti. Approximation of probabilistic reachability for chemical reaction networks using the linear noise approximation. In QEST, pages 72-88, 2016.","title":"Approximation of probabilistic reachability for chemical reaction networks using the linear noise approximation","authors":"Luca Bortolussi, Luca Cardelli, Marta Kwiatkowska, Luca Laurenti","year":"2016","venue":"QEST","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cardelli et al., 2018] Luca Cardelli, Marta Kwiatkowska, Luca Laurenti, and Andrea Patane. Robustness guarantees for Bayesian inference with Gaussian processes. arXiv preprint arXiv:1809.06452, 2018.","title":"Robustness guarantees for Bayesian inference with Gaussian processes","authors":"Luca Cardelli, Marta Kwiatkowska, Luca Laurenti, Andrea Patane","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Carlini and Wagner, 2016] Nicholas Carlini and David Wagner. Towards Evaluating the Robustness of Neural Networks. arXiv e-prints, page arXiv:1608.04644, Aug 2016.","title":"Towards Evaluating the Robustness of Neural Networks","authors":"Wagner Carlini, Nicholas Carlini, David Wagner","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chernoff, 1952] Herman Chernoff. A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations. The Annals of Mathematical Statistics, 23(4):493-507, 1952.","title":"A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations","authors":"Herman Chernoff","year":"1952","venue":"The Annals of Mathematical Statistics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cohen et al., 2019] Jeremy M Cohen, Elan Rosenfeld, and J Zico Kolter. Certified adversarial robustness via randomized smoothing. arXiv preprint arXiv:1902.02918, 2019.","title":"Certified adversarial robustness via randomized smoothing","authors":"Elan Jeremy M Cohen, J Zico Rosenfeld, Kolter","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dvijotham et al., 2018] Krishnamurthy Dvijotham, Marta Garnelo, Alhussein Fawzi, and Pushmeet Kohli. Verification of deep probabilistic models. arXiv:1812.02795, 2018.","title":"Verification of deep probabilistic models","authors":"Krishnamurthy Dvijotham, Marta Garnelo, Alhussein Fawzi, Pushmeet Kohli","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Feinman et al., 2017] Reuben Feinman, Ryan R Curtin, Saurabh Shintre, and Andrew B Gardner. Detecting adversarial samples from artifacts. arXiv:1703.00410, 2017.","title":"Detecting adversarial samples from artifacts","authors":"Reuben Feinman, Saurabh Ryan R Curtin, Andrew B Shintre, Gardner","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gal and Ghahramani, 2016] Yarin Gal and Zoubin Ghahramani. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In ICML, pages 1050-1059, 2016.","title":"Dropout as a Bayesian approximation: Representing model uncertainty in deep learning","authors":"Ghahramani Gal, Yarin Gal, Zoubin Ghahramani","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gal, 2016] Yarin Gal. Uncertainty in deep learning. PhD thesis, University of Cambridge, 2016.","title":"Uncertainty in deep learning","authors":"Yarin Gal","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Goodfellow et al., 2014] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.","title":"Explaining and harnessing adversarial examples","authors":"Ian J Goodfellow, Jonathon Shlens, Christian Szegedy","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Grosse et al., 2018] Kathrin Grosse, David Pfaff, Michael T Smith, and Michael Backes. The limitations of model uncertainty in adversarial settings. arXiv:1812.02606, 2018.","title":"The limitations of model uncertainty in adversarial settings","authors":"Kathrin Grosse, David Pfaff, Michael T Smith, Michael Backes","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Huang et al., 2017] Xiaowei Huang, Marta Kwiatkowska, Sen Wang, and Min Wu. Safety verification of deep neural networks. In CAV, pages 3-29. Springer, 2017.","title":"Safety verification of deep neural networks","authors":"Xiaowei Huang, Marta Kwiatkowska, Sen Wang, Min Wu","year":"2017","venue":"CAV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Jegourel et al., 2018] Cyrille Jegourel, Jun Sun, and Jin Song Dong. On the sequential Massart algorithm for statistical model checking. In ISoLA, pages 287-304. Springer, 2018.","title":"On the sequential Massart algorithm for statistical model checking","authors":"Cyrille Jegourel, Jun Sun, Jin Song, Dong","year":"2018","venue":"ISoLA","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Katz et al., 2017] Guy Katz, Clark Barrett, David L Dill, Kyle Julian, and Mykel J Kochenderfer. Reluplex: An efficient SMT solver for verifying deep neural networks. In CAV, pages 97-117. Springer, 2017.","title":"Reluplex: An efficient SMT solver for verifying deep neural networks","authors":"Guy Katz, Clark Barrett, David L Dill, Kyle Julian, Mykel, Kochenderfer","year":"2017","venue":"CAV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kendall et al., 2015] Alex Kendall, Vijay Badrinarayanan, and Roberto Cipolla. Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. arXiv preprint arXiv:1511.02680, 2015.","title":"Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding","authors":"Alex Kendall, Vijay Badrinarayanan, Roberto Cipolla","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[LeCun and Cortes, 2010] Yann LeCun and Corinna Cortes. MNIST handwritten digit database. 2010.","title":"MNIST handwritten digit database","authors":"Lecun, Cortes, Yann Lecun, Corinna Cortes","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[MacKay, 1992] David JC MacKay. A practical Bayesian frame- work for backpropagation networks. Neural computation, 4(3):448-472, 1992.","title":"A practical Bayesian framework for backpropagation networks","authors":"J C David, Mackay","year":"1992","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Madry et al., 2017] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards Deep Learning Models Resistant to Adversarial Attacks. arXiv e-prints, June 2017.","title":"Towards Deep Learning Models Resistant to Adversarial Attacks","authors":"A Madry, A Makelov, L Schmidt, D Tsipras, A Vladu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Massart, 1990] Pascal Massart. The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality. The annals of Probability, pages 1269-1283, 1990.","title":"The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality","authors":"Pascal Massart","year":"1990","venue":"The annals of Probability","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[McAllister et al., 2017] R. McAllister, Y. Gal, A. Kendall, M. Van Der Wilk, A. Shah, Ro. Cipolla, and A. Weller. Concrete problems for autonomous vehicle safety: Advantages of Bayesian deep learning. IJCAI, 2017.","title":"Concrete problems for autonomous vehicle safety: Advantages of Bayesian deep learning","authors":"R Mcallister, Y Gal, A Kendall, M Van Der Wilk, A Shah, Ro Cipolla, A Weller","year":"2017","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2017/661"},{"position":25,"raw":"[Myshkov and Julier, 2016] Pavel Myshkov and Simon Julier. Posterior distribution analysis for Bayesian inference in neural networks. In Workshop on Bayesian Deep Learning, 2016.","title":"Posterior distribution analysis for Bayesian inference in neural networks","authors":"Julier Myshkov, Pavel Myshkov, Simon Julier","year":"2016","venue":"Workshop on Bayesian Deep Learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Neal, 2012] Radford M. Neal. MCMC using Hamiltonian dynamics. arXiv e-prints, page arXiv:1206.1901, June 2012.","title":"MCMC using Hamiltonian dynamics","authors":"M Radford, Neal","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Neyshabur et al., 2017] Behnam Neyshabur, Srinadh Bhojana- palli, David McAllester, and Nati Srebro. Exploring gener- alization in deep learning. In NIPS, pages 5947-5956, 2017.","title":"Exploring generalization in deep learning","authors":"Srinadh Behnam Neyshabur, David Bhojanapalli, Nati Mcallester, Srebro","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Rawat et al., 2017] Ambrish Rawat, Martin Wistuba, and Maria- Irina Nicolae. Adversarial phenomenon in the eyes of Bayesian deep learning. arXiv preprint arXiv:1711.08244, 2017.","title":"Adversarial phenomenon in the eyes of Bayesian deep learning","authors":"Ambrish Rawat, Martin Wistuba, Maria-Irina Nicolae","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Ruan et al., 2018] Wenjie Ruan, Xiaowei Huang, and Marta Kwiatkowska. Reachability analysis of deep neural networks with provable guarantees. In IJCAI, pages 2651-2659. AAAI Press, 2018.","title":"Reachability analysis of deep neural networks with provable guarantees","authors":"Wenjie Ruan, Xiaowei Huang, Marta Kwiatkowska","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/368"},{"position":30,"raw":"[Stallkamp et al., 2012] J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel. Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural Networks, (0):-, 2012.","title":"Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition","authors":"J Stallkamp, M Schlipsing, J Salmen, C Igel","year":"2012","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Tran et al., 2016] Dustin Tran, Alp Kucukelbir, Adji B. Dieng, Maja Rudolph, Dawen Liang, and David M. Blei. Edward: A library for probabilistic modeling, inference, and criticism. arXiv preprint arXiv:1610.09787, 2016.","title":"Edward: A library for probabilistic modeling, inference, and criticism","authors":"Dustin Tran, Alp Kucukelbir, B Adji, Maja Dieng, Dawen Rudolph, David M Liang, Blei","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Webb et al., 2018] Stefan Webb, Tom Rainforth, Yee Whye Teh, and M Pawan Kumar. Statistical verification of neural networks. arXiv preprint arXiv:1811.07209, 2018.","title":"Statistical verification of neural networks","authors":"Stefan Webb, Tom Rainforth, Yee Whye Teh, M Pawan Kumar","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wicker et al., 2018] Matthew Wicker, Xiaowei Huang, and Marta Kwiatkowska. Feature-guided black-box safety testing of deep neural networks. In TACAS, pages 408-426. Springer, 2018.","title":"Feature-guided black-box safety testing of deep neural networks","authors":"Matthew Wicker, Xiaowei Huang, Marta Kwiatkowska","year":"2018","venue":"TACAS","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wu et al., 2018] Min Wu, Matthew Wicker, Wenjie Ruan, Xiaowei Huang, and Marta Kwiatkowska. A game-based approximate verification of deep neural networks with provable guarantees. To appear in Theoretical Computer Science. CoRR, abs/1807.03571, 2018.","title":"A game-based approximate verification of deep neural networks with provable guarantees","authors":"Wu","year":"2018","venue":"Theoretical Computer Science","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}