{"type":"ScholarlyArticle","title":"Interpreting and Evaluating Neural Network Robustness","authors":[{"name":"Fuxun Yu","slug":"fuxun-yu","given":"Fuxun","family":"Yu","affiliation":""},{"name":"Zhuwei Qin","slug":"zhuwei-qin","given":"Zhuwei","family":"Qin","affiliation":""},{"name":"Chenchen Liu","slug":"chenchen-liu","given":"Chenchen","family":"Liu","affiliation":"University of Maryland , Baltimore County"},{"name":"Liang Zhao","slug":"liang-zhao","given":"Liang","family":"Zhao","affiliation":""},{"name":"Yanzhi Wang","slug":"yanzhi-wang","given":"Yanzhi","family":"Wang","affiliation":"Northeastern University"},{"name":"Xiang Chen","slug":"xiang-chen","given":"Xiang","family":"Chen","affiliation":""}],"abstract":"Recently, adversarial deception becomes one of the most considerable threats to deep neural networks. However, compared to extensive research in new designs of various adversarial attacks and defenses, the neural networks' intrinsic robustness property is still lack of thorough investigation. This work aims to qualitatively interpret the adversarial attack and defense mechanisms through loss visualization, and establish a quantitative metric to evaluate the model's intrinsic robustness. The proposed robustness metric identifies the upper bound of a model's prediction divergence in the given domain and thus indicates whether the model can maintain a stable prediction. With extensive experiments, our metric demonstrates several advantages over conventional testing accuracy based robustness estimation: (1) it provides a uniformed evaluation to models with different structures and parameter scales; (2) it over-performs conventional accuracy based robustness evaluation and provides a more reliable evaluation that is invariant to different test settings; (3) it can be fast generated without considerable testing cost.","doi":"10.24963/ijcai.2019/583","url":"https://ijcai.org/proceedings/2019/583","pdf":"https://ijcai.org/proceedings/2019/0583.pdf","markdown":"https://ijcai.org/proceedings/2019/583.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/583","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":583,"pages":{"first":4199,"last":4205},"keywords":["Machine Learning: Adversarial Machine Learning","Machine Learning: Interpretability"],"videos":[],"references":[{"position":1,"raw":"[Carlini and Wagner, 2017] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), pages 39-57. IEEE, 2017.","title":"Towards evaluating the robustness of neural networks","authors":"Wagner Carlini, Nicholas Carlini, David Wagner","year":"2017","venue":"IEEE Symposium on Security and Privacy (SP)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Dinh et al., 2017] Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. Sharp minima can general- ize for deep nets. In International Conference on Machine Learning, pages 1019-1028, 2017.","title":"Sharp minima can generalize for deep nets","authors":"Laurent Dinh, Razvan Pascanu, Samy Bengio, Yoshua Bengio","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Goodfellow et al., 2015] Ian J Goodfellow, Oriol Vinyals, and Andrew M Saxe. Qualitatively characterizing neural network optimization problems. In International Confer- ence on Learning Representations (ICLR), 2015.","title":"Qualitatively characterizing neural network optimization problems","authors":"Ian J Goodfellow, Oriol Vinyals, Andrew M Saxe","year":"2015","venue":"International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Hinton et al., 2012] Geoffrey Hinton, Li Deng, Dong Yu, George Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Brian Kingsbury, et al. Deep neural networks for acoustic modeling in speech recognition. IEEE Signal processing magazine, 29, 2012.","title":"Deep neural networks for acoustic modeling in speech recognition","authors":"Geoffrey Hinton, Li Deng, Dong Yu, George Dahl, Abdel-Rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Brian Kingsbury","year":"2012","venue":"IEEE Signal processing magazine","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Huval et al., 2015] Brody Huval, Tao Wang, Sameep Tan- don, Jeff Kiske, Will Song, Joel Pazhayampallil, Mykhaylo Andriluka, Pranav Rajpurkar, Toki Migimatsu, Royce Cheng-Yue, et al. An empirical evaluation of deep learning on highway driving. arXiv preprint arXiv:1504.01716, 2015.","title":"An empirical evaluation of deep learning on highway driving","authors":"Brody Huval, Tao Wang, Sameep Tandon, Jeff Kiske, Will Song, Joel Pazhayampallil, Mykhaylo Andriluka, Pranav Rajpurkar, Toki Migimatsu, Royce Cheng-Yue","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ian J. Goodfellow, 2014] Christian Szegedy Ian J. Goodfel- low, Jonathon Shlens. Explaining and harnessing adver- sarial examples. arXiv preprint arXiv:1412.6572, 2014.","title":"Explaining and harnessing adversarial examples","authors":"Ian J Goodfellow, Christian Szegedy, Ian J Goodfellow, Jonathon Shlens","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kannan et al., 2018] Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing. arXiv preprint arXiv:1803.06373, 2018.","title":"Adversarial logit pairing","authors":"Alexey Harini Kannan, Ian Kurakin, Goodfellow","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Keskar et al., 2017] Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large-batch training for deep learning: Generalization gap and sharp minima. In International Conference on Learning Representations (ICLR), 2017.","title":"On large-batch training for deep learning: Generalization gap and sharp minima","authors":"Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, Ping Tak, Peter Tang","year":"2017","venue":"International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kurakin et al., 2016] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXiv preprint arXiv:1607.02533, 2016.","title":"Adversarial examples in the physical world","authors":"Alexey Kurakin, Ian Goodfellow, Samy Bengio","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liu et al., 2018] Qi Liu, Tao Liu, Zihao Liu, Yanzhi Wang, Yier Jin, and Wujie Wen. Security analysis and enhance- ment of model compressed deep learning systems under adversarial attacks. In Proceedings of the 23rd Asia and South Pacific Design Automation Conference, pages 721- 726. IEEE Press, 2018.","title":"Security analysis and enhancement of model compressed deep learning systems under adversarial attacks","authors":"Qi Liu, Tao Liu, Zihao Liu, Yanzhi Wang, Yier Jin, Wujie Wen","year":"2018","venue":"Proceedings of the 23rd Asia and South Pacific Design Automation Conference","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Madry et al., 2018] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. 2018.","title":"Towards deep learning models resistant to adversarial attacks","authors":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Papernot et al., 2016] Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a defense to adversarial perturbations against deep neu- ral networks. In 2016 IEEE Symposium on Security and Privacy (SP), pages 582-597. IEEE, 2016.","title":"Distillation as a defense to adversarial perturbations against deep neural networks","authors":"Nicolas Papernot, Patrick Mcdaniel, Xi Wu, Somesh Jha, Ananthram Swami","year":"2016","venue":"IEEE Symposium on Security and Privacy (SP)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Papernot et al., 2017] Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Anan- thram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia Con- ference on Computer and Communications Security, pages 506-519. ACM, 2017.","title":"Practical black-box attacks against machine learning","authors":"Papernot","year":"2017","venue":"Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ross and Doshi-Velez, 2018] Andrew Slavin Ross and Fi- nale Doshi-Velez. Improving the adversarial robustness and interpretability of deep neural networks by regulariz- ing their input gradients. In AAAI, 2018.","title":"Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients","authors":"Ross, Doshi-Velez ;, Andrew Slavin, Ross, Finale Doshi-Velez","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Szegedy et al., 2013] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.","title":"Intriguing properties of neural networks","authors":"Szegedy","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Szegedy et al., 2016] Christian Szegedy, Vincent Van- houcke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818-2826, 2016.","title":"Rethinking the inception architecture for computer vision","authors":"Szegedy","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Xie et al., 2018] Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan Yuille, and Kaiming He. Feature denois- ing for improving adversarial robustness. arXiv preprint arXiv:1812.03411, 2018.","title":"Feature denoising for improving adversarial robustness","authors":"Xie","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}