{"type":"ScholarlyArticle","title":"Model-Agnostic Adversarial Detection by Random Perturbations","authors":[{"name":"Bo Huang","slug":"bo-huang","given":"Bo","family":"Huang","affiliation":"Dongguan University of Technology , Dongguan , China; Shenzhen University , Shenzhen , China"},{"name":"Yi Wang","slug":"yi-wang","given":"Yi","family":"Wang","affiliation":"Dongguan University of Technology , Dongguan , China"},{"name":"Wei Wang","slug":"wei-wang","given":"Wei","family":"Wang","affiliation":"The University of New South Wales , Sydney , Australia"}],"abstract":"Adversarial examples induce model classification errors on purpose, which has raised concerns on the security aspect of machine learning techniques. Many existing countermeasures are compromised by adaptive adversaries and transferred examples. We propose a model-agnostic approach to resolve the problem by analysing the model responses to an input under random perturbations, and study the robustness of detecting norm-bounded adversarial distortions in a theoretical framework. Extensive evaluations are performed on the MNIST, CIFAR-10 and ImageNet datasets. The results demonstrate that our detection method is effective and resilient against various attacks including black-box attacks and the powerful CW attack with four adversarial adaptations.","doi":"10.24963/ijcai.2019/651","url":"https://ijcai.org/proceedings/2019/651","pdf":"https://ijcai.org/proceedings/2019/0651.pdf","markdown":"https://ijcai.org/proceedings/2019/651.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/651","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":651,"pages":{"first":4689,"last":4696},"keywords":["Machine Learning: Adversarial Machine Learning","Multidisciplinary Topics and Applications: Security and Privacy"],"videos":[],"references":[{"position":1,"raw":"[Akhtar et al., 2018] Naveed Akhtar, Jian Liu, and Ajmal Mian. Defense Against Universal Adversarial Perturba- tions. In IEEE Conf. Comput. Vis. Pattern Recognit., pages 3389-3398, June 2018.","title":"Defense Against Universal Adversarial Perturbations","authors":"Naveed Akhtar, Jian Liu, Ajmal Mian","year":"2018","venue":"IEEE Conf. Comput. Vis. Pattern Recognit","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Athalye et al., 2018] Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Ex- amples. In Int. Conf. Mach. Learn., pages 274-283, Stock- holm, Sweden, July 2018.","title":"Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples","authors":"Anish Athalye, Nicholas Carlini, David Wagner","year":"2018","venue":"Int. Conf. Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Biggio et al., 2013] Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion Attacks against Machine Learning at Test Time. In Hendrik Blockeel, Kristian Kersting, Siegfried Nijssen, and Filip Železný, editors, Mach. Learn. Knowl. Discov. Databases, pages 387-402, Berlin, Heidelberg, 2013.","title":"Evasion Attacks against Machine Learning at Test Time","authors":"Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndić, Pavel Laskov, Giorgio Giacinto, Fabio Roli","year":"2013","venue":"Mach. Learn. Knowl. Discov. Databases","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Carlini and Wagner, 2017a] Nicholas Carlini and David Wagner. Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods. In ACM Work. Artif. Intell. Secur., pages 3-14, Dallas, Texas, USA, October 2017.","title":"Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods","authors":"Wagner Carlini, Nicholas Carlini, David Wagner","year":"2017","venue":"ACM Work. Artif. Intell. Secur","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Carlini and Wagner, 2017b] Nicholas Carlini and David Wagner. Towards Evaluating the Robustness of Neural Networks. In IEEE Symp. Secur. Priv., pages 39-57, San Jose, CA, USA, May 2017.","title":"Towards Evaluating the Robustness of Neural Networks","authors":"Wagner Carlini, Nicholas Carlini, David Wagner","year":"2017","venue":"IEEE Symp. Secur. Priv","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cisse et al., 2017] Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. Par- seval Networks: Improving Robustness to Adversarial Ex- amples. In Int. Conf. Mach. Learn., pages 854-863, Syd- ney, Australia, August 2017.","title":"Parseval Networks: Improving Robustness to Adversarial Examples","authors":"Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, Nicolas Usunier","year":"2017","venue":"Int. Conf. Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fawzi et al., 2016] Alhussein Fawzi, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. Robustness of classifiers: from adversarial to random noise. In Neural Inf. Process. Syst., pages 1632-1640, Barcelona, Spain, December 2016.","title":"Robustness of classifiers: from adversarial to random noise","authors":"Fawzi","year":"2016","venue":"Neural Inf. Process. Syst","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goodfellow et al., 2015] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and Harness- ing Adversarial Examples. In Int. Conf. Learn. Represent., pages 1-11, San Diego, CA, USA, May 2015.","title":"Explaining and Harnessing Adversarial Examples","authors":"Ian J Goodfellow, Jonathon Shlens, Christian Szegedy","year":"2015","venue":"Int. Conf. Learn. Represent","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Grosse et al., 2017] Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, and Patrick McDaniel. On the (Statistical) Detection of Adversarial Examples. In IEEE Int. Conf. Comput. Vis., pages 446-454, Venice, Italy, October 2017.","title":"On the (Statistical) Detection of Adversarial Examples","authors":"Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, Patrick Mcdaniel","year":"2017","venue":"IEEE Int. Conf. Comput. Vis","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Guo et al., 2017] Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens van der Maaten. Countering Adver- sarial Images using Input Transformations. In Int. Conf. Learn. Represent., pages 1-12, Vancouver, Canada, Octo- ber 2017.","title":"Countering Adversarial Images using Input Transformations","authors":"Chuan Guo, Mayank Rana, Moustapha Cisse, Laurens Van Der Maaten","year":"2017","venue":"Int. Conf. Learn. Represent","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kurakin et al., 2017] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial Machine Learning at Scale. In Int. Conf. Learn. Represent., pages 1-17, Toulon, France, October 2017. Lecuyer et al., 2018] Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certi- fied Robustness to Adversarial Examples with Differential Privacy. In Int. Conf. Learn. Represent., arXiv preprint, pages 1-18, Vancouver, BC, Canada, February 2018.","title":"Adversarial Machine Learning at Scale","authors":"Alexey Kurakin, Ian Goodfellow, Samy Bengio, ; Lecuyer","year":"2017","venue":"Certified Robustness to Adversarial Examples with Differential Privacy","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lu et al., 2017] Jiajun Lu, Theerasit Issaranon, and David Forsyth. SafetyNet: Detecting and Rejecting Adversar- ial Examples Robustly. In IEEE Int. Conf. Comput. Vis., pages 446-454, October 2017.","title":"SafetyNet: Detecting and Rejecting Adversarial Examples Robustly","authors":"Jiajun Lu, Theerasit Issaranon, David Forsyth","year":"2017","venue":"IEEE Int. Conf. Comput. Vis","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ma et al., 2018] Xingjun Ma, Bo Li, Yisen Wang, Sarah M. Erfani, Sudanthi Wijewickrema, Grant Schoenebeck, Dawn Song, Michael E. Houle, and James Bailey. Charac- terizing Adversarial Subspaces Using Local Intrinsic Di- mensionality. In Int. Conf. Learn. Represent., pages 1-15, Vancouver, Canada, February 2018.","title":"Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality","authors":"Xingjun Ma, Bo Li, Yisen Wang, Sarah M Erfani, Sudanthi Wijewickrema, Grant Schoenebeck, Dawn Song, Michael E Houle, James Bailey","year":"2018","venue":"Int. Conf. Learn. Represent","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Madry et al., 2018] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards Deep Learning Models Resistant to Adversarial Attacks. In Int. Conf. Learn. Represent., arXiv preprint, pages 1-27, Vancouver, Canada, February 2018.","title":"Towards Deep Learning Models Resistant to Adversarial Attacks","authors":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu","year":"2018","venue":"Int. Conf. Learn. Represent","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Moosavi-Dezfooli et al., 2016] Seyed-Mohsen Moosavi- Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deep- Fool: a simple and accurate method to fool deep neural networks. In IEEE Conf. Comput. Vis. Pattern Recognit., pages 2574-2582, June 2016.","title":"Deep-Fool: a simple and accurate method to fool deep neural networks","authors":"Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Pascal Frossard","year":"2016","venue":"IEEE Conf. Comput. Vis. Pattern Recognit","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Papernot et al., 2016] Nicolas Papernot, Patrick Mcdaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Anan- thram Swami. The limitations of deep learning in adver- sarial settings. In IEEE Eur. Symp. Secur. Priv., pages 372- 387, April 2016.","title":"The limitations of deep learning in adversarial settings","authors":"Nicolas Papernot, Patrick Mcdaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, Ananthram Swami","year":"2016","venue":"IEEE Eur. Symp. Secur. Priv","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Papernot et al., 2018] Nicolas Papernot, Patrick McDaniel, Arunesh Sinha, and Michael Wellman. Towards the Sci- ence of Security and Privacy in Machine Learning. In IEEE Eur. Symp. Secur. Priv., pages 1-19, London, UK, September 2018.","title":"Towards the Science of Security and Privacy in Machine Learning","authors":"Papernot","year":"2018","venue":"IEEE Eur. Symp. Secur. Priv","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Su et al., 2018] Dong Su, Huan Zhang, Hongge Chen, Jin- feng Yi, Pin-Yu Chen, and Yupeng Gao. Is Robustness the Cost of Accuracy? A Comprehensive Study on the Ro- bustness of 18 Deep Image Classification Models. In Eur. Conf. Comput. Vis., pages 644-661, Munich, Germany, September 2018. Springer, Cham. [Szegedy et al., 2014] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In Int. Conf. Learn. Represent., pages 1-10, Banff, Canada, April 2014.","title":"Is Robustness the Cost of Accuracy? A Comprehensive Study on the Robustness of 18 Deep Image Classification Models","authors":"Dong Su, Huan Zhang, Hongge Chen, Jinfeng Yi, Pin-Yu Chen, Yupeng Gao","year":"2014","venue":"Int. Conf. Learn. Represent","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Tramèr et al., 2018] Florian Tramèr, Alexey Kurakin, Nico- las Papernot, Dan Boneh, and Patrick McDaniel. Ensem- ble Adversarial Training: Attacks and Defenses. In Int. Conf. Learn. Represent., pages 1-20, Vancouver, Canada, May 2018.","title":"Ensemble Adversarial Training: Attacks and Defenses","authors":"Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Dan Boneh, Patrick Mcdaniel","year":"2018","venue":"Int. Conf. Learn. Represent","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xu et al., 2017] Weilin Xu, David Evans, and Yanjun Qi. Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks. In Netw. Distrib. Syst. Secur. Symp., pages 1-15, San Diego, CA, USA, February 2017.","title":"Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks","authors":"Weilin Xu, David Evans, Yanjun Qi","year":"2017","venue":"Netw. Distrib. Syst. Secur. Symp","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}