{"type":"ScholarlyArticle","title":"Detecting Adversarial Attacks via Subset Scanning of Autoencoder Activations and Reconstruction Error","authors":[{"name":"Celia Cintas","slug":"celia-cintas","given":"Celia","family":"Cintas","affiliation":"IBM Research Africa , Nairobi , Kenya."},{"name":"Skyler Speakman","slug":"skyler-speakman","given":"Skyler","family":"Speakman","affiliation":"IBM Research Africa , Nairobi , Kenya."},{"name":"Victor Akinwande","slug":"victor-akinwande","given":"Victor","family":"Akinwande","affiliation":"IBM Research Africa , Nairobi , Kenya."},{"name":"William Ogallo","slug":"william-ogallo","given":"William","family":"Ogallo","affiliation":"IBM Research Africa , Nairobi , Kenya."},{"name":"Komminist Weldemariam","slug":"komminist-weldemariam","given":"Komminist","family":"Weldemariam","affiliation":"IBM Research Africa , Nairobi , Kenya."},{"name":"Srihari Sridharan","slug":"srihari-sridharan","given":"Srihari","family":"Sridharan","affiliation":"IBM Research Africa , Nairobi , Kenya."},{"name":"Edward McFowland","slug":"edward-mcfowland","given":"Edward","family":"McFowland","affiliation":"Carlson School of Management , University of Minnesota , USA."}],"abstract":"Reliably detecting attacks in a given set of inputs is of high practical relevance because of the vulnerability of neural networks to adversarial examples.\r\nThese altered inputs create a security risk in applications with real-world consequences, such as self-driving cars, robotics and financial services.\r\nWe propose an unsupervised method for detecting adversarial attacks in inner layers of autoencoder (AE) networks by maximizing a non-parametric measure of anomalous node activations. \r\nPrevious work in this space has shown AE networks can detect anomalous images by thresholding the reconstruction error produced by the final layer. Furthermore, other detection methods rely on data augmentation or specialized training techniques which must be asserted before training time. In contrast, we use subset scanning methods from the anomalous pattern detection domain to enhance detection power without labeled examples of the noise, retraining or data augmentation methods. In addition to an anomalous “score” our proposed method also returns the subset of nodes within the AE network that contributed to that score. This will allow future work to pivot from detection to visualisation and explainability. Our scanning approach shows consistently higher detection power than existing detection methods across several adversarial noise models and a wide range of perturbation strengths.","doi":"10.24963/ijcai.2020/122","url":"https://ijcai.org/proceedings/2020/122","pdf":"https://ijcai.org/proceedings/2020/0122.pdf","markdown":"https://ijcai.org/proceedings/2020/122.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/122","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Computer Vision","track":"Main track","number":122,"pages":{"first":876,"last":882},"keywords":["Data Mining: Big Data, Large-Scale Systems","Computer Vision: Statistical Methods and Machine Learning","Machine Learning: Deep Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24342/file_24342.m3u8","length_sec":287,"provider":"hls"}],"references":[{"position":1,"raw":"[Beggel et al., 2019] Laura Beggel, Michael Pfeiffer, and Bernd Bischl. Robust anomaly detection in im- ages using adversarial autoencoders. arXiv preprint arXiv:1901.06355, 2019.","title":"Robust anomaly detection in images using adversarial autoencoders","authors":"Laura Beggel, Michael Pfeiffer, Bernd Bischl","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Berk and Jones, 1979] Robert H. Berk and Douglas. H. Jones. Goodness-of-fit test statistics that dominate the Kol- mogorov statistics. Zeitschrift fär Wahrscheinlichkeitsthe- orie und Verwandte Gebiete, 47:47-59, 1979.","title":"Goodness-of-fit test statistics that dominate the Kolmogorov statistics","authors":"Robert H Berk, Douglas H Jones","year":"1979","venue":"Zeitschrift fär Wahrscheinlichkeitstheorie und Verwandte Gebiete","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen and Neill, 2014] Feng Chen and Daniel B. Neill. Non- parametric scan statistics for event detection and forecast- ing in heterogeneous social media graphs. In KDD '14, pages 1166-1175, 2014.","title":"Nonparametric scan statistics for event detection and forecasting in heterogeneous social media graphs","authors":"Chen, Neill ; ] Feng Chen, Daniel B Neill","year":"2014","venue":"KDD '14","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2019] Jianbo Chen, Michael I Jordan, and Mar- tin J Wainwright. Hopskipjumpattack: A query-efficient decision-based attack. arXiv preprint arXiv:1904.02144, 3, 2019.","title":"Hopskipjumpattack: A query-efficient decision-based attack","authors":"Jianbo Chen, Michael I Jordan, Martin J Wainwright","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Davis and Goadrich, 2006] Jesse Davis and Mark Goadrich. The relationship between precision-recall and roc curves. In Proceedings of the 23rd international conference on Machine learning, pages 233-240. ACM, 2006.","title":"The relationship between precision-recall and roc curves","authors":"Goadrich Davis, Jesse Davis, Mark Goadrich","year":"2006","venue":"Proceedings of the 23rd international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Frosst et al., 2018] Nicholas Frosst, Sara Sabour, and Ge- offrey Hinton. Darccc: Detecting adversaries by recon- struction from class conditional capsules. arXiv preprint arXiv:1811.06969, 2018.","title":"Darccc: Detecting adversaries by reconstruction from class conditional capsules","authors":"Nicholas Frosst, Sara Sabour, Geoffrey Hinton","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goodfellow et al., 2015] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. CoRR, abs/1412.6572, 2015.","title":"Explaining and harnessing adversarial examples","authors":"Ian J Goodfellow, Jonathon Shlens, Christian Szegedy","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Krizhevsky et al., 2009] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny im- ages. Technical report, Citeseer, 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky, Geoffrey Hinton","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kurakin et al., 2016] Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial examples in the physical world. CoRR, abs/1607.02533, 2016.","title":"Adversarial examples in the physical world","authors":"Alexey Kurakin, Ian J Goodfellow, Samy Bengio","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[LeCun et al., 1998] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Madry et al., 2017] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017.","title":"Towards deep learning models resistant to adversarial attacks","authors":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[McFowland et al., 2018] Edward McFowland, III, Sriram Somanchi, and Daniel B. Neill. Efficient Discovery of Heterogeneous Treatment Effects in Randomized Exper- iments via Anomalous Pattern Detection. ArXiv e-prints, March 2018.","title":"Efficient Discovery of Heterogeneous Treatment Effects in Randomized Experiments via Anomalous Pattern Detection","authors":"Edward Mcfowland, Sriram Iii, Daniel B Somanchi, Neill","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[McFowland III et al., 2013] Edward McFowland III, Skyler D. Speakman, and Daniel B. Neill. Fast gener- alized subset scan for anomalous pattern detection. The Journal of Machine Learning Research, 14(1):1533-1561, Jun 2013.","title":"Fast generalized subset scan for anomalous pattern detection","authors":"Edward Mcfowland, Iii, Skyler D Speakman, Daniel B Neill","year":"2013","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Meng and Chen, 2017] Dongyu Meng and Hao Chen. Mag- net: a two-pronged defense against adversarial examples. In ACM CCS'17, 2017.","title":"Magnet: a two-pronged defense against adversarial examples","authors":"Chen ; ] Dongyu Meng, Hao Meng, Chen","year":"2017","venue":"ACM CCS'17","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Moosavi-Dezfooli et al., 2016] Seyed-Mohsen Moosavi- Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deep- fool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE CVPR'16, pages 2574-2582, 2016.","title":"Deepfool: a simple and accurate method to fool deep neural networks","authors":"Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Pascal Frossard","year":"2016","venue":"Proceedings of the IEEE CVPR'16","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Neill, 2012] Daniel B. Neill. Fast subset scan for spatial pattern detection. Journal of the Royal Statistical Society (Series B: Statistical Methodology), 74(2):337-360, 2012.","title":"Fast subset scan for spatial pattern detection","authors":"B Daniel, Neill","year":"2012","venue":"Journal of the Royal Statistical Society (Series B: Statistical Methodology)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Nicolae et al., 2018] Maria-Irina Nicolae, Mathieu Sinn, Minh Ngoc Tran, Ambrish Rawat, Martin Wistuba, Valentina Zantedeschi, Nathalie Baracaldo, Bryant Chen, Heiko Ludwig, Ian Molloy, and Ben Edwards. Adversarial Robustness Toolbox v0.7.0. CoRR, 1807.01069, 2018.","title":"Adversarial Robustness Toolbox v0.7.0","authors":"Maria-Irina Nicolae, Mathieu Sinn, Minh Ngoc Tran, Ambrish Rawat, Martin Wistuba, Valentina Zantedeschi, Nathalie Baracaldo, Bryant Chen, Heiko Ludwig, Ian Molloy, Ben Edwards","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Papernot and McDaniel, 2016] Nicolas Papernot and Patrick D. McDaniel. On the effectiveness of defensive distillation. CoRR, abs/1607.05113, 2016.","title":"On the effectiveness of defensive distillation","authors":"Nicolas Papernot, Patrick D Mcdaniel","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sakurada and Yairi, 2014] Mayu Sakurada and Takehisa Yairi. Anomaly detection using autoencoders with non- linear dimensionality reduction. In Proceedings of the MLSDA'14, page 4. ACM, 2014.","title":"Anomaly detection using autoencoders with nonlinear dimensionality reduction","authors":"Yairi Sakurada, Mayu Sakurada, Takehisa Yairi","year":"2014","venue":"Proceedings of the MLSDA'14","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Samangouei et al., 2018] Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Defense-gan: Protecting classifiers against adversarial attacks using generative models. arXiv preprint arXiv:1805.06605, 2018.","title":"Defense-gan: Protecting classifiers against adversarial attacks using generative models","authors":"Pouya Samangouei, Maya Kabkab, Rama Chellappa","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Schölkopf et al., 2001] Bernhard Schölkopf, John C Platt, John Shawe-Taylor, Alex J Smola, and Robert C Williamson. Estimating the support of a high-dimensional distribution. Neural computation, 13(7):1443-1471, 2001.","title":"Estimating the support of a high-dimensional distribution","authors":"Bernhard Schölkopf, John C Platt, John Shawe-Taylor, Alex J Smola, Robert C Williamson","year":"2001","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Szegedy et al., 2013] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Er- han, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. CoRR, abs/1312.6199, 2013.","title":"Intriguing properties of neural networks","authors":"Szegedy","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xiao et al., 2017] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for bench- marking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.","title":"Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms","authors":"Han Xiao, Kashif Rasul, Roland Vollgraf","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhai et al., 2016] Shuangfei Zhai, Yu Cheng, Weining Lu, and Zhongfei Zhang. Deep structured energy based models for anomaly detection. arXiv preprint arXiv:1605.07717, 2016.","title":"Deep structured energy based models for anomaly detection","authors":"Shuangfei Zhai, Yu Cheng, Weining Lu, Zhongfei Zhang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhou and Paffenroth, 2017] Chong Zhou and Randy C Paf- fenroth. Anomaly detection with robust deep autoen- coders. In ACM SIGKDD'17, pages 665-674. ACM, 2017.","title":"Anomaly detection with robust deep autoencoders","authors":"Chong Zhou, Randy C Paffenroth","year":"2017","venue":"ACM SIGKDD'17","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}