{"type":"ScholarlyArticle","title":"Spear: Evaluate the Adversarial Robustness of Compressed Neural Models","authors":[{"name":"Chong Yu","slug":"chong-yu","given":"Chong","family":"Yu","affiliation":"Academy for Engineering and Technology , Fudan University"},{"name":"Tao Chen","slug":"tao-chen","given":"Tao","family":"Chen","affiliation":"School for Information Science and Technology , Fudan University"},{"name":"Zhongxue Gan","slug":"zhongxue-gan","given":"Zhongxue","family":"Gan","affiliation":"Academy for Engineering and Technology , Fudan University"},{"name":"Jiayuan Fan","slug":"jiayuan-fan","given":"Jiayuan","family":"Fan","affiliation":"Academy for Engineering and Technology , Fudan University"}],"abstract":"As Artificial Intelligence evolves, the neural models vulnerable to adversarial attacks may produce fatal results in critical applications. This paper mainly discusses the robustness of the compressed neural models facing adversarial attacks. A few studies discuss the interaction between model compression and adversarial attack. However, they focus on the robustness against the traditional attacks designed for the dense models, not the attacks intended explicitly for the compressed models, using sparsity and quantization techniques. Compressed models often have fewer parameters and smaller sizes that are more friendly to resource-limited devices than dense models, so they are widely deployed in various edge and mobile devices. However, introducing the sparsity and quantization into neural models further imposes higher attack risks. A specific adversarial attack method (Spear) is proposed to generate the particular adversarial attack samples for evaluating the robustness of the compressed models. The Spear attack finds minimal perturbations to create the attack samples to maximize the different behaviors between the compressed and dense reference models. We demonstrate the proposed Spear attack technique can generally be applied to various networks and tasks through quantitative and ablation experiments.","doi":"10.24963/ijcai.2024/177","url":"https://ijcai.org/proceedings/2024/177","pdf":"https://ijcai.org/proceedings/2024/0177.pdf","markdown":"https://ijcai.org/proceedings/2024/177.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/177","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Computer Vision","track":"Main Track","number":177,"pages":{"first":1598,"last":1606},"keywords":["Computer Vision: CV: Adversarial learning, adversarial attack and defense methods","Machine Learning: ML: Robustness","Machine Learning: ML: Adversarial machine learning"],"videos":[],"references":[{"position":1,"raw":"[Carlini and Wagner, 2017] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pages 39-57. Ieee, 2017.","title":"Towards evaluating the robustness of neural networks","authors":"Wagner Carlini, Nicholas Carlini, David Wagner","year":"2017","venue":"ieee symposium on security and privacy (sp)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cheng et al., 2021] Yupeng Cheng, Qing Guo, Felix Juefei- Xu, Wei Feng, Shang-wei Lin, Weisi Lin, and Yang Liu. Pasadena: Perceptually aware and stealthy adversarial de- noise attack. IEEE Transactions on Multimedia, 2021.","title":"Pasadena: Perceptually aware and stealthy adversarial denoise attack","authors":"Yupeng Cheng, Qing Guo, Felix Juefei-Xu, Wei Feng, Weisi Shang-Wei Lin, Yang Lin, Liu","year":"2021","venue":"IEEE Transactions on Multimedia","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 248-255. IEEE, 2009.","title":"Imagenet: A largescale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Goodfellow et al., 2014] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.","title":"Explaining and harnessing adversarial examples","authors":"Ian J Goodfellow, Jonathon Shlens, Christian Szegedy","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gui et al., 2019] Shupeng Gui, Haotao N Wang, Haichuan Yang, Chen Yu, Zhangyang Wang, and Ji Liu. Model com- pression with adversarial robustness: A unified optimiza- tion framework. Advances in Neural Information Process- ing Systems, 32:1285-1296, 2019.","title":"Model compression with adversarial robustness: A unified optimization framework","authors":"Gui","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2017] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE International Conference on Computer Vision, pages 2961-2969, 2017.","title":"Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn","authors":"He","year":"2017","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hinton et al., 2015] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.","title":"Distilling the knowledge in a neural network","authors":"Geoffrey Hinton, Oriol Vinyals, Jeff Dean","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kianpour and Wen, 2019] Mazaher Kianpour and Shao- Fang Wen. Timing attacks on machine learning: State of the art. In Proceedings of SAI Intelligent Systems Confer- ence, pages 111-125. Springer, 2019.","title":"Timing attacks on machine learning: State of the art","authors":"Wen ; Kianpour, Mazaher Kianpour, Shao-Fang Wen","year":"2019","venue":"Proceedings of SAI Intelligent Systems Conference","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Krishnamoorthi, 2018] Raghuraman Krishnamoorthi. Quantizing deep convolutional networks for efficient in- ference: A whitepaper. arXiv preprint arXiv:1806.08342, 2018.","title":"Quantizing deep convolutional networks for efficient inference: A whitepaper","authors":"Raghuraman Krishnamoorthi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Krizhevsky et al., 2009] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny im- ages. 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky, Geoffrey Hinton","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kurakin et al., 2018] Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In Artificial intelligence safety and security, pages 99-112.","title":"Adversarial examples in the physical world","authors":"Alexey Kurakin, Ian J Goodfellow, Samy Bengio","year":null,"venue":"Artificial intelligence safety and security","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"Chapman and Hall/CRC, 2018.","title":null,"authors":"Hall / Chapman, Crc","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[LeCun et al., 1998] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Com- mon objects in context. In European Conference on Com- puter Vision, pages 740-755. Springer, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence, Zitnick","year":"2014","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lin et al., 2017] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense ob- ject detection. In Proceedings of the IEEE International Conference on Computer Vision, pages 2980-2988, 2017.","title":"Focal loss for dense object detection","authors":"Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár","year":"2017","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2022] Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, et al. Swin transformer v2: Scaling up capacity and resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12009-12019, 2022.","title":"Swin transformer v2: Scaling up capacity and resolution","authors":"Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Madry et al., 2017] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017.","title":"Towards deep learning models resistant to adversarial attacks","authors":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Micikevicius et al., 2022] Paulius Micikevicius, Dusan Stosic, Neil Burgess, Marius Cornea, Pradeep Dubey, Richard Grisenthwaite, Sangwon Ha, Alexander Hei- necke, Patrick Judd, John Kamalu, et al. Fp8 formats for deep learning. arXiv preprint arXiv:2209.05433, 2022.","title":"Fp8 formats for deep learning","authors":"Paulius Micikevicius, Dusan Stosic, Neil Burgess, Marius Cornea, Pradeep Dubey, Richard Grisenthwaite, Sangwon Ha, Alexander Heinecke, Patrick Judd, John Kamalu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Moosavi-Dezfooli et al., 2016] Seyed-Mohsen Moosavi- Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deep- fool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2574-2582, 2016.","title":"Deepfool: a simple and accurate method to fool deep neural networks","authors":"Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Pascal Frossard","year":"2016","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[NVIDIA, 2017] NVIDIA. NVIDIA Tesla V100 GPU Architecture. https://images.nvidia.com/content/ volta-architecture/pdf/volta-architecture-whitepaper.pdf, 2017. Accessed: 2024-05-31.","title":"NVIDIA. NVIDIA Tesla V100 GPU Architecture","authors":null,"year":"2017","venue":null,"doi":null,"url":"https://images.nvidia.com/content/volta-architecture/pdf/volta-architecture-whitepaper.pdf","internal_url":null},{"position":22,"raw":"2020a] NVIDIA. NVIDIA A100 Ten- sor Core GPU Architecture. https://images. nvidia.com/aem-dam/en-zz/Solutions/data-center/ nvidia-ampere-architecture-whitepaper.pdf, 2020. Ac- cessed: 2024-05-31.","title":"NVIDIA. NVIDIA A100 Tensor Core GPU Architecture","authors":null,"year":"2020","venue":null,"doi":null,"url":"https://images.nvidia.com/aem-dam/en-zz/Solutions/data-center/nvidia-ampere-architecture-whitepaper.pdf","internal_url":null},{"position":23,"raw":"2020b] NVIDIA. NVIDIA Tensor Core. https: //www.nvidia.com/en-us/data-center/tensor-cores, 2020. Accessed: 2024-05-31.","title":"NVIDIA. NVIDIA Tensor Core","authors":null,"year":"2020","venue":null,"doi":null,"url":"https://www.nvidia.com/en-us/data-center/tensor-cores,2020","internal_url":null},{"position":24,"raw":"[Papernot et al., 2016] Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Anan- thram Swami. The limitations of deep learning in adversar- ial settings. In 2016 IEEE European Symposium on Secu- rity and Privacy (EuroS&P), pages 372-387. IEEE, 2016.","title":"The limitations of deep learning in adversarial settings","authors":"Nicolas Papernot, Patrick Mcdaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, Ananthram Swami","year":"2016","venue":"IEEE European Symposium on Security and Privacy (EuroS&P)","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Papernot et al., 2017] Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Anan- thram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia Con- ference on Computer and Communications Security, pages 506-519, 2017.","title":"Practical black-box attacks against machine learning","authors":"Nicolas Papernot, Patrick Mcdaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, Ananthram Swami","year":"2017","venue":"Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Paszke et al., 2017] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary De- Vito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In NIPS-W, 2017.","title":"Automatic differentiation in pytorch","authors":"Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary De-Vito, Zeming Lin, Alban Desmaison, Luca Antiga, Adam Lerer","year":"2017","venue":"NIPS-W","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Sap et al., 2020] Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A Smith, and Yejin Choi. Social bias frames: Reasoning about social and power implications of language. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 5477- 5490, 2020.","title":"Social bias frames: Reasoning about social and power implications of language","authors":"Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A Smith, Yejin Choi","year":"2020","venue":"Proceedings of the Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Schwinn et al., 2023] Leo Schwinn, René Raab, An Nguyen, Dario Zanca, and Bjoern Eskofier. Ex- ploring misclassifications of robust neural networks to enhance adversarial attacks. Applied Intelligence, pages 1-17, 2023.","title":"Exploring misclassifications of robust neural networks to enhance adversarial attacks","authors":"Leo Schwinn, René Raab, An Nguyen, Dario Zanca, Bjoern Eskofier","year":"2023","venue":"Applied Intelligence","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Shumailov et al., 2019] Ilia Shumailov, Yiren Zhao, Robert Mullins, and Ross Anderson. To compress or not to com- press: Understanding the interactions between adversarial attacks and neural network compression. Proceedings of Machine Learning and Systems, 1:230-240, 2019.","title":"To compress or not to compress: Understanding the interactions between adversarial attacks and neural network compression","authors":"Ilia Shumailov, Yiren Zhao, Robert Mullins, Ross Anderson","year":"2019","venue":"Proceedings of Machine Learning and Systems","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Song et al., 2020] Chang Song, Elias Fallon, and Hai Li. Improving adversarial robustness in weight-quantized neural networks. arXiv preprint arXiv:2012.14965, 2020.","title":"Improving adversarial robustness in weight-quantized neural networks","authors":"Chang Song, Elias Fallon, Hai Li","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Szegedy et al., 2013] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.","title":"Intriguing properties of neural networks","authors":"Szegedy","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Szegedy et al., 2014] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations, 2014.","title":"Intriguing properties of neural networks","authors":"Szegedy","year":"2014","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Ye et al., 2019] Shaokai Ye, Kaidi Xu, Sijia Liu, Hao Cheng, Jan-Henrik Lambrechts, Huan Zhang, Aojun Zhou, Kaisheng Ma, Yanzhi Wang, and Xue Lin. Ad- versarial robustness vs. model compression, or both? In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 111-120, 2019.","title":"Adversarial robustness vs. model compression, or both?","authors":"Ye","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yu, 2021] Chong Yu. Minimally invasive surgery for sparse neural networks in contrastive manner. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 3589-3598, 2021.","title":"Minimally invasive surgery for sparse neural networks in contrastive manner","authors":"Chong Yu","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhou et al., 2016] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2921-2929, 2016.","title":"Learning deep features for discriminative localization","authors":"Zhou","year":"2016","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhu and Gupta, 2018] Michael Zhu and Suyog Gupta. To prune, or not to prune: exploring the efficacy of pruning for model compression. In International Conference on Learning Representations, 2018.","title":"To prune, or not to prune: exploring the efficacy of pruning for model compression","authors":"Michael Zhu, Suyog Gupta","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}