{"type":"ScholarlyArticle","title":"A Few Seconds Can Change Everything: Fast Decision-based Attacks against DNNs","authors":[{"name":"Ningping Mou","slug":"ningping-mou","given":"Ningping","family":"Mou","affiliation":"‡ The Key Laboratory of Aerospace Information Security and Trusted Computing , Ministry of Education , School of Cyber Sci- ence and Engineering , Wuhan University , Wuhan 430072 , Hubei , China."},{"name":"Baolin Zheng","slug":"baolin-zheng","given":"Baolin","family":"Zheng","affiliation":"§ School of Computer Science , Wuhan University , Wuhan 430072 , Hubei , China."},{"name":"Qian Wang","slug":"qian-wang","given":"Qian","family":"Wang","affiliation":"‡ The Key Laboratory of Aerospace Information Security and Trusted Computing , Ministry of Education , School of Cyber Sci- ence and Engineering , Wuhan University , Wuhan 430072 , Hubei , China.; Wuhan University , Wuhan , China"},{"name":"Yunjie Ge","slug":"yunjie-ge","given":"Yunjie","family":"Ge","affiliation":"‡ The Key Laboratory of Aerospace Information Security and Trusted Computing , Ministry of Education , School of Cyber Sci- ence and Engineering , Wuhan University , Wuhan 430072 , Hubei , China."},{"name":"Binqing Guo","slug":"binqing-guo","given":"Binqing","family":"Guo","affiliation":"‡ The Key Laboratory of Aerospace Information Security and Trusted Computing , Ministry of Education , School of Cyber Sci- ence and Engineering , Wuhan University , Wuhan 430072 , Hubei , China."}],"abstract":"Previous researches have demonstrated deep learning models' vulnerabilities to decision-based adversarial attacks, which craft adversarial examples based solely on information from output decisions (top-1 labels). However, existing decision-based attacks have two major limitations, i.e., expensive query cost and being easy to detect. To bridge the gap and enlarge real threats to commercial applications, we propose a novel and efficient decision-based attack against black-box models, dubbed FastDrop, which only requires a few queries and work well under strong defenses. The crux of the innovation is that, unlike existing adversarial attacks that rely on gradient estimation and additive noise, FastDrop generates adversarial examples by dropping information in the frequency domain. Extensive experiments on three datasets demonstrate that FastDrop can escape the detection of the state-of-the-art (SOTA) black-box defenses and reduce the number of queries by 13~133× under the same level of perturbations compared with the SOTA attacks. FastDrop only needs 10~20 queries to conduct an attack against various black-box models within 1s. Besides, on commercial vision APIs provided by Baidu and Tencent, FastDrop achieves an attack success rate (ASR) of 100% with 10 queries on average, which poses a real and severe threat to real-world applications.","doi":"10.24963/ijcai.2022/464","url":"https://ijcai.org/proceedings/2022/464","pdf":"https://ijcai.org/proceedings/2022/0464.pdf","markdown":"https://ijcai.org/proceedings/2022/464.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/464","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":464,"pages":{"first":3342,"last":3350},"keywords":["Computer Vision: Adversarial learning, adversarial attack and defense methods","Machine Learning: Adversarial Machine Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986674","length_sec":79,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984817","length_sec":296,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bhagoji et al., 2018] Arjun Nitin Bhagoji, Warren He, Bo Li, and Dawn Song. Practical black-box attacks on deep neural networks using efficient query mechanisms. In Proc. of ECCV, pages 158-174, 2018.","title":"Practical black-box attacks on deep neural networks using efficient query mechanisms","authors":"Nitin Arjun, Warren Bhagoji, Bo He, Dawn Li, Song","year":"2018","venue":"Proc. of ECCV","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Brendel et al., 2018] Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Re- liable attacks against black-box machine learning models. In Proc. of ICLR, 2018.","title":"Decision-based adversarial attacks: Reliable attacks against black-box machine learning models","authors":"Wieland Brendel, Jonas Rauber, Matthias Bethge","year":"2018","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Carlini and Wagner, 2017] Nicholas Carlini and David A. Wagner. Adversarial examples are not easily detected: By- passing ten detection methods. In Proc. of ACM Workshop on Artificial Intelligence and Security, pages 3-14, 2017.","title":"Adversarial examples are not easily detected: Bypassing ten detection methods","authors":"Nicholas Carlini, David A Wagner","year":"2017","venue":"Proc. of ACM Workshop on Artificial Intelligence and Security","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2017] Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. ZOO: Zeroth order opti- mization based black-box attacks to deep neural networks without training substitute models. In Proc. of ACM Work- shop on Artificial Intelligence and Security, pages 15-26, 2017.","title":"ZOO: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models","authors":"Chen","year":"2017","venue":"Proc. of ACM Workshop on Artificial Intelligence and Security","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2020] Jianbo Chen, Michael I. Jordan, and Mar- tin J. Wainwright. Hopskipjumpattack: A query-efficient decision-based attack. In Proc. of IEEE S&P, pages 1277- 1294, 2020.","title":"Hopskipjumpattack: A query-efficient decision-based attack","authors":"Jianbo Chen, Michael I Jordan, Martin J Wainwright","year":"2020","venue":"Proc. of IEEE S&P","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2021] Guangyao Chen, Peixi Peng, Li Ma, Jia Li, Lin Du, and Yonghong Tian. Amplitude-phase recom- bination: Rethinking robustness of convolutional neural networks in frequency domain. In Proc. of IEEE ICCV, pages 458-467, 2021.","title":"Amplitude-phase recombination: Rethinking robustness of convolutional neural networks in frequency domain","authors":"Guangyao Chen, Peixi Peng, Li Ma, Jia Li, Lin Du, Yonghong Tian","year":"2021","venue":"Proc. of IEEE ICCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cheng et al., 2019] Minhao Cheng, Thong Le, Pin-Yu Chen, Huan Zhang, Jinfeng Yi, and Cho-Jui Hsieh. Query- efficient hard-label black-box attack: An optimization- based approach. In Proc. of ICLR, 2019.","title":"Queryefficient hard-label black-box attack: An optimizationbased approach","authors":"Cheng","year":"2019","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Coates et al., 2011] Adam Coates, Andrew Y. Ng, and Honglak Lee. An analysis of single-layer networks in un- supervised feature learning. In Proc. of AISTATS, pages 215-223, 2011.","title":"An analysis of single-layer networks in unsupervised feature learning","authors":"Adam Coates, Andrew Y Ng, Honglak Lee","year":"2011","venue":"Proc. of AISTATS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In Proc. of IEEE CVPR, pages 248-255, 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"Proc. of IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Duan et al., 2021] Ranjie Duan, Yuefeng Chen, Dantong Niu, Yun Yang, A. Kai Qin, and Yuan He. Advdrop: Ad- versarial attack to dnns by dropping information. In Proc. of IEEE ICCV, pages 7486-7495, 2021.","title":"Advdrop: Adversarial attack to dnns by dropping information","authors":"Duan","year":"2021","venue":"Proc. of IEEE ICCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Goodfellow et al., 2015] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Proc. of ICLR, 2015.","title":"Explaining and harnessing adversarial examples","authors":"Ian J Goodfellow, Jonathon Shlens, Christian Szegedy","year":"2015","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Guo et al., 2019] Chuan Guo, Jacob R. Gardner, Yurong You, Andrew Gordon Wilson, and Kilian Q. Weinberger. Simple black-box adversarial attacks. In Proc. of ACM ICML, pages 2484-2493, 2019.","title":"Simple black-box adversarial attacks","authors":"Chuan Guo, Jacob R Gardner, Yurong You, Andrew Gordon Wilson, Kilian Q Weinberger","year":"2019","venue":"Proc. of ACM ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proc. of IEEE CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proc. of IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Howard et al., 2019] Andrew Howard, Ruoming Pang, Hartwig Adam, Quoc V. Le, Mark Sandler, Bo Chen, Weijun Wang, Liang-Chieh Chen, Mingxing Tan, Grace Chu, Vijay Vasudevan, and Yukun Zhu. Searching for mobilenetv3. In Proc. of IEEE ICCV, pages 1314-1324, 2019.","title":"Searching for mobilenetv3","authors":"Andrew Howard, Ruoming Pang, Hartwig Adam, Quoc V Le, Mark Sandler, Bo Chen, Weijun Wang, Liang-Chieh Chen, Mingxing Tan, Grace Chu, Vijay Vasudevan, Yukun Zhu","year":"2019","venue":"Proc. of IEEE ICCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hu et al., 2019] Shengyuan Hu, Tao Yu, Chuan Guo, Wei- Lun Chao, and Kilian Q. Weinberger. A new defense against adversarial images: Turning a weakness into a strength. In Proc. of NeurIPS, pages 1633-1644, 2019.","title":"A new defense against adversarial images: Turning a weakness into a strength","authors":"Shengyuan Hu, Tao Yu, Chuan Guo, Wei-Lun Chao, Kilian Q Weinberger","year":"2019","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2020] Huiying Li, Shawn Shan, Emily Wenger, Ji- ayun Zhang, Haitao Zheng, and Ben Y. Zhao. Blacklight: Defending black-box adversarial attacks on deep neural networks. arXiv preprint arXiv:2006.14042, 2020.","title":"Blacklight: Defending black-box adversarial attacks on deep neural networks","authors":"Huiying Li, Shawn Shan, Emily Wenger, Jiayun Zhang, Haitao Zheng, Ben Y Zhao","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Madry et al., 2018] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In Proc. of ICLR, 2018.","title":"Towards deep learning models resistant to adversarial attacks","authors":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu","year":"2018","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nilsback and Zisserman, 2008] Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In Proc. of IEEE ICVGIP, pages 722-729, 2008.","title":"Automated flower classification over a large number of classes","authors":"Maria-Elena Nilsback, Andrew Zisserman","year":"2008","venue":"Proc. of IEEE ICVGIP","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Prakash et al., 2018] Aaditya Prakash, Nick Moran, Solomon Garber, Antonella DiLillo, and James A. Storer. Deflecting adversarial attacks with pixel deflection. In Proc. of IEEE CVPR, pages 8571-8580, 2018.","title":"Deflecting adversarial attacks with pixel deflection","authors":"Aaditya Prakash, Nick Moran, Solomon Garber, Antonella Dilillo, James A Storer","year":"2018","venue":"Proc. of IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rahmati et al., 2020] Ali Rahmati, Seyed-Mohsen Moosavi-Dezfooli, Pascal Frossard, and Huaiyu Dai. Geoda: A geometric framework for black-box adversarial attacks. In Proc. of IEEE CVPR, pages 8443-8452, 2020.","title":"Geoda: A geometric framework for black-box adversarial attacks","authors":"Ali Rahmati, Seyed-Mohsen Moosavi-Dezfooli, Pascal Frossard, Huaiyu Dai","year":"2020","venue":"Proc. of IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shin and Song, 2017] Richard Shin and Dawn Song. Jpeg- resistant adversarial images. In Proc. of NIPS Workshop on Machine Learning and Computer Security, 2017.","title":"Jpegresistant adversarial images","authors":"Richard Shin, Dawn Song","year":"2017","venue":"Proc. of NIPS Workshop on Machine Learning and Computer Security","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shukla et al., 2021] Satya Narayan Shukla, Anit Kumar Sahu, Devin Willmott, and J. Zico Kolter. Simple and effi- cient hard label black-box adversarial attacks in low query budget regimes. In Proc. of ACM SIGKDD, pages 1461- 1469, 2021.","title":"Simple and efficient hard label black-box adversarial attacks in low query budget regimes","authors":"Anit Satya Narayan Shukla, Devin Kumar Sahu, J Zico Willmott, Kolter","year":"2021","venue":"Proc. of ACM SIGKDD","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Simonyan and Zisserman, 2015] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. In Proc. of ICLR, 2015.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2015","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2020a] Haohan Wang, Xindi Wu, Zeyi Huang, and Eric P. Xing. High-frequency component helps explain the generalization of convolutional neural networks. In Proc. of IEEE CVPR, pages 8681-8691, 2020.","title":"High-frequency component helps explain the generalization of convolutional neural networks","authors":"Wang","year":"2020","venue":"Proc. of IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2020b] Zifan Wang, Yilin Yang, Ankit Shrivas- tava, Varun Rawal, and Zihao Ding. Towards frequency- based explanation for robust CNN. arXiv preprint arXiv:2005.03141, 2020.","title":"Towards frequencybased explanation for robust CNN","authors":"Wang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xu et al., 2018] Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. In Proc. of NDSS, 2018.","title":"Feature squeezing: Detecting adversarial examples in deep neural networks","authors":"Weilin Xu, David Evans, Yanjun Qi","year":"2018","venue":"Proc. of NDSS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}