{"type":"ScholarlyArticle","title":"Imperceptible Backdoor Attack: From Input Space to Feature Representation","authors":[{"name":"Nan Zhong","slug":"nan-zhong","given":"Nan","family":"Zhong","affiliation":"School of Computer Science , Fudan University"},{"name":"Zhenxing Qian","slug":"zhenxing-qian","given":"Zhenxing","family":"Qian","affiliation":"School of Computer Science , Fudan University"},{"name":"Xinpeng Zhang","slug":"xinpeng-zhang","given":"Xinpeng","family":"Zhang","affiliation":"School of Computer Science , Fudan University"}],"abstract":"Backdoor attacks are rapidly emerging threats to deep neural networks (DNNs). In the backdoor attack scenario, attackers usually implant the backdoor into the target model by manipulating the training dataset or training process. Then, the compromised model behaves normally for benign input yet makes mistakes when the pre-defined trigger appears. In this paper, we analyze the drawbacks of existing attack approaches and propose a novel imperceptible backdoor attack. We treat the trigger pattern as a special kind of noise following a multinomial distribution. A U-net-based network is employed to generate concrete parameters of multinomial distribution for each benign input. This elaborated trigger ensures that our approach is invisible to both humans and statistical detection. Besides the design of the trigger, we also consider the robustness of our approach against model diagnose-based defences. We force the feature representation of malicious input stamped with the trigger to be entangled with the benign one. We demonstrate the effectiveness and robustness against multiple state-of-the-art defences through extensive datasets and networks. Our trigger only modifies less than 1\\% pixels of a benign image while the modification magnitude is 1. Our source code is available at https://github.com/Ekko-zn/IJCAI2022-Backdoor.","doi":"10.24963/ijcai.2022/242","url":"https://ijcai.org/proceedings/2022/242","pdf":"https://ijcai.org/proceedings/2022/0242.pdf","markdown":"https://ijcai.org/proceedings/2022/242.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/242","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":242,"pages":{"first":1736,"last":1742},"keywords":["Computer Vision: Adversarial learning, adversarial attack and defense methods","AI Ethics, Trust, Fairness: Trustworthy AI"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986644","length_sec":87,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984787","length_sec":345,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Carlini and Wagner, 2017] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pages 39-57. IEEE, 2017.","title":"Towards evaluating the robustness of neural networks","authors":"Wagner Carlini, Nicholas Carlini, David Wagner","year":"2017","venue":"ieee symposium on security and privacy (sp)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2019] Huili Chen, Cheng Fu, Jishen Zhao, and Farinaz Koushanfar. Deepinspect: A black-box trojan detection and mitigation framework for deep neural net- works. In IJCAI, pages 4658-4664, 2019.","title":"Deepinspect: A black-box trojan detection and mitigation framework for deep neural networks","authors":"Huili Chen, Cheng Fu, Jishen Zhao, Farinaz Koushanfar","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/647"},{"position":3,"raw":"[Gu et al., 2017] Tianyu Gu, Brendan Dolan-Gavitt, and Sid- dharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733, 2017.","title":"Badnets: Identifying vulnerabilities in the machine learning model supply chain","authors":"Tianyu Gu, Brendan Dolan-Gavitt, Siddharth Garg","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[He et al., 2017] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross B. Girshick. Mask R-CNN. In IEEE In- ternational Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017, pages 2980-2988, 2017.","title":"Mask R-CNN","authors":"He","year":"2017","venue":"IEEE International Conference on Computer Vision, ICCV 2017","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Houben et al., 2013] Sebastian Houben, Johannes Stal- lkamp, Jan Salmen, Marc Schlipsing, and Christian Igel. Detection of traffic signs in real-world images: The ger- man traffic sign detection benchmark. In The 2013 in- ternational joint conference on neural networks (IJCNN), pages 1-8. Ieee, 2013.","title":"Detection of traffic signs in real-world images: The german traffic sign detection benchmark","authors":"Sebastian Houben, Johannes Stallkamp, Jan Salmen, Marc Schlipsing, Christian Igel","year":"2013","venue":"The 2013 international joint conference on neural networks (IJCNN)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Li et al., 2020a] Shaofeng Li, Minhui Xue, Benjamin Zhao, Haojin Zhu, and Xinpeng Zhang. Invisible backdoor at- tacks on deep neural networks via steganography and reg- ularization. IEEE Transactions on Dependable and Secure Computing, 2020.","title":"Invisible backdoor attacks on deep neural networks via steganography and regularization","authors":"Li","year":"2020","venue":"IEEE Transactions on Dependable and Secure Computing","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Li et al., 2020b] Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. Neural attention distillation: Erasing backdoor triggers from deep neural networks. In International Conference on Learning Rep- resentations, 2020.","title":"Neural attention distillation: Erasing backdoor triggers from deep neural networks","authors":"Li","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li et al., 2021] Yuezun Li, Yiming Li, Baoyuan Wu, Longkang Li, Ran He, and Siwei Lyu. Invisible backdoor attack with sample-specific triggers. 2021.","title":"Invisible backdoor attack with sample-specific triggers","authors":"Yuezun Li, Yiming Li, Baoyuan Wu, Longkang Li, Ran He, Siwei Lyu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liu et al., 2015] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.","title":"Deep learning face attributes in the wild","authors":"Ziwei Liu, Ping Luo, Xiaogang Wang, Xiaoou Tang","year":"2015","venue":"Proceedings of International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2018] Kang Liu, Brendan Dolan-Gavitt, and Sid- dharth Garg. Fine-pruning: Defending against backdoor- ing attacks on deep neural networks. In International Sym- posium on Research in Attacks, Intrusions, and Defenses, pages 273-294. Springer, 2018.","title":"Fine-pruning: Defending against backdooring attacks on deep neural networks","authors":"Kang Liu, Brendan Dolan-Gavitt, Siddharth Garg","year":"2018","venue":"International Symposium on Research in Attacks, Intrusions, and Defenses","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Long et al., 2015] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for seman- tic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3431- 3440, 2015.","title":"Fully convolutional networks for semantic segmentation","authors":"Jonathan Long, Evan Shelhamer, Trevor Darrell","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Nguyen and Tran, 2021] Tuan Anh Nguyen and Anh Tuan Tran. Wanet -imperceptible warping-based backdoor at- tack. In International Conference on Learning Represen- tations. OpenReview.net, 2021.","title":"Wanet -imperceptible warping-based backdoor attack","authors":"Anh Tuan, Anh Nguyen, Tran Tuan","year":"2021","venue":"International Conference on Learning Representations. OpenReview.net","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pages 234-241. Springer, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"International Conference on Medical image computing and computerassisted intervention","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Tancik et al., 2020] Matthew Tancik, Ben Mildenhall, and Ren Ng. Stegastamp: Invisible hyperlinks in physical pho- tographs. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.","title":"Stegastamp: Invisible hyperlinks in physical photographs","authors":"Matthew Tancik, Ben Mildenhall, Ren Ng","year":"2020","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wang et al., 2019] Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. Neural cleanse: Identifying and mitigat- ing backdoor attacks in neural networks. In 2019 IEEE Symposium on Security and Privacy (SP), pages 707-723. IEEE, 2019.","title":"Neural cleanse: Identifying and mitigating backdoor attacks in neural networks","authors":"Wang","year":"2019","venue":"IEEE Symposium on Security and Privacy (SP)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2021] Lun Wang, Zaynah Javed, Xian Wu, Wenbo Guo, Xinyu Xing, and Dawn Song. Backdoorl: Backdoor attack against competitive reinforcement learn- ing. IJCAI, 2021.","title":"Backdoorl: Backdoor attack against competitive reinforcement learning","authors":"Lun Wang, Zaynah Javed, Xian Wu, Wenbo Guo, Xinyu Xing, Dawn Song","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/509"},{"position":18,"raw":"[Wu and Wang, 2021] Dongxian Wu and Yisen Wang. Ad- versarial neuron pruning purifies backdoored deep mod- els. Advances in Neural Information Processing Systems, 34, 2021.","title":"Adversarial neuron pruning purifies backdoored deep models","authors":"Wang ; Wu, Dongxian Wu, Yisen Wang","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zeng et al., 2021] Yi Zeng, Won Park, Z Morley Mao, and Ruoxi Jia. Rethinking the backdoor attacks' triggers: A frequency perspective. IEEE International Conference on Computer Vision (ICCV), 2021.","title":"Rethinking the backdoor attacks' triggers: A frequency perspective","authors":"Yi Zeng, Won Park, Morley Mao, Ruoxi Jia","year":"2021","venue":"IEEE International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhong et al., 2021] Nan Zhong, Zhenxing Qian, and Xin- peng Zhang. Undetectable adversarial examples based on microscopical regularization. In 2021 IEEE International Conference on Multimedia and Expo (ICME), pages 1-6. IEEE, 2021.","title":"Undetectable adversarial examples based on microscopical regularization","authors":"Nan Zhong, Zhenxing Qian, Xinpeng Zhang","year":"2021","venue":"2021 IEEE International Conference on Multimedia and Expo (ICME)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhou et al., 2021] Xinzhe Zhou, Wenhao Jiang, Sheng Qi, and Yadong Mu. Multi-target invisibly trojaned networks for visual recognition and detection. IJCAI, 2021.","title":"Multi-target invisibly trojaned networks for visual recognition and detection","authors":"Zhou","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/477"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}