{"type":"ScholarlyArticle","title":"Eliminating Backdoor Triggers for Deep Neural Networks Using Attention Relation Graph Distillation","authors":[{"name":"Jun Xia","slug":"jun-xia","given":"Jun","family":"Xia","affiliation":"MoE Eng. Research Center of SW/HW Co-Design Tech. and App. , East China Normal University"},{"name":"Ting Wang","slug":"ting-wang","given":"Ting","family":"Wang","affiliation":"MoE Eng. Research Center of SW/HW Co-Design Tech. and App. , East China Normal University"},{"name":"Jiepin Ding","slug":"jiepin-ding","given":"Jiepin","family":"Ding","affiliation":"MoE Eng. Research Center of SW/HW Co-Design Tech. and App. , East China Normal University"},{"name":"Xian Wei","slug":"xian-wei","given":"Xian","family":"Wei","affiliation":"MoE Eng. Research Center of SW/HW Co-Design Tech. and App. , East China Normal University"},{"name":"Mingsong Chen","slug":"mingsong-chen","given":"Mingsong","family":"Chen","affiliation":"MoE Eng. Research Center of SW/HW Co-Design Tech. and App. , East China Normal University; Shanghai Institute of Intelligent Science and Technology , Tongji University"}],"abstract":"Due to the prosperity of Artificial Intelligence (AI) techniques, more and more backdoors are designed by adversaries to attack Deep Neural Networks (DNNs). Although the state-of-the-art method Neural Attention Distillation (NAD) can effectively erase backdoor triggers from DNNs, it still suffers from non-negligible Attack Success Rate (ASR) together with lowered classification ACCuracy (ACC), since NAD focuses on backdoor defense using attention features (i.e., attention maps) of the same order. In this paper, we introduce a novel backdoor defense framework named Attention Relation Graph Distillation (ARGD), which fully explores the correlation among attention features with different orders using our proposed Attention Relation Graphs (ARGs). Based on the alignment of ARGs between teacher and student models during knowledge distillation, ARGD can more effectively eradicate backdoors than NAD. Comprehensive experimental results show that, against six latest backdoor attacks, ARGD outperforms NAD by up to 94.85% reduction in ASR, while ACC can be improved by up to 3.23%.","doi":"10.24963/ijcai.2022/206","url":"https://ijcai.org/proceedings/2022/206","pdf":"https://ijcai.org/proceedings/2022/0206.pdf","markdown":"https://ijcai.org/proceedings/2022/206.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/206","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":206,"pages":{"first":1481,"last":1487},"keywords":["Machine Learning: Adversarial Machine Learning","Multidisciplinary Topics and Applications: Security and Privacy","Computer Vision: Adversarial learning, adversarial attack and defense methods"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986666","length_sec":91,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984809","length_sec":332,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Adi et al., 2018] Yossi Adi, Carsten Baum, Moustapha Cissé, Benny Pinkas, and Joseph Keshet. Turning your weakness into a strength: Watermarking deep neural net- works by backdooring. In USENIX Security Symposium, 2018.","title":"Turning your weakness into a strength: Watermarking deep neural networks by backdooring","authors":"Yossi Adi, Carsten Baum, Moustapha Cissé, Benny Pinkas, Joseph Keshet","year":"2018","venue":"USENIX Security Symposium","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bryant et al., 2019] Chen Bryant, Carvalho Wilka, Bara- caldo Nathalie, Ludwig Heiko, Edwards Benjamin, Lee Taesung, Molloy Ian, and Srivastava Biplav. Detecting backdoor attacks on deep neural networks by activation clustering. In AAAI, pages 39-57, 2019.","title":"Detecting backdoor attacks on deep neural networks by activation clustering","authors":"Chen Bryant, Carvalho Wilka, Baracaldo Nathalie, Ludwig Heiko, Edwards Benjamin, Lee Taesung, Ian Molloy, Srivastava Biplav","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Carlini and Wagner, 2017] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In IEEE S&P, pages 39-57, 2017.","title":"Towards evaluating the robustness of neural networks","authors":"Wagner Carlini, Nicholas Carlini, David Wagner","year":"2017","venue":"IEEE S&P","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2017] Xinyun Chen, Chang Liu, Bo Li, Kim- berly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526, 2017.","title":"Targeted backdoor attacks on deep learning systems using data poisoning","authors":"Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, Dawn Song","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chou et al., 2020] Edward Chou, Florian Tramèr, and Gi- ancarlo Pellegrino. Sentinet: Detecting localized universal attacks against deep learning systems. In IEEE S&P Work- shops, pages 48-54, 2020.","title":"Sentinet: Detecting localized universal attacks against deep learning systems","authors":"Edward Chou, Florian Tramèr, Giancarlo Pellegrino","year":"2020","venue":"IEEE S&P Workshops","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gu et al., 2019] Tianyu Gu, Kang Liu, Brendan Dolan- Gavitt, and Siddharth Garg. Badnets: Evaluating back- dooring attacks on deep neural networks. IEEE Access, 7:47230-47244, 2019.","title":"Badnets: Evaluating backdooring attacks on deep neural networks","authors":"Tianyu Gu, Kang Liu, Brendan Dolan-Gavitt, Siddharth Garg","year":"2019","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Komodakis and Zagoruyko, 2017] Nikos Komodakis and Sergey Zagoruyko. Paying more attention to attention: im- proving the performance of convolutional neural networks via attention transfer. In ICLR, 2017.","title":"Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer","authors":"Zagoruyko Komodakis, Nikos Komodakis, Sergey Zagoruyko","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li et al., 2020a] Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. Neural attention distillation: Erasing backdoor triggers from deep neural networks. In ICLR, 2020.","title":"Neural attention distillation: Erasing backdoor triggers from deep neural networks","authors":"Li","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2020b] Yiming Li, Tongqing Zhai, Baoyuan Wu, Yong Jiang, Zhifeng Li, and Shutao Xia. Rethink- ing the trigger of backdoor attack. arXiv preprint arXiv:2004.04692, 2020.","title":"Rethinking the trigger of backdoor attack","authors":"Li","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2017] Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. Trojaning attack on neural networks. In NDSS, 2017.","title":"Trojaning attack on neural networks","authors":"Liu","year":"2017","venue":"NDSS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2019] Yufan Liu, Jiajiong Cao, Bing Li, Chun- feng Yuan, Weiming Hu, Yangxi Li, and Yunqiang Duan. Knowledge distillation via instance relationship graph. In CVPR, pages 7096-7104, 2019.","title":"Knowledge distillation via instance relationship graph","authors":"Liu","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2020] Yunfei Liu, Xingjun Ma, James Bailey, and Feng Lu. Reflection backdoor: A natural backdoor attack on deep neural networks. In ECCV, pages 182-199, 2020.","title":"Reflection backdoor: A natural backdoor attack on deep neural networks","authors":"Yunfei Liu, Xingjun Ma, James Bailey, Feng Lu","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Papernot et al., 2016] Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a defense to adversarial perturbations against deep neu- ral networks. In IEEE S&P, pages 582-597, 2016.","title":"Distillation as a defense to adversarial perturbations against deep neural networks","authors":"Nicolas Papernot, Patrick Mcdaniel, Xi Wu, Somesh Jha, Ananthram Swami","year":"2016","venue":"IEEE S&P","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Pau et al., 2020] Rodríguez Pau, Dorta Diego, Velazquez, Cucurull Guillem, M. Gonfaus Josep, Roca F. Xavier, and Gonzàlez Jordi. Pay attention to the activations: A mod- ular attention mechanism for fine-grained image recogni- tion. IEEE TM, 22(2):502-514, 2020.","title":"Pay attention to the activations: A modular attention mechanism for fine-grained image recognition","authors":"Rodríguez Pau, Dorta Diego, Cucurull Velazquez, M Gonfaus Guillem, Roca F Josep, Gonzàlez Xavier, Jordi","year":"2020","venue":"IEEE TM","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pei et al., 2021] Yulong Pei, Yanyun Qu, and Junping Zhang. Self-boosting for feature distillation. In IJCAI, 2021.","title":"Self-boosting for feature distillation","authors":"Pei","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/131"},{"position":17,"raw":"[Pirsiavash et al., 2009] Hamed Pirsiavash, Deva Ramanan, and Charless C Fowlkes. Bilinear classifiers for visual recognition. In NIPS, 2009.","title":"Bilinear classifiers for visual recognition","authors":"Hamed Pirsiavash, Deva Ramanan, Charless C Fowlkes","year":"2009","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ren et al., 2021] Jie Ren, Mingjie Li, Zexu Liu, and Quan- shi Zhang. Interpreting and disentangling feature compo- nents of various complexity from DNNs. In ICML, 2021.","title":"Interpreting and disentangling feature components of various complexity from DNNs","authors":"Jie Ren, Mingjie Li, Zexu Liu, Quanshi Zhang","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Song et al., 2021] Yunfei Song, Tian Liu, Tongquan Wei, Xiangfeng Wang, Zhe Tao, and Mingsong Chen. FDA 3 : Federated defense against adversarial attacks for cloud- based iiot applications. IEEE TII, 17:7830-7838, 2021.","title":"FDA 3 : Federated defense against adversarial attacks for cloudbased iiot applications","authors":"Yunfei Song, Tian Liu, Tongquan Wei, Xiangfeng Wang, Zhe Tao, Mingsong Chen","year":"2021","venue":"IEEE TII","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tran et al., 2018] Brandon Tran, Jerry Li, and Aleksander Madry. Spectral signatures in backdoor attacks. In NeurIPS, 2018.","title":"Spectral signatures in backdoor attacks","authors":"Brandon Tran, Jerry Li, Aleksander Madry","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Turner et al., 2018] Alexander Turner, Dimitris Tsipras, and Aleksander Madry. Clean-label backdoor attacks. https: //people.csail.mit.edu/madry/lab/, 2018. Accessed: 2018- 09-27.","title":"Clean-label backdoor attacks","authors":"Alexander Turner, Dimitris Tsipras, Aleksander Madry","year":"2018","venue":null,"doi":null,"url":"https://people.csail.mit.edu/madry/lab/,2018","internal_url":null},{"position":22,"raw":"[Wang et al., 2019] Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In IEEE S&P, pages 707-723, 2019.","title":"Neural cleanse: Identifying and mitigating backdoor attacks in neural networks","authors":"Wang","year":"2019","venue":"IEEE S&P","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wenger et al., 2021] Emily Wenger, Josephine Passananti, Arjun Nitin Bhagoji, Yuanshun Yao, Haitao Zheng, and Ben Y. Zhao. Backdoor attacks against deep learning sys- tems in the physical world. In CVPR, pages 6206-6215, 2021.","title":"Backdoor attacks against deep learning systems in the physical world","authors":"Emily Wenger, Josephine Passananti, Nitin Arjun, Yuanshun Bhagoji, Haitao Yao, Ben Y Zheng, Zhao","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhao et al., 2020] Pu Zhao, Pin-Yu Chen, Payel Das, Karthikeyan Natesan Ramamurthy, and Xue Lin. Bridg- ing mode connectivity in loss landscapes and adversarial robustness. In ICLR, 2020.","title":"Bridging mode connectivity in loss landscapes and adversarial robustness","authors":"Pu Zhao, Pin-Yu Chen, Payel Das, Karthikeyan Natesan Ramamurthy, Xue Lin","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhong et al., 2020] Haoti Zhong, Cong Liao, Cinzia Squic- ciarini, Anna, Sencun Zhu, and Miller David. Backdoor embedding in convolutional neural network models via in- visible perturbation. In CODASPY, pages 97-108, 2020.","title":"Backdoor embedding in convolutional neural network models via invisible perturbation","authors":"Zhong","year":"2020","venue":"CODASPY","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}