{"type":"ScholarlyArticle","title":"Imperio: Language-Guided Backdoor Attacks for Arbitrary Model Control","authors":[{"name":"Ka-Ho Chow","slug":"ka-ho-chow","given":"Ka-Ho","family":"Chow","affiliation":"The University of Hong Kong"},{"name":"Wenqi Wei","slug":"wenqi-wei","given":"Wenqi","family":"Wei","affiliation":"Fordham University"},{"name":"Lei Yu","slug":"lei-yu","given":"Lei","family":"Yu","affiliation":"Rensselaer Polytechnic Institute"}],"abstract":"Natural language processing (NLP) has received unprecedented attention. While advancements in NLP models have led to extensive research into their backdoor vulnerabilities, the potential for these advancements to introduce new backdoor threats remains unexplored. This paper proposes Imperio, which harnesses the language understanding capabilities of NLP models to enrich backdoor attacks. Imperio provides a new model control experience. Demonstrated through controlling image classifiers, it empowers the adversary to manipulate the victim model with arbitrary output through language-guided instructions. This is achieved using a language model to fuel a conditional trigger generator, with optimizations designed to extend its language understanding capabilities to backdoor instruction interpretation and execution. Our experiments across three datasets, five attacks, and nine defenses confirm Imperio's effectiveness. It can produce contextually adaptive triggers from text descriptions and control the victim model with desired outputs, even in scenarios not encountered during training. The attack reaches a high success rate without compromising the accuracy of clean inputs and exhibits resilience against representative defenses. Supplementary materials are available at https://khchow.com/Imperio.","doi":"10.24963/ijcai.2024/78","url":"https://ijcai.org/proceedings/2024/78","pdf":"https://ijcai.org/proceedings/2024/0078.pdf","markdown":"https://ijcai.org/proceedings/2024/78.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/78","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Computer Vision","track":"Main Track","number":78,"pages":{"first":704,"last":712},"keywords":["AI Ethics, Trust, Fairness: ETF: Safety and robustness","Computer Vision: CV: Adversarial learning, adversarial attack and defense methods","AI Ethics, Trust, Fairness: ETF: Trustworthy AI"],"videos":[],"references":[{"position":1,"raw":"[Chow et al., 2020] Ka-Ho Chow, Ling Liu, Mehmet Emre Gursoy, Stacey Truex, Wenqi Wei, and Yanzhao Wu. Un- derstanding object detection through an adversarial lens. In European Symposium on Research in Computer Secu- rity (ESORICS), pages 460-481. Springer, 2020.","title":"Understanding object detection through an adversarial lens","authors":"Ka-Ho Chow, Ling Liu, Mehmet Emre Gursoy, Stacey Truex, Wenqi Wei, Yanzhao Wu","year":"2020","venue":"European Symposium on Research in Computer Security (ESORICS)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chung et al., 2022] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416, 2022.","title":"Scaling instruction-finetuned language models","authors":"Chung Hyung Won, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cinà et al., 2023] Antonio Emanuele Cinà, Kathrin Grosse, Ambra Demontis, Sebastiano Vascon, Werner Zellinger, Bernhard A Moser, Alina Oprea, Battista Biggio, Marcello Pelillo, and Fabio Roli. Wild patterns reloaded: A survey of machine learning security against training data poison- ing. ACM Computing Surveys, 55(13s):1-39, 2023.","title":"Wild patterns reloaded: A survey of machine learning security against training data poisoning","authors":"Antonio Emanuele, Cinà, Kathrin Grosse, Ambra Demontis, Sebastiano Vascon, Werner Zellinger, Bernhard A Moser, Alina Oprea, Battista Biggio, Marcello Pelillo, Fabio Roli","year":"2023","venue":"ACM Computing Surveys","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Das et al., 2018] Nilaksh Das, Madhuri Shanbhogue, Shang-Tse Chen, Fred Hohman, Siwei Li, Li Chen, Michael E Kounavis, and Duen Horng Chau. Shield: Fast, practical defense and vaccination for deep learning using jpeg compression. In ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (SIGKDD), pages 196-204, 2018.","title":"Shield: Fast, practical defense and vaccination for deep learning using jpeg compression","authors":"Das","year":"2018","venue":"ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (SIGKDD)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understand- ing. In Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL- HLT), 2019.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Doan et al., 2022] Khoa D Doan, Yingjie Lao, and Ping Li. Marksman backdoor: Backdoor attacks with arbitrary tar- get class. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pages 38260-38273, 2022.","title":"Marksman backdoor: Backdoor attacks with arbitrary target class","authors":"Yingjie Khoa D Doan, Ping Lao, Li","year":"2022","venue":"Advances in Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Du et al., 2022] Wei Du, Yichun Zhao, Boqun Li, Gong- shen Liu, and Shilin Wang. Ppt: Backdoor attacks on pre- trained models via poisoned prompt tuning. In Interna- tional Joint Conference on Artificial Intelligence (IJCAI), pages 680-686, 2022.","title":"Ppt: Backdoor attacks on pretrained models via poisoned prompt tuning","authors":"Wei Du, Yichun Zhao, Boqun Li, Gongshen Liu, Shilin Wang","year":"2022","venue":"International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gao et al., 2019] Yansong Gao, Change Xu, Derui Wang, Shiping Chen, Damith C Ranasinghe, and Surya Nepal. Strip: A defence against trojan attacks on deep neural net- works. In Annual Computer Security Applications Confer- ence (ASCAC), pages 113-125, 2019.","title":"Strip: A defence against trojan attacks on deep neural networks","authors":"Yansong Gao, Change Xu, Derui Wang, Shiping Chen, Surya Damith C Ranasinghe, Nepal","year":"2019","venue":"Annual Computer Security Applications Conference (ASCAC)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gu et al., 2019] Tianyu Gu, Brendan Dolan-Gavitt, and Sid- dharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. IEEE Access, 2019.","title":"Badnets: Identifying vulnerabilities in the machine learning model supply chain","authors":"Tianyu Gu, Brendan Dolan-Gavitt, Siddharth Garg","year":"2019","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hou et al., 2022] Linshan Hou, Zhongyun Hua, Yuhong Li, and Leo Yu Zhang. M-to-n backdoor paradigm: A stealthy and fuzzy attack to deep learning models. arXiv preprint arXiv:2211.01875, 2022.","title":"M-to-n backdoor paradigm: A stealthy and fuzzy attack to deep learning models","authors":"Hou","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hu et al., 2022] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In International Conference on Learn- ing Representations (ICLR), 2022.","title":"Lora: Low-rank adaptation of large language models","authors":"J Edward, Yelong Hu, Phillip Shen, Zeyuan Wallis, Yuanzhi Allen-Zhu, Shean Li, Lu Wang, Weizhu Wang, Chen","year":"2022","venue":"International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kasneci et al., 2023] Enkelejda Kasneci, Kathrin Seßler, Stefan Küchemann, Maria Bannert, Daryna Demen- tieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan Günnemann, Eyke Hüllermeier, et al. Chatgpt for good? on opportunities and challenges of large language mod- els for education. Learning and Individual Differences, 103:102274, 2023.","title":"Chatgpt for good? on opportunities and challenges of large language models for education","authors":"Kasneci","year":"2023","venue":"Learning and Individual Differences","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kumar et al., 2020] Ram Shankar Siva Kumar, Magnus Nyström, John Lambert, Andrew Marshall, Mario Go- ertzel, Andi Comissoneru, Matt Swann, and Sharon Xia. Adversarial machine learning-industry perspectives. In IEEE Security and Privacy Workshops (SPW), pages 69- 75, 2020.","title":"Adversarial machine learning-industry perspectives","authors":"Ram Shankar, Siva Kumar, Magnus Nyström, John Lambert, Andrew Marshall, Mario Goertzel, Andi Comissoneru, Matt Swann, Sharon Xia","year":"2020","venue":"IEEE Security and Privacy Workshops (SPW)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2022] Yiming Li, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. Backdoor learning: A survey. IEEE Trans- actions on Neural Networks and Learning Systems, 2022.","title":"Backdoor learning: A survey","authors":"Yiming Li, Yong Jiang, Zhifeng Li, Shu-Tao Xia","year":"2022","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2023] Yige Li, Xixiang Lyu, Xingjun Ma, Nodens Koren, Lingjuan Lyu, Bo Li, and Yu-Gang Jiang. Recon- structive neuron pruning for backdoor defense. In Interna- tional Conference on Machine Learning (ICML), 2023.","title":"Reconstructive neuron pruning for backdoor defense","authors":"Li","year":"2023","venue":"International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2018] Kang Liu, Brendan Dolan-Gavitt, and Sid- dharth Garg. Fine-pruning: Defending against backdoor- ing attacks on deep neural networks. In International Sym- posium on Research in Attacks, Intrusions, and Defenses (RAID), pages 273-294. Springer, 2018.","title":"Fine-pruning: Defending against backdooring attacks on deep neural networks","authors":"Kang Liu, Brendan Dolan-Gavitt, Siddharth Garg","year":"2018","venue":"International Symposium on Research in Attacks, Intrusions, and Defenses (RAID)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019.","title":"Roberta: A robustly optimized bert pretraining approach","authors":"Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[OpenAI, 2023] OpenAI. GPT-4 technical report, 2023.","title":null,"authors":"Openai","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pan et al., 2022] Xudong Pan, Mi Zhang, Beina Sheng, Ji- aming Zhu, and Min Yang. Hidden trigger backdoor at- tack on NLP models via linguistic style manipulation. In USENIX Security Symposium, pages 3611-3628, 2022.","title":"Hidden trigger backdoor attack on NLP models via linguistic style manipulation","authors":"Xudong Pan, Mi Zhang, Beina Sheng, Jiaming Zhu, Min Yang","year":"2022","venue":"USENIX Security Symposium","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pang et al., 2023] Lu Pang, Tao Sun, Haibin Ling, and Chao Chen. Backdoor cleansing with unlabeled data. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.","title":"Backdoor cleansing with unlabeled data","authors":"Lu Pang, Tao Sun, Haibin Ling, Chao Chen","year":"2023","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rigaki and Garcia, 2023] Maria Rigaki and Sebastian Gar- cia. A survey of privacy attacks in machine learning. ACM Computing Surveys, 56(4):1-34, 2023.","title":"Maria Rigaki and Sebastian Garcia. A survey of privacy attacks in machine learning","authors":"Garcia Rigaki","year":"2023","venue":"ACM Computing Surveys","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Salem et al., 2022] Ahmed Salem, Rui Wen, Michael Backes, Shiqing Ma, and Yang Zhang. Dynamic backdoor attacks against machine learning models. In IEEE Eu- ropean Symposium on Security and Privacy (EuroS&P), pages 703-718, 2022.","title":"Dynamic backdoor attacks against machine learning models","authors":"Ahmed Salem, Rui Wen, Michael Backes, Shiqing Ma, Yang Zhang","year":"2022","venue":"IEEE European Symposium on Security and Privacy (EuroS&P)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Selvaraju et al., 2017] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explana- tions from deep networks via gradient-based localization. In IEEE International Conference on Computer Vision (ICCV), pages 618-626, 2017.","title":"Grad-cam: Visual explanations from deep networks via gradient-based localization","authors":"Michael Ramprasaath R Selvaraju, Abhishek Cogswell, Ramakrishna Das, Devi Vedantam, Dhruv Parikh, Batra","year":"2017","venue":"IEEE International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shi et al., 2023] Jiawen Shi, Yixin Liu, Pan Zhou, and Lichao Sun. Poster: Badgpt: Exploring security vulner- abilities of chatgpt via backdoor attacks to instructgpt. In Network and Distributed Systems Security Symposium (NDSS), 2023.","title":"Poster: Badgpt: Exploring security vulnerabilities of chatgpt via backdoor attacks to instructgpt","authors":"Jiawen Shi, Yixin Liu, Pan Zhou, Lichao Sun","year":"2023","venue":"Network and Distributed Systems Security Symposium (NDSS)","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Si et al., 2023] Wai Man Si, Michael Backes, Yang Zhang, and Ahmed Salem. Two-in-One: A model hijacking at- tack against text generation models. In USENIX Security Symposium, 2023.","title":"Two-in-One: A model hijacking attack against text generation models","authors":"Man Wai, Michael Si, Yang Backes, Ahmed Zhang, Salem","year":"2023","venue":"USENIX Security Symposium","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Szegedy et al., 2014] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations (ICLR), 2014.","title":"Intriguing properties of neural networks","authors":"Szegedy","year":"2014","venue":"International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Thirunavukarasu et al., 2023] Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. Large language models in medicine. Nature Medicine, 29(8):1930-1940, 2023.","title":"Large language models in medicine","authors":"Arun James Thirunavukarasu, Darren Shu, Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, Daniel Shu, Wei Ting","year":"2023","venue":"Nature Medicine","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Touvron et al., 2023] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023.","title":"Llama 2: Open foundation and fine-tuned chat models","authors":"Touvron","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2019] Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In IEEE Symposium on Security and Privacy (SP), pages 707-723, 2019.","title":"Neural cleanse: Identifying and mitigating backdoor attacks in neural networks","authors":"Wang","year":"2019","venue":"IEEE Symposium on Security and Privacy (SP)","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wu and Wang, 2021] Dongxian Wu and Yisen Wang. Ad- versarial neuron pruning purifies backdoored deep mod- els. In Advances in Neural Information Processing Sys- tems (NeurIPS), volume 34, pages 16913-16925, 2021.","title":"Adversarial neuron pruning purifies backdoored deep models","authors":"Wang ; Wu, Dongxian Wu, Yisen Wang","year":"2021","venue":"Advances in Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Xiao et al., 2022] Yu Xiao, Liu Cong, Zheng Mingwen, Wang Yajie, Liu Xinrui, Song Shuxiao, Ma Yuexuan, and Zheng Jun. A multitarget backdooring attack on deep neu- ral networks with random location trigger. International Journal of Intelligent Systems, 37(3):2567-2583, 2022.","title":"A multitarget backdooring attack on deep neural networks with random location trigger","authors":"Yu Xiao, Liu Cong, Zheng Mingwen, Wang Yajie, Liu Xinrui, Song Shuxiao, Ma Yuexuan, Zheng Jun","year":"2022","venue":"International Journal of Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Xu et al., 2018] Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. In Network and Distributed Systems Se- curity Symposium (NDSS), 2018.","title":"Feature squeezing: Detecting adversarial examples in deep neural networks","authors":"Weilin Xu, David Evans, Yanjun Qi","year":"2018","venue":"Network and Distributed Systems Security Symposium (NDSS)","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Xue et al., 2020] Mingfu Xue, Can He, Jian Wang, and Weiqiang Liu. One-to-N & N-to-one: Two advanced back- door attacks against deep learning models. IEEE Trans- actions on Dependable and Secure Computing (TDSC), 19(3):1562-1578, 2020.","title":"One-to-N & N-to-one: Two advanced backdoor attacks against deep learning models","authors":"Xue","year":"2020","venue":"IEEE Transactions on Dependable and Secure Computing (TDSC)","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yan et al., 2023] Jun Yan, Vansh Gupta, and Xiang Ren. Bite: Textual backdoor attacks with iterative trigger injec- tion. In Annual Meeting of the Association for Computa- tional Linguistics (ACL), pages 12951-12968, 2023.","title":"Bite: Textual backdoor attacks with iterative trigger injection","authors":"Yan","year":"2023","venue":"Annual Meeting of the Association for Computational Linguistics (ACL)","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zeng et al., 2021] Yi Zeng, Si Chen, Won Park, Zhuoqing Mao, Ming Jin, and Ruoxi Jia. Adversarial unlearning of backdoors via implicit hypergradient. In International Conference on Learning Representations (ICLR), 2021.","title":"Adversarial unlearning of backdoors via implicit hypergradient","authors":"Yi Zeng, Si Chen, Won Park, Zhuoqing Mao, Ming Jin, Ruoxi Jia","year":"2021","venue":"International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhao et al., 2023] Shuai Zhao, Jinming Wen, Luu Anh Tuan, Junbo Zhao, and Jie Fu. Prompt as triggers for backdoor attack: Examining the vulnerability in language models. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023.","title":"Prompt as triggers for backdoor attack: Examining the vulnerability in language models","authors":"Shuai Zhao, Jinming Wen, Anh Luu, Junbo Tuan, Jie Zhao, Fu","year":"2023","venue":"Conference on Empirical Methods in Natural Language Processing (EMNLP)","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zheng et al., 2022] Runkai Zheng, Rongjun Tang, Jianze Li, and Li Liu. Data-free backdoor removal based on chan- nel lipschitzness. In European Conference on Computer Vision (ECCV), 2022.","title":"Data-free backdoor removal based on channel lipschitzness","authors":"Zheng","year":"2022","venue":"European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}