{"type":"ScholarlyArticle","title":"PPT: Backdoor Attacks on Pre-trained Models via Poisoned Prompt Tuning","authors":[{"name":"Wei Du","slug":"wei-du","given":"Wei","family":"Du","affiliation":"University"},{"name":"Yichun Zhao","slug":"yichun-zhao","given":"Yichun","family":"Zhao","affiliation":"University"},{"name":"Boqun Li","slug":"boqun-li","given":"Boqun","family":"Li","affiliation":"University"},{"name":"Gongshen Liu","slug":"gongshen-liu","given":"Gongshen","family":"Liu","affiliation":"University"},{"name":"Shilin Wang","slug":"shilin-wang","given":"Shilin","family":"Wang","affiliation":"University"}],"abstract":"Recently, prompt tuning has shown remarkable performance as a new learning paradigm, which freezes pre-trained language models (PLMs) and only tunes some soft prompts. A fixed PLM only needs to be loaded with different prompts to adapt different downstream tasks. However, the prompts associated with PLMs may be added with some malicious behaviors, such as backdoors. The victim model will be implanted with a backdoor by using the poisoned prompt. In this paper, we propose to obtain the poisoned prompt for PLMs and corresponding downstream tasks by prompt tuning. We name this Poisoned Prompt Tuning method \"PPT\". The poisoned prompt can lead a shortcut between the specific trigger word and the target label word to be created for the PLM. So the attacker can simply manipulate the prediction of the entire model by just a small prompt. Our experiments on various text classification tasks show that PPT can achieve a 99% attack success rate with almost no accuracy sacrificed on original task. We hope this work can raise the awareness of the possible security threats hidden in the prompt.","doi":"10.24963/ijcai.2022/96","url":"https://ijcai.org/proceedings/2022/96","pdf":"https://ijcai.org/proceedings/2022/0096.pdf","markdown":"https://ijcai.org/proceedings/2022/96.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/96","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"AI Ethics, Trust, Fairness","track":"Main Track","number":96,"pages":{"first":680,"last":686},"keywords":["AI Ethics, Trust, Fairness: Safety & Robustness","Natural Language Processing: Language Models","Natural Language Processing: Other"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986754","length_sec":89,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984897","length_sec":199,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Devlin et al., 2018] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understand- ing. arXiv preprint arXiv:1810.04805, 2018.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ding et al., 2021] Ning Ding, Shengding Hu, Weilin Zhao, Yulin Chen, Zhiyuan Liu, Hai-Tao Zheng, and Maosong Sun. Openprompt: An open-source framework for prompt-learning. arXiv preprint arXiv:2111.01998, 2021.","title":"Openprompt: An open-source framework for prompt-learning","authors":"Ning Ding, Shengding Hu, Weilin Zhao, Yulin Chen, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Founta et al., 2018] Antigoni Maria Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. Large scale crowd- sourcing and characterization of twitter abusive behavior. In Twelfth International AAAI Conference on Web and So- cial Media, 2018.","title":"Large scale crowdsourcing and characterization of twitter abusive behavior","authors":"Maria Antigoni, Constantinos Founta, Despoina Djouvas, Ilias Chatzakou, Jeremy Leontiadis, Gianluca Blackburn, Athena Stringhini, Michael Vakali, Nicolas Sirivianos, Kourtellis","year":"2018","venue":"Twelfth International AAAI Conference on Web and Social Media","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gu et al., 2017] Tianyu Gu, Brendan Dolan-Gavitt, and Sid- dharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733, 2017.","title":"Badnets: Identifying vulnerabilities in the machine learning model supply chain","authors":"Tianyu Gu, Brendan Dolan-Gavitt, Siddharth Garg","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hambardzumyan et al., 2021] Karen Hambardzumyan, Hrant Khachatrian, and Jonathan May. Warp: Word- level adversarial reprogramming. arXiv preprint arXiv:2101.00121, 2021.","title":"Warp: Wordlevel adversarial reprogramming","authors":"Karen Hambardzumyan, Hrant Khachatrian, Jonathan May","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Jia et al., 2021] Jinyuan Jia, Yupei Liu, and Neil Zhen- qiang Gong. Badencoder: Backdoor attacks to pre-trained encoders in self-supervised learning. arXiv preprint arXiv:2108.00352, 2021.","title":"Badencoder: Backdoor attacks to pre-trained encoders in self-supervised learning","authors":"Jia","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kurita et al., 2020] Keita Kurita, Paul Michel, and Graham Neubig. Weight poisoning attacks on pre-trained models. arXiv preprint arXiv:2004.06660, 2020.","title":"Weight poisoning attacks on pre-trained models","authors":"Keita Kurita, Paul Michel, Graham Neubig","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lester et al., 2021] Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691, 2021.","title":"The power of scale for parameter-efficient prompt tuning","authors":"Brian Lester, Rami Al-Rfou, Noah Constant","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li and Liang, 2021] Xiang Lisa Li and Percy Liang. Prefix- tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190, 2021.","title":"Prefixtuning: Optimizing continuous prompts for generation","authors":"Lisa Xiang, Percy Li, Liang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liu et al., 2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019.","title":"Roberta: A robustly optimized bert pretraining approach","authors":"Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2021] Xiao Liu, Kaixuan Ji, Yicheng Fu, Zhengxiao Du, Zhilin Yang, and Jie Tang. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint arXiv:2110.07602, 2021.","title":"P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks","authors":"Xiao Liu, Kaixuan Ji, Yicheng Fu, Zhengxiao Du, Zhilin Yang, Jie Tang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Maas et al., 2011] Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Pro- ceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pages 142-150, 2011.","title":"Learning word vectors for sentiment analysis","authors":"Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, Christopher Potts","year":"2011","venue":"Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Metsis et al., 2006] Vangelis Metsis, Ion Androutsopoulos, and Georgios Paliouras. Spam filtering with naive bayes- which naive bayes? In CEAS, volume 17, pages 28-69.","title":"Vangelis Metsis, Ion Androutsopoulos, and Georgios Paliouras. Spam filtering with naive bayeswhich naive bayes?","authors":"Metsis","year":"2006","venue":"CEAS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Raffel et al., 2019] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683, 2019.","title":"Exploring the limits of transfer learning with a unified text-to-text transformer","authors":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Rajpurkar et al., 2016] Pranav Rajpurkar, Jian Zhang, Kon- stantin Lopyrev, and Percy Liang. Squad: 100,000+ ques- tions for machine comprehension of text. arXiv preprint arXiv:1606.05250, 2016.","title":"Squad: 100,000+ questions for machine comprehension of text","authors":"Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, Percy Liang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Saha et al., 2021] Aniruddha Saha, Ajinkya Tejankar, Soroush Abbasi Koohpayegani, and Hamed Pirsiavash. Backdoor attacks on self-supervised learning. arXiv preprint arXiv:2105.10123, 2021.","title":"Backdoor attacks on self-supervised learning","authors":"Aniruddha Saha, Ajinkya Tejankar, Soroush Abbasi Koohpayegani, Hamed Pirsiavash","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sakkis et al., 2003] Georgios Sakkis, Ion Androutsopoulos, Georgios Paliouras, Vangelis Karkaletsis, Constantine D Spyropoulos, and Panagiotis Stamatopoulos. A memory- based approach to anti-spam filtering for mailing lists. In- formation retrieval, 6(1):49-73, 2003.","title":"A memorybased approach to anti-spam filtering for mailing lists","authors":"Georgios Sakkis, Ion Androutsopoulos, Georgios Paliouras, Vangelis Karkaletsis, Constantine D Spyropoulos, Panagiotis Stamatopoulos","year":"2003","venue":"Information retrieval","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shin et al., 2020] Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. Au- toprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980, 2020.","title":"Autoprompt: Eliciting knowledge from language models with automatically generated prompts","authors":"Taylor Shin, Yasaman Razeghi, Robert L Logan, I V, Eric Wallace, Sameer Singh","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Socher et al., 2013] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1631-1642, 2013.","title":"Recursive deep models for semantic compositionality over a sentiment treebank","authors":"Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, Christopher Potts","year":"2013","venue":"Proceedings of the 2013 conference on empirical methods in natural language processing","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2019] Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Superglue: A stick- ier benchmark for general-purpose language understand- ing systems. arXiv preprint arXiv:1905.00537, 2019.","title":"Superglue: A stickier benchmark for general-purpose language understanding systems","authors":"Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, Samuel R Bowman","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yang et al., 2021] Wenkai Yang, Lei Li, Zhiyuan Zhang, Xuancheng Ren, Xu Sun, and Bin He. Be careful about poisoned word embeddings: Exploring the vulnerability of the embedding layers in nlp models. arXiv preprint arXiv:2103.15543, 2021.","title":"Be careful about poisoned word embeddings: Exploring the vulnerability of the embedding layers in nlp models","authors":"Wenkai Yang, Lei Li, Zhiyuan Zhang, Xuancheng Ren, Xu Sun, Bin He","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zampieri et al., 2019] Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. Semeval-2019 task 6: Identifying and categoriz- ing offensive language in social media (offenseval). arXiv preprint arXiv:1903.08983, 2019.","title":"Semeval-2019 task 6: Identifying and categorizing offensive language in social media","authors":"Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, Ritesh Kumar","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2021] Zhengyan Zhang, Guangxuan Xiao, Yongwei Li, Tian Lv, Fanchao Qi, Zhiyuan Liu, Yasheng Wang, Xin Jiang, and Maosong Sun. Red alarm for pre- trained models: Universal vulnerability to neuron-level backdoor attacks. In ICML 2021 Workshop on Adversarial Machine Learning, 2021.","title":"Red alarm for pretrained models: Universal vulnerability to neuron-level backdoor attacks","authors":"Zhang","year":"2021","venue":"ICML 2021 Workshop on Adversarial Machine Learning","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}