{"type":"ScholarlyArticle","title":"Character As Pixels: A Controllable Prompt Adversarial Attacking Framework for Black-Box Text Guided Image Generation Models","authors":[{"name":"Ziyi Kou","slug":"ziyi-kou","given":"Ziyi","family":"Kou","affiliation":"University of Notre Dame"},{"name":"Shichao Pei","slug":"shichao-pei","given":"Shichao","family":"Pei","affiliation":"University of Notre Dame"},{"name":"Yijun Tian","slug":"yijun-tian","given":"Yijun","family":"Tian","affiliation":"University of Notre Dame"},{"name":"Xiangliang Zhang","slug":"xiangliang-zhang","given":"Xiangliang","family":"Zhang","affiliation":"University of Notre Dame"}],"abstract":"In this paper, we study a controllable prompt adversarial attacking problem for text guided image generation (Text2Image) models in the black-box scenario, where the goal is to attack specific visual subjects (e.g., changing a brown dog to white) in a generated image by slightly, if not imperceptibly, perturbing the characters of the driven prompt (e.g., ``brown'' to ``br0wn''). Our study is motivated by the limitations of current Text2Image attacking approaches that still rely on manual trials to create adversarial prompts. To address such limitations, we develop CharGrad, a character-level gradient based attacking framework that replaces specific characters of a prompt with pixel-level similar ones by interactively learning the perturbation direction for the prompt and updating the attacking examiner for the generated image based on a novel proxy perturbation representation for characters.  We evaluate CharGrad using the texts from two public image captioning datasets. Results demonstrate that CharGrad outperforms existing text adversarial attacking approaches on attacking various subjects of generated images by black-box Text2Image models in a more effective and efficient way with less perturbation on the characters of the prompts.","doi":"10.24963/ijcai.2023/109","url":"https://ijcai.org/proceedings/2023/109","pdf":"https://ijcai.org/proceedings/2023/0109.pdf","markdown":"https://ijcai.org/proceedings/2023/109.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/109","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":109,"pages":{"first":983,"last":990},"keywords":["Computer Vision: CV: Neural generative models, auto encoders, GANs","Computer Vision: CV: Adversarial learning, adversarial attack and defense methods"],"videos":[],"references":[{"position":1,"raw":"[Boucher et al., 2022] Nicholas Boucher, Ilia Shumailov, Ross Anderson, and Nicolas Papernot. Bad characters: Imperceptible nlp attacks. In 2022 IEEE Symposium on Security and Privacy (SP), pages 1987-2004. IEEE, 2022.","title":"Bad characters: Imperceptible nlp attacks","authors":"Nicholas Boucher, Ilia Shumailov, Ross Anderson, Nicolas Papernot","year":"2022","venue":"2022 IEEE Symposium on Security and Privacy (SP)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Clark et al., 2022] Jonathan H Clark, Dan Garrette, Iulia Turc, and John Wieting. Canine: Pre-training an effi- cient tokenization-free encoder for language representa- tion. Transactions of the Association for Computational Linguistics, 10:73-91, 2022.","title":"Canine: Pre-training an efficient tokenization-free encoder for language representation","authors":"Jonathan H Clark, Dan Garrette, Iulia Turc, John Wieting","year":"2022","venue":"Transactions of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Devlin et al., 2018] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understand- ing. arXiv preprint arXiv:1810.04805, 2018.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ho et al., 2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.","title":"Denoising diffusion probabilistic models","authors":"Jonathan Ho, Ajay Jain, Pieter Abbeel","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Jin et al., 2020] Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is bert really robust? a strong baseline for natural language attack on text classification and entail- ment. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8018-8025, 2020.","title":"Is bert really robust? a strong baseline for natural language attack on text classification and entailment","authors":"Di Jin, Zhijing Jin, Joey Tianyi Zhou, Peter Szolovits","year":"2020","venue":"Proceedings of the AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, and Piotr Dollár. Microsoft coco: Common objects in context, 2014. cite arxiv:1405.0312.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C Lawrence Zitnick, Piotr Dollár","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Liu et al., 2022] Huijun Liu, Jie Yu, Jun Ma, Shasha Li, Bin Ji, Zibo Yi, Miaomiao Li, Long Peng, and Xiaodong Liu. Textual adversarial attacks by exchanging text-self words. International Journal of Intelligent Systems, 2022.","title":"Textual adversarial attacks by exchanging text-self words","authors":"Huijun Liu, Jie Yu, Jun Ma, Shasha Li, Bin Ji, Zibo Yi, Miaomiao Li, Long Peng, Xiaodong Liu","year":"2022","venue":"International Journal of Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Maheshwary et al., 2021] Rishabh Maheshwary, Saket Ma- heshwary, and Vikram Pudi. Generating natural language attacks in a hard label black box setting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 13525-13533, 2021.","title":"Generating natural language attacks in a hard label black box setting","authors":"Rishabh Maheshwary, Saket Maheshwary, Vikram Pudi","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Radford et al., 2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from nat- ural language supervision. In International Conference on Machine Learning, pages 8748-8763. PMLR, 2021.","title":"Learning transferable visual models from natural language supervision","authors":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ramesh et al., 2022] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022.","title":"Hierarchical text-conditional image generation with clip latents","authors":"Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, Mark Chen","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Rando et al., 2022] Javier Rando, Daniel Paleka, David Lindner, Lennard Heim, and Florian Tramèr. Red- teaming the stable diffusion safety filter. arXiv preprint arXiv:2210.04610, 2022.","title":"Redteaming the stable diffusion safety filter","authors":"Javier Rando, Daniel Paleka, David Lindner, Lennard Heim, Florian Tramèr","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ren et al., 2019] Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. Generating natural language adversarial examples through probability weighted word saliency. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 1085-1097, 2019.","title":"Generating natural language adversarial examples through probability weighted word saliency","authors":"Ren","year":"2019","venue":"Proceedings of the 57th annual meeting of the association for computational linguistics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Rombach et al., 2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10695, 2022.","title":"High-resolution image synthesis with latent diffusion models","authors":"Rombach","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Struppek et al., 2022a] Lukas Struppek, Dominik Hinters- dorf, and Kristian Kersting. The biased artist: Exploiting cultural biases via homoglyphs in text-guided image gen- eration models. arXiv preprint arXiv:2209.08891, 2022.","title":"The biased artist: Exploiting cultural biases via homoglyphs in text-guided image generation models","authors":"Struppek","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Struppek et al., 2022b] Lukas Struppek, Dominik Hinters- dorf, and Kristian Kersting. Rickrolling the artist: Inject- ing invisible backdoors into text-guided image generation models. arXiv preprint arXiv:2211.02408, 2022.","title":"Rickrolling the artist: Injecting invisible backdoors into text-guided image generation models","authors":"Struppek","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tian et al., 2023] Yijun Tian, Kaiwen Dong, Chunhui Zhang, Chuxu Zhang, and Nitesh V Chawla. Heteroge- neous graph masked autoencoders. In AAAI, 2023.","title":"Heterogeneous graph masked autoencoders","authors":"Yijun Tian, Kaiwen Dong, Chunhui Zhang, Chuxu Zhang, Nitesh V Chawla","year":"2023","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ye et al., 2022a] Muchao Ye, Jinghui Chen, Chenglin Miao, Ting Wang, and Fenglong Ma. Leapattack: Hard-label adversarial attack on text via gradient-based optimization. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2307- 2315, 2022.","title":"Leapattack: Hard-label adversarial attack on text via gradient-based optimization","authors":"Ye","year":"2022","venue":"Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ye et al., 2022b] Muchao Ye, Chenglin Miao, Ting Wang, and Fenglong Ma. Texthoaxer: Budgeted hard-label ad- versarial attacks on text. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 36, pages 3877- 3884, 2022.","title":"Texthoaxer: Budgeted hard-label adversarial attacks on text","authors":"Ye","year":"2022","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Young et al., 2014] Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2:67-78, 2014.","title":"From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions","authors":"Peter Young, Alice Lai, Micah Hodosh, Julia Hockenmaier","year":"2014","venue":"Transactions of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}