{"type":"ScholarlyArticle","title":"Understanding and Exploiting Phase Sensitivity for Attacking Large Vision–Language Models","authors":[{"name":"Daizong Liu","slug":"daizong-liu","given":"Daizong","family":"Liu","affiliation":""},{"name":"Junhao Dong","slug":"junhao-dong","given":"Junhao","family":"Dong","affiliation":""},{"name":"Xiang Fang","slug":"xiang-fang","given":"Xiang","family":"Fang","affiliation":""},{"name":"Hongyang He","slug":"hongyang-he","given":"Hongyang","family":"He","affiliation":""},{"name":"Keyan Jin","slug":"keyan-jin","given":"Keyan","family":"Jin","affiliation":""},{"name":"Zhongliang Guo","slug":"zhongliang-guo","given":"Zhongliang","family":"Guo","affiliation":""},{"name":"Xiaoye Qu","slug":"xiaoye-qu","given":"Xiaoye","family":"Qu","affiliation":""},{"name":"Keke Tang","slug":"keke-tang","given":"Keke","family":"Tang","affiliation":""}],"abstract":"Although Large Vision-Language Models (LVLMs) have demonstrated remarkable reasoning capabilities across various downstream multimodal tasks, they are proven to be vulnerable to carefully designed adversarial examples. Existing LVLM attackers show that exploring external components of adversarial guidance (e.g., forcing adversarial alignment, resembling harmful features) can help improve adversarial effects. However, leveraging the intrinsic patterns of LVLMs to induce adversarial perturbation generation by exploring how LVLMs perceive images has not been deeply studied. Inspired by the cognitive science, in this paper, we make the first attempt to investigate the interference of adversarial perturbation from the perspectives of image phase, and find that LVLMs are sensitive to the phase-aware image structure. Motivated by this, we propose a novel LVLM attack method called BadPhase with further backdoor designs, to implant adversarial phase as triggers into any image inputs via data poisoning so as to control the LVLMs’ predictions. A textual trigger and a backdoor perturbation switcher are also introduced to activate the malicious behavior only when both triggers are present. The whole backdoor optimization is implemented at the test-time to reduce the resource reliance. Experiments on four popular LVLMs and three benchmarks demonstrate the effectiveness of our proposed method.","doi":"10.24963/ijcai.2026/159","url":"https://ijcai.org/proceedings/2026/159","pdf":"https://ijcai.org/proceedings/2026/0159.pdf","markdown":"https://ijcai.org/proceedings/2026/159.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/159","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Computer Vision","track":"Main Track","number":159,"pages":{"first":1424,"last":1432},"keywords":["Computer Vision: Adversarial learning, adversarial attack and defense methods","Computer Vision: Vision, language and reasoning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}