Feint and Attack: Jailbreaking and Protecting LLMs via Attention Distribution Modeling

Pu, Rui; Li, Chaozhuo; Ha, Rui; Chen, Zejian; Zhang, Litian; Liu, Zheng; Qiu, Lirong; Ye, Zaisheng

doi:10.24963/ijcai.2025/56

Feint and Attack: Jailbreaking and Protecting LLMs via Attention Distribution Modeling

Rui Pu, Chaozhuo Li, Rui Ha, Zejian Chen, Litian Zhang, Zheng Liu, Lirong Qiu, Zaisheng Ye

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 493-501.

https://doi.org/10.24963/ijcai.2025/56

Keywords

AI Ethics, Trust, Fairness: ETF: Trustworthy AI
AI Ethics, Trust, Fairness: ETF: Safety and robustness

Abstract

Most jailbreak methods for large language models (LLMs) focus on superficially improving attack success through manually defined rules. However, they fail to uncover the underlying mechanisms within target LLMs that explain why an attack succeeds or fails. In this paper, we propose investigating the phenomenon of jailbreaks and defenses for LLMs from the perspective of attention distributions within the models. A preliminary experiment reveals that the success of a jailbreak is closely linked to the LLM's attention on sensitive words.Inspired by this interesting finding, we propose incorporating critical signals derived from internal attention distributions within LLMs, namely Attention Intensity on Sensitive Words and Attention Dispersion Entropy, to guide both attacks and defenses. Drawing inspiration from the concept of "Feint and Attack", we introduce an attention-guided jailbreak model, ABA, which redirects the model's attention to benign contexts, and an attention-based defense model, ABD, designed to detect attacks by analyzing internal attention entropy. Experimental results demonstrate the superiority of our proposal when compared to SOTA baselines.

Rui Pu, Chaozhuo Li, Rui Ha, Zejian Chen, Litian Zhang, Zheng Liu, Lirong Qiu, and Zaisheng Ye. 2025. Feint and Attack: Jailbreaking and Protecting LLMs via Attention Distribution Modeling. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25. IJCAI, 493–501. https://doi.org/10.24963/ijcai.2025/56

@inproceedings{ijcai2025p56,
  title     = {Feint and Attack: Jailbreaking and Protecting LLMs via Attention Distribution Modeling},
  author    = {Pu, Rui and Li, Chaozhuo and Ha, Rui and Chen, Zejian and Zhang, Litian and Liu, Zheng and Qiu, Lirong and Ye, Zaisheng},
  booktitle = {Proceedings of the Thirty-Fourth International Joint Conference on
               Artificial Intelligence, {IJCAI-25}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {James Kwok},
  pages     = {493--501},
  year      = {2025},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2025/56},
  url       = {https://doi.org/10.24963/ijcai.2025/56},
}

TY  - CPAPER
TI  - Feint and Attack: Jailbreaking and Protecting LLMs via Attention Distribution Modeling
AU  - Pu, Rui
AU  - Li, Chaozhuo
AU  - Ha, Rui
AU  - Chen, Zejian
AU  - Zhang, Litian
AU  - Liu, Zheng
AU  - Qiu, Lirong
AU  - Ye, Zaisheng
T2  - Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2025
SP  - 493
EP  - 501
DO  - 10.24963/ijcai.2025/56
UR  - https://doi.org/10.24963/ijcai.2025/56
ER  -