---
title: "IdentityMask: A Robust Face-Centric Privacy Protection Against Unauthorized Personalization of Diffusion Models"
authors:
  - "Weiwei Tan"
  - "Rui Wang"
  - "Lihua Jing"
  - "Yanjun Zhang"
  - "Runbo Li"
  - "Leo Yu Zhang"
doi: "10.24963/ijcai.2026/187"
url: "https://ijcai.org/proceedings/2026/187"
pdf: "https://ijcai.org/proceedings/2026/0187.pdf"
venue: "Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence"
year: 2026
pages: "1676-1684"
track: "Main Track"
keywords:
  - "Computer Vision: Adversarial learning, adversarial attack and defense methods"
---

# IdentityMask: A Robust Face-Centric Privacy Protection Against Unauthorized Personalization of Diffusion Models

Weiwei Tan, Rui Wang, Lihua Jing, Yanjun Zhang, Runbo Li, Leo Yu Zhang

*Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence*, pages 1676-1684. DOI: [10.24963/ijcai.2026/187](https://doi.org/10.24963/ijcai.2026/187)

## Abstract

Unauthorized personalization based on diffusion models pose a severe and growing threat to digital privacy by enabling the unauthorized replication and exploitation of individual identities. Existing disrupting-based defenses primarily add invisible perturbations arbitrarily across the entire image space to disrupt the generation process. However, we reveal that these methods fundamentally overlook the spatio-temporal dynamics of the personalization process, resulting in inefficient optimization that fails to sufficiently disrupt the core identity encoding mechanism. To mitigate these limitations, we propose IdentityMask, a robust protection framework that shifts the paradigm from arbitrary confusion to precise, targeted feature corruption. By anchoring the perturbation on subject-specific semantics and prioritizing the most critical diffusion timesteps, our framework ensures the disruption is maximized precisely where the identity is encoded. Additionally, a novel manifold projection strategy is introduced to embed the adversarial signals into the intrinsic structure of the image, rendering the protection resilient against state-of-the-art purification. Extensive experiments across diverse datasets, personalization techniques, and defense settings demonstrate that IdentityMask consistently outperforms prior state-of-the-art approaches in both protection efficacy and robustness.