FORGE - Facial Optimization for Robust Generative Enhancement Attack

Remarkable advancements in latent diffusion models (LDMs) allow photo-realistic image generation from simple text prompts. However, their misuse by malicious actors has raised ethical concerns, particularly regarding facial forgery and privacy violations. Recent studies have proposed anti-customization methods to prevent this issue. Unfortunately, their focus on protective performance often results in noticeable perturbations, making them impractical for real-world applications. Moreover, these methods struggle to remain effective against common image transformations such as JPEG compression and Gaussian blur. In this paper, we propose a novel approach, namely FORGE, that enhances both imperceptibility and robustness against common image transformations. Our method leverages face-parsing-guided perturbation amplification and superpixel-based random noise scaling to selectively apply adversarial perturbations to perceptually important facial regions, optimizing the trade-off between defense effectiveness and visual imperceptibility. Furthermore, we integrate Expectation over Transformation (EoT) to improve robustness against various image transformations, allowing our perturbations to remain effective against compression and geometric modifications. Extensive experiments on CelebA-HQ and VGGFace2 demonstrate that FORGE significantly reduces perturbation visibility while maintaining defense performance compared to existing approaches.