AEGIS: An Inference-Time Defense Against Visual Synonym Jailbreak Attacks on Text-to-Image Models
This paper presents AEGIS (Adaptive Evasion Guard via Identification and Steering), a defense mechanism against Visual Synonym Attacks (VSA) on text-to-image diffusion models. VSA is a jailbreak technique where seemingly benign prompts generate prohibited imagery through implicit visual associations, bypassing existing safety filters that focus on explicit unsafe concepts. The authors identify that VSA and explicit unsafe prompts converge through sparse semantic-injecting attention heads during generation. AEGIS applies similarity-aware repulsion only at these vulnerable attention heads at inference time. Evaluated against 16 baselines on SD 1.4, SD 2.1, and FLUX.1, AEGIS reduces attack success rates to near zero for in-domain attacks while preserving benign image fidelity and avoiding over-suppression of visually similar but safe concepts.
Key quotes
Existing alignment paradigms... leave a blind spot for visual synonym attacks (VSA), a jailbreak where benign-looking prompts elicit prohibited imagery through implicit visual associations.
The core challenge is that VSA hides the unsafe target at the textual surface while revealing it through generation-time visual-semantic convergence.
Our mechanistic analysis shows that VSA and explicit unsafe prompts converge through sparse semantic-injecting attention heads, which serve as inference-time bottlenecks for prohibited visual semantics.
We propose AEGIS (Adaptive Evasion Guard via Identification and Steering), an inference-time defense that applies similarity-aware repulsion only at the identified vulnerable heads.
On SD 1.4, it reduces ASR to 0.00/0.03 for in-domain violence/nudity VSA and achieves ASRs ≤ 0.09 on out-of-domain explicit and adversarial attacks.
From the article
You might also wanna read
JaiLIP: A new image-based jailbreak attack method for Vision-Language Models
This paper introduces JaiLIP (Jailbreaking with Loss-guided Image Perturbation), a novel attack method targeting Vision-Language Models (VLM
arxiv.org·10d ago
Adversarial Poetry Functions as Universal Jailbreak Technique for Large Language Models
Research demonstrates that adversarial poetry serves as an effective universal jailbreak technique for Large Language Models (LLMs). Across
arxiv.org·7mo agoJavelinGuard: Low-Cost Transformer Architectures for LLM Security
Study Reveals Domain-Camouflaged Injection Attacks Bypass LLM Detection Systems
This research paper identifies a critical vulnerability in injection detectors used to protect LLM agents. The authors demonstrate that when
arxiv.org·1mo ago
RAG Poisoning: How Attackers Corrupt AI Knowledge Bases Through Document Injection
RAG poisoning is a cybersecurity attack where adversaries inject malicious or fabricated documents into retrieval-augmented generation (RAG)
aminrj.com·3mo ago
Cross-Trace Verification Protocol: A Framework for Detecting Malicious Code in AI-Generated Programs
Researchers present Cross-Trace Verification Protocol (CTVP), a novel AI control framework for detecting malicious code generated by large l
arxiv.org·5mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.