Research map / Large language models / Language, safety and interpretability
The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMs
Zichen Wen, Jiashu Qu, Zhaorun Chen, Xiaoya Lu, Dongrui Liu, Zhiyuan Liu, Ruixi Wu, Yicun Yang, Xiangqi Jin, Haoyun Xu, Xuyang Liu, Weijia Li and 4 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add interleaved mask-text prompting (DiJA) over ReNeLLM | dLLM jailbreak · HarmBench Dream-Instruct | ASR-e (%): 6.5 -> 57.5 | Improved |
| add masked regions to the prompt (ablation) | DiJA · HarmBench LLaDA-8B-Instruct | ASR-e (%): 14.0 -> 55.5 | Improved |
| add benign separators between masks (ablation) | DiJA · HarmBench LLaDA-8B-Instruct | ASR-e (%): 16.8 -> 55.5 | Improved |
| add LLM prompt refinement with few-shot demos (ablation) | DiJA · HarmBench LLaDA-8B-Instruct | ASR-e (%): 54.8 -> 55.5 | No clear effect |
| use GPT-4o instead of Qwen2.5-7B-Instruct as prompt constructor | DiJA · HarmBench LLaDA-Instruct | ASR-e (%): 55.5 -> 60.0 | Improved |
| increase number of mask spans from 1 to 3 (50 token budget) (ablation) | DiJA · HarmBench LLaDA-8B-Instruct | ASR-k (%): 62.3 -> 96.3 | Improved |
| increase number of mask spans from 3 to 10 (ablation) | DiJA · HarmBench LLaDA-8B-Instruct | ASR-k (%): 96.3 -> 87.5 | Hurt |
| apply defenses Self-reminder and RPO | DiJA · three dLLMs, multiple benchmarks | ASR-e / SRS | No clear effect |
| fine-tune with refusal-aware denoising alignment | LLaDA-Instruct · HarmBench | ASR-e (%): 55.5 -> 32.5 | Improved |
| fine-tune with refusal-aware denoising alignment | LLaDA-Instruct · JailBreakBench | ASR-e (%): 81.0 -> 25.0 | Improved |
| refusal-aware alignment effect on general capability | LLaDA-Instruct · general benchmarks (7) | accuracy: GPQA 32.4 -> 27.5; GSM8K 78.5 -> 78.9; MMLU-Pro 35.1 -> 36.1 | Mixed |
| use too few masked tokens (10) or too many (50) (ablation) | DiJA · JailbreakBench and StrongREJECT | ASR | Hurt |
Closest papers by meaning
- Safety Alignment Should be Made More Than Just a Few Tokens Deep (ICLR 2025 · similarity 0.91)
- MaskForge: Structure-Aware Adaptive Attacks for Jailbreaking Diffusion Large Language Models (NeurIPS 2026 · similarity 0.91)
- Fail-Closed Alignment for Large Language Models (NeurIPS 2026 · similarity 0.90)
- Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution Strategy (CVPR 2025 · similarity 0.90)
- Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models? (CVPR 2025 · similarity 0.89)
- Safety Layers in Aligned Large Language Models: The Key to LLM Security (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
