Research map / Image generation / Text-to-image diffusion
You Don’t Need All That Attention: Surgical Memorization Mitigation in Text-to-Image Diffusion Models
Kairan Zhao, Eleni Triantafillou, Peter Triantafillou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| attenuate cross-attention at all detected spike tokens instead of only EOT/padding | CA attenuation vs Ren et al. · SD v2.0 template memorization | SSCD: 0.36 -> 0.19 | Improved |
| attenuate cross-attention at all detected spike tokens instead of only EOT/padding | CA attenuation vs Ren et al. · SD v1.4 template memorization | SSCD: 0.60 -> 0.53 | Improved |
| use CA attenuation alone (no GUARD repulsion term) (ablation) | CA attenuation · SD v2.0 template memorization | FID: 245.850 vs 212.727 for CA-in-GUARD | Hurt |
| embed CA attenuation as the positive target of GUARD (attraction-repulsion guidance) (ablation) | CA attenuation · SD v1.4 verbatim memorization | SSCD: 0.109 -> 0.079 | Improved |
| embed CA attenuation as the positive target of GUARD (ablation) | CA attenuation · SD v1.4 verbatim memorization | CLIP score: 0.282 -> 0.266 | Hurt |
| embed CA attenuation as the positive target of GUARD (ablation) | CA attenuation · SD v1.4 verbatim memorization | FID: 164.660 -> 158.115 | Improved |
| embed CA attenuation as the positive target of GUARD (ablation) | CA attenuation · SD v2.0 template memorization | SSCD: 0.193 -> 0.193 | No clear effect |
| attenuate CA only at EOT token (prior approach) | Ren et al. · SD v1.4 template memorization | SSCD: 0.602 vs 0.530 for CA attenuation | Hurt |
| CA-in-GUARD instead of best prior method | CA-in-GUARD · SD v1.4 template memorization | SSCD: 0.479 -> 0.517 (Han et al. best prior is lower, 0.479) | Improved |
| CA-in-GUARD instead of no mitigation | Stable Diffusion · SD v1.4 verbatim memorization | SSCD: 0.875 -> 0.079 | Improved |
| CA-in-GUARD instead of no mitigation | Stable Diffusion · SD v1.4 verbatim memorization | CLIP score: 0.346 -> 0.266 | Hurt |
Closest papers by meaning
- Towards Memorization-Free Diffusion Models (CVPR 2024 · similarity 0.89)
- Detecting, Explaining, and Mitigating Memorization in Diffusion Models (ICLR 2024 · similarity 0.89)
- Adjusting Initial Noise to Mitigate Memorization in Text-to-Image Diffusion Models (NeurIPS 2025 · similarity 0.89)
- Semantic Surgery: Zero-Shot Concept Erasure in Diffusion Models (NeurIPS 2025 · similarity 0.88)
- Exploring Local Memorization in Diffusion Models via Bright Ending Attention (ICLR 2025 · similarity 0.88)
- How Diffusion Models Memorize (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
