Research map / Image generation / Text-to-image diffusion
MYOPIA: Protecting Face Privacy from Malicious Personalized Text-to-Image Synthesis via Unlearnable Examples
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MYOPIA, which adds JND- and EoT-constrained error-minimizing perturbations (unlearnable examples) to face images so DreamBooth-style Stable Diffusion fine-tuning fails to capture the target's facial identity.
Problem
Personalized text-to-image models like DreamBooth can be misused with a few face photos to generate harmful imagery of a person; existing defenses (adversarial cloaking such as Anti-DreamBooth) distort generations and are more perceptible.
Approach
Adapt unlearnable examples to diffusion models: alternating error-minimizing perturbation optimization over the denoising loss, with Gaussian-sampled large timestamps, JND-based spatial constraints, and EoT for robustness to blur and JPEG, so the model overfits perturbations rather than facial features.
Key results
- On VGGFace2 with SD 2.1, MYOPIA reduces ISM from 0.47 (no defense) to 0.30 for prompt 1 and from 0.32 to 0.10 for prompt 2, with SSIM 0.878 versus Anti-DreamBooth's 0.746.
- MYOPIA achieves lower FID than Anti-DreamBooth on VGGFace2 (262 vs 448 for prompt 1), indicating less distortion of generated images.
- Transfer across SD versions (trained on V1.4 or V1.5, tested on V2.1) keeps ISM at 0.09 to 0.32 without ensembles.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MYOPIA-protected examples instead of clean images | DreamBooth fine-tuning · VGGFace2, Prompt 1 | ISM: 0.47 -> 0.30 | Helped |
| Add MYOPIA-protected examples instead of clean images | DreamBooth fine-tuning · CelebA-HQ, Prompt 2 | ISM: 0.20 -> 0.14 | Helped |
| Use MYOPIA rather than Anti-DreamBooth protection | DreamBooth fine-tuning · VGGFace2, Prompt 2 | FID: 401 -> 368 | Helped |
| Use MYOPIA rather than Anti-DreamBooth protection | DreamBooth fine-tuning · CelebA-HQ, Prompt 1 | FID: 338 -> 214 | Helped |
| Evaluate MYOPIA with Stable Diffusion 1.5 instead of 2.1 (ablation) | MYOPIA · VGGFace2, Prompt 1 | ISM: 0.30 -> 0.35 | Hurt |
| Evaluate MYOPIA with Stable Diffusion 1.4 instead of 2.1 (ablation) | MYOPIA · VGGFace2, Prompt 2 | ISM: 0.10 -> 0.19 | Hurt |
| Set JND weight λ to 0.995 instead of 0.985 (ablation) | MYOPIA · VGGFace2, Prompt 2 | ISM: 0.10 -> 0.07 | Helped |
| Set JND weight λ to 0.975 instead of 0.985 (ablation) | MYOPIA · VGGFace2, Prompt 1 | ISM: 0.30 -> 0.39 | Hurt |
| Set perturbation bound e to 0.05 instead of 0.5 (ablation) | MYOPIA · VGGFace2, Prompt 1 | ISM: 0.30 -> 0.42 | Hurt |
| Reduce protected-image poisoning rate from 4/4 to 2/4 (ablation) | MYOPIA-protected DreamBooth training set · VGGFace2, Prompt 1 | ISM: 0.30 -> 0.40 | Hurt |
| Generate protected examples with Stable Diffusion 1.4 and test on 2.1 (ablation) | MYOPIA · VGGFace2, Prompt 2 | ISM: 0.32 -> 0.09 | Helped |
| Apply Gaussian blur with kernel size 9 to protected images (ablation) | MYOPIA · VGGFace2, Prompt 1 | ISM: 0.30 -> 0.37 | Hurt |
Surprising findings
- Protection degrades sharply when only one of four training images is protected (ISM 0.44 for prompt 1 at 1/4 poisoning rate).
Limitations
- Protection weakens when most fine-tuning images are clean (fewer than 3 of 4 protected).
- Evaluated only on Stable Diffusion versions 1.4, 1.5 and 2.1 and on 50 identities per dataset.
- Anti-DreamBooth comparison shows comparable ISM and BRISQUE rather than clear superiority on all metrics.
Experimental setup
Datasets: VGGFace2 CelebA-HQ
Benchmarks: VGGFace2 CelebA-HQ
Metrics: ISM BRISQUE FID SSIM LPIPS
Baselines: Anti-DreamBooth No defense
Builds on: Unlearnable examples DreamBooth Anti-DreamBooth Projected Gradient Descent Expectation of Transformation
Closest papers by meaning
- Edit Away and My Face Will not Stay: Personal Biometric Defense against Malicious Generative Editing (CVPR 2025 · similarity 0.86)
- Perturb a Model, Not an Image: Towards Robust Privacy Protection via Anti-Personalized Diffusion Models (NeurIPS 2025 · similarity 0.85)
- SimAC: A Simple Anti-Customization Method for Protecting Face Privacy against Text-to-Image Synthesis of Diffusion Models (CVPR 2024 · similarity 0.84)
- DiffAM: Diffusion-based Adversarial Makeup Transfer for Facial Privacy Protection (CVPR 2024 · similarity 0.84)
- Transferable Facial Privacy Protection against Blind Face Restoration via Domain-Consistent Adversarial Obfuscation (ICML 2024 · similarity 0.84)
- DreamSalon: A Staged Diffusion Framework for Preserving Identity-Context in Editable Face Generation (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.