Research map / Image generation / Text-to-image diffusion
MetaCloak: Preventing Unauthorized Subject-driven Text-to-image Diffusion-based Synthesis via Meta-learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MetaCloak, a meta-learning poisoning method with EOT transformation sampling and denoising-error maximization that makes face images robust to protect them from DreamBooth-style personalization.
Problem
Protecting user images from unauthorized fine-tuning-based subject-driven text-to-image personalization (e.g., DreamBooth) is hard; existing poisoning defenses use heuristic bilevel solutions and break under simple transformations like Gaussian filtering.
Approach
Solves the bilevel poisoning problem via meta-learning over a pool of steps-staggered surrogate diffusion models for transferable perturbations, adds EOT over data transformations, and uses a simple denoising-error maximization loss instead of ground-truth quality metrics.
Key results
- On VGGFace2 under transformation training, MetaCloak reduces SDS to 0.432 versus 0.801 for the best prior baseline (EASPL), a 36.9% degradation vs previous SoTA as stated.
- On CelebA-HQ, MetaCloak reduces SDS to 0.305 vs 0.755 for ASPL, reported as a 45.0% degradation.
- On Replicate online training service, full fine-tuning SDS drops from 98.9% to 21.8%; LoRA fine-tuning is more resistant.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add meta-learning and EOT transformation sampling (ablation) | MetaCloak · VGGFace2, Trans. Training | SDS: 0.879 -> 0.432 | Helped |
| Add meta-learning and EOT transformation sampling (ablation) | MetaCloak · VGGFace2, Trans. Training | IMSCLIP: 0.712 -> 0.644 | Helped |
| Add meta-learning and EOT transformation sampling (ablation) | MetaCloak · VGGFace2, Trans. Training | IMSVGG: 0.193 -> -0.151 | Helped |
| Add meta-learning and EOT transformation sampling (ablation) | MetaCloak · VGGFace2, Trans. Training | CLIP-IQAC: -0.090 -> -0.440 | Helped |
| Add meta-learning and EOT transformation sampling (ablation) | MetaCloak · VGGFace2, Trans. Training | LIQE: 0.891 -> 0.562 | Helped |
| Add EOT transformation sampling to meta-learning (ablation) | MetaCloak · VGGFace2, Trans. Training | SDS: 0.787 -> 0.432 | Helped |
| Add meta-learning to EOT (ablation) | MetaCloak · VGGFace2, Trans. Training | SDS: 0.515 -> 0.432 | Helped |
| Use MetaCloak instead of ASPL | DreamBooth training on VGGFace2 poisons · VGGFace2, Trans. Training | SDS: 0.879 ± 0.321 -> 0.432* ± 0.489 | Helped |
| Use MetaCloak instead of EASPL | DreamBooth training on CelebA-HQ poisons · CelebA-HQ, Trans. Training | IMSVGG: -0.113 ± 0.834 -> -0.637* ± 0.686 | Helped |
Surprising findings
- LoRA fine-tuning appears more resilient to MetaCloak perturbations than full fine-tuning on Replicate.
- Meta-learning alone without EOT did not improve robustness under transformations (SDS 0.787 vs 0.879 baseline), while EOT alone gave larger gains (SDS 0.515).
Limitations
- Robustness to adversarial purification (SR, TVM, JPEG) is partial; SR restores much of the generation ability.
- Protection is weaker under LoRA fine-tuning and on some online services.
- Evaluated only on face datasets with 50 identities per dataset.
Experimental setup
Datasets: VGGFace2 CelebA-HQ
Benchmarks: VGGFace2 CelebA-HQ
Metrics: Subject Detection Score Identity Matching Score CLIP-IQA-C LIQE
Baselines: ASPL E-ASPL FSMG AdvDM Glaze PhotoGuard
Builds on: Anti-DreamBooth DreamBooth MAML AdvDM Unlearnable examples
Closest papers by meaning
- Anti-Diffusion: Preventing Abuse of Modifications of Diffusion-Based Models (AAAI 2025 · similarity 0.87)
- Perturb a Model, Not an Image: Towards Robust Privacy Protection via Anti-Personalized Diffusion Models (NeurIPS 2025 · similarity 0.87)
- Detect-and-Guide: Self-regulation of Diffusion Models for Safe Text-to-Image Generation via Guideline Token Optimization (CVPR 2025 · similarity 0.87)
- Distraction is All You Need: Memory-Efficient Image Immunization against Diffusion-Based Image Editing (CVPR 2024 · similarity 0.86)
- DiffusionGuard: A Robust Defense Against Malicious Diffusion-based Image Editing (ICLR 2025 · similarity 0.86)
- WOUAF: Weight Modulation for User Attribution and Fingerprinting in Text-to-Image Diffusion Models (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.