Research map / Image generation / Image restoration
Erase Diffusion: Empowering Object Removal Through Calibrating Diffusion Pathways
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EraDiff, a fine-tuned SD2-Inpaint for object removal that uses a Chain-Rectifying Optimization (synthetic object-to-background mix-up diffusion chains) and a Self-Rectifying Attention mask to suppress artifacts.
Problem
Diffusion-based erase inpainting (object removal) often fails to eliminate target objects and generates unexpected objects or artifacts, because standard diffusion training paths go from noise to images containing the object rather than from object to background.
Approach
Builds synthetic object-background image pairs by matting and pasting objects, creates mix-up latent states to define new diffusion transition pathways from objects to backgrounds (CRO), and replaces self-attention with a mask-based attention that makes foreground regions attend to background tokens (SRA).
Key results
- EraDiff achieves the best Local FID (3.799) and best LPIPS (0.192) on a 10,000-sample OpenImages V5 test subset, versus 8.852 Local FID for SD2-Inpaint.
- In the ablation, removing both CRO and SRA drops GPT-4o object-elimination success from 83.43% to 27.80%.
- Its FID (6.540) is mid-range, behind SD2-Inpaint (3.805) and above LaMa (7.533) only in some cases, so it does not lead on FID.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EraDiff instead of LaMa | Object removal on OpenImages V5 test · OpenImages V5 test | Local FID: 6.091 -> 3.799 | Helped |
| Use EraDiff instead of LaMa | Object removal on OpenImages V5 test · OpenImages V5 test | LPIPS: 0.219 -> 0.192 | Helped |
| Use EraDiff instead of SD2-Inpaint | Object removal on OpenImages V5 test · OpenImages V5 test | FID: 3.805 -> 6.540 | Hurt |
| Add CRO to EraDiff (ablation) | EraDiff · OpenImages V5 test | Local FID: 5.713 -> 3.799 | Helped |
| Add CRO to EraDiff (ablation) | EraDiff · OpenImages V5 test | GPT score: 72.96% -> 83.43% | Helped |
| Add SRA to EraDiff (ablation) | EraDiff · OpenImages V5 test | Local FID: 4.950 -> 3.799 | Helped |
| Add SRA to EraDiff (ablation) | EraDiff · OpenImages V5 test | GPT score: 78.54% -> 83.43% | Helped |
| Add CRO and SRA to EraDiff (ablation) | EraDiff · OpenImages V5 test | GPT score: 27.80% -> 83.43% | Helped |
| Use the mix-up strategy during training (ablation) | EraDiff · OpenImages V5 test | training convergence | Helped |
Surprising findings
- The SD2-Inpaint baseline achieves the lowest FID (3.805) yet frequently fails to remove objects, showing FID does not capture elimination quality.
Limitations
- FID is not best; EraDiff is only mid-range on FID.
- Evaluation relies largely on GPT-4o judgments and a 20-expert user study with a single benchmark.
- Training details such as number of GPUs and training steps are not fully specified.
Experimental setup
Datasets: OpenImages V5
Benchmarks: OpenImages V5
Metrics: FID LPIPS Local FID GPT-4o pairwise win rate user study score
Baselines: SD2-Inpaint LaMa Inst-Inpaint PowerPaint
Builds on: Stable Diffusion inpainting (SD2-Inpaint) DDIM LaMa
Closest papers by meaning
- Attentive Eraser: Unleashing Diffusion Model’s Object Removal Potential via Self-Attention Redirection Guidance (AAAI 2025 · similarity 0.86)
- Diffusion Handles Enabling 3D Edits for Diffusion Models by Lifting Activations to 3D (CVPR 2024 · similarity 0.84)
- DiffEditor: Boosting Accuracy and Flexibility on Diffusion-based Image Editing (CVPR 2024 · similarity 0.83)
- Erasing Undesirable Influence in Diffusion Models (CVPR 2025 · similarity 0.83)
- OpenSDI: Spotting Diffusion-Generated Images in the Open World (CVPR 2025 · similarity 0.83)
- A Diffusion-Based Framework for Occluded Object Movement (AAAI 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.