Research map / Detection and segmentation / Open-set and few-shot detection
SmartEraser: Remove Anything from Images using Masked-Region Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SmartEraser, a Stable Diffusion-based object removal model trained with a Masked-Region Guidance paradigm (keeping the masked region in the input) on a synthetic 1M-triplet dataset Syn4Removal.
Problem
Object removal with user masks in the standard mask-and-inpaint paradigm discards the masked region, causing the model to regenerate objects and distort surrounding context.
Approach
Retain the masked region in the model input as guidance (input [m, x] instead of [m, x⊙(1−m)]), enabled by a synthetic copy-paste dataset that provides input/mask/ground-truth triplets, plus mask enhancement augmentation and CLIP-based visual guidance injected into the text conditioning.
Key results
- On RORD-Val, SmartEraser reaches FID 16.030 and PSNR 19.994 vs. PowerPaint's 24.058 and 18.100.
- On DEFACTO-Val, SmartEraser achieves FID 3.405 and PSNR 25.363, the best among all compared methods.
- In a 30-person user study, SmartEraser received 0.56/0.65/0.58 preference shares for image quality, FB-consistency, and removal accuracy, far above baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace mask-and-inpaint with masked-region guidance (ablation) | SmartEraser · RORD-Val | FID: 33.131 -> 24.526 | Helped |
| replace mask-and-inpaint with masked-region guidance (ablation) | SmartEraser · RORD-Val | PSNR: 16.535 -> 17.176 | Helped |
| add mask enhancement (ablation) | SmartEraser with masked-region guidance · RORD-Val | FID: 24.526 -> 19.421 | Helped |
| add mask enhancement (ablation) | SmartEraser with masked-region guidance · RORD-Val | PSNR: 17.176 -> 18.917 | Helped |
| add CLIP-based visual guidance (ablation) | SmartEraser with masked-region guidance and mask enhancement · RORD-Val | FID: 19.421 -> 16.030 | Helped |
| add CLIP-based visual guidance (ablation) | SmartEraser with masked-region guidance and mask enhancement · RORD-Val | PSNR: 18.917 -> 19.994 | Helped |
| use SmartEraser instead of PowerPaint | object removal methods · RORD-Val | FID: 24.058 -> 16.030 | Helped |
| use SmartEraser instead of PowerPaint | object removal methods · DEFACTO-Val | FID: 6.582 -> 3.405 | Helped |
| use SmartEraser instead of PowerPaint | object removal methods · Syn4Removal-Val | FID: 5.025 -> 4.386 | Helped |
Surprising findings
- Retaining the masked region in the input causes a shortcut risk (copying masked content) for standard inpainting-style data, which is why a copy-paste triplet dataset was needed.
Limitations
- Mask shape mismatch between training and user masks can leave target objects partially removed or context altered.
- Evaluation on RORD-Val and DEFACTO-Val is partly derived from existing datasets; user study uses only 20 image sets with 30 participants.
Experimental setup
Datasets: Syn4Removal OpenImages-v7 COCONut SAM-1B MSCOCO RORD-Val DEFACTO-Val
Benchmarks: RORD-Val DEFACTO-Val Syn4Removal-Val
Metrics: FID CMMD ReMOVE LPIPS SSIM PSNR user preference
Baselines: ZITS++ MAT LaMa BLD RePaint SD-Inpaint CLIPAway PowerPaint
Builds on: Stable Diffusion v1.5 CLIP AlphaCLIP PowerPaint CLIPAway
Closest papers by meaning
- RORem: Training a Robust Object Remover with Human-in-the-Loop (CVPR 2025 · similarity 0.87)
- GeoRemover: Removing Objects and Their Causal Visual Artifacts (NeurIPS 2025 · similarity 0.86)
- Instruct2See: Learning to Remove Any Obstructions Across Distributions (ICML 2025 · similarity 0.86)
- MiniMax-Remover: Taming Bad Noise Helps Video Object Removal (NeurIPS 2025 · similarity 0.85)
- Attentive Eraser: Unleashing Diffusion Model’s Object Removal Potential via Self-Attention Redirection Guidance (AAAI 2025 · similarity 0.84)
- Paint by Inpaint: Learning to Add Image Objects by Removing Them First (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.