Research map / Detection and segmentation / Open-set and few-shot detection
EntityErasure: Erasing Entity Cleanly via Amodal Entity Segmentation and Completion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EntityErasure, a diffusion inpainting method that erases objects by first predicting amodal entity segmentation and then completing entities with entity attention restricted to matching segments, plus two segmentation-based metrics (MSN, MARS) for measuring generated sundries.
Problem
Object erasure with diffusion inpainting often generates unwanted sundries (new spurious objects or textures) in the erased region, and existing metrics like FID and CLIP distance fail to measure this.
Approach
Decompose erasure into amodal entity segmentation followed by entity-restricted completion: entity attention masks cross-attention so each inpainting region only draws content from the same predicted entity in the non-inpainting area, plus image-side classifier-free guidance with an all-zero condition image.
Key results
- On the EntitySeg test set, EntityErasure reduces MSN to 0.122 (vs 0.554 for PowerPaint and 0.679 for CLIPAway) and MARS to 0.037 with ReMOVE 86.24.
- On COCO, MSN drops to 0.506 versus 2.654 for PowerPaint, with FID 22.53 vs 25.68.
- Ablation shows entity attention reduces MSN from 0.212 to 0.122 relative to standard cross attention.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use entity attention guided by amodal entity segmentation instead of standard cross attention (ablation) | EntityErasure · EntitySeg test | MSN: 0.212 -> 0.122 | Helped |
| use entity attention guided by amodal entity segmentation instead of standard cross attention (ablation) | EntityErasure · EntitySeg test | MARS: 0.077 -> 0.037 | Helped |
| use entity attention guided by amodal entity segmentation instead of standard cross attention (ablation) | EntityErasure · EntitySeg test | FID: 57.1 -> 58.0 | Hurt |
| use entity attention guided by amodal entity segmentation instead of standard cross attention (ablation) | EntityErasure · EntitySeg test | ReMOVE: 85.82 -> 86.24 | Helped |
| increase Image-CFG weight from 1 to 3 (ablation) | EntityErasure · EntitySeg test | MSN: 0.449 -> 0.122 | Helped |
| increase Image-CFG weight from 1 to 3 (ablation) | EntityErasure · EntitySeg test | MARS: 0.0783 -> 0.0369 | Helped |
| increase Image-CFG weight from 1 to 3 (ablation) | EntityErasure · EntitySeg test | ReMOVE: 84.86 -> 86.24 | Helped |
| increase Text-CFG weight from 1 to 3 while Image-CFG is 1 (ablation) | EntityErasure · EntitySeg test | MSN: 0.449 -> 0.425 | No clear effect |
| apply entity attention to the non-inpainting area (ablation) | EntityErasure · object erasure images | non-inpainting region preservation | Hurt |
| use EntityErasure rather than MAT | object erasure · EntitySeg test | MSN: 0.433 -> 0.122 | Helped |
| use EntityErasure rather than MAT | object erasure · COCO val2017 | MSN: 1.876 -> 0.506 | Helped |
| use EntityErasure rather than MAT | object erasure · EntitySeg test | ReMOVE: 86.67 -> 86.24 | Hurt |
Surprising findings
- Applying entity attention to non-inpainting regions causes unwanted changes there, so only element-wise feature addition is used outside the mask.
- Larger sampling steps (50) did not yield clear improvement over 20 steps.
Limitations
- Performance depends on the quality of amodal entity segmentation.
- Relies on manual mask extension to remove shadows of erased objects.
- Metrics rely on an external open-world segmentation model (EntitySeg) to detect sundries.
Experimental setup
Datasets: Open Images V7 EntitySeg test dataset COCO val2017
Benchmarks: EntitySeg test dataset COCO val2017
Metrics: MSN MARS ReMOVE FID inference time
Baselines: MAT SD-Inpainting CLIPAway PowerPaint
Builds on: BrushNet ODISE Mask2Former Freestyle IP-Adapter
Closest papers by meaning
- SOHES: Self-supervised Open-world Hierarchical Entity Segmentation (ICLR 2024 · similarity 0.83)
- EntitySAM: Segment Everything in Video (CVPR 2025 · similarity 0.82)
- Open-World Amodal Appearance Completion (CVPR 2025 · similarity 0.82)
- MACE: Mass Concept Erasure in Diffusion Models (CVPR 2024 · similarity 0.81)
- Entity-NeRF: Detecting and Removing Moving Entities in Urban Scenes (CVPR 2024 · similarity 0.81)
- Paint by Inpaint: Learning to Add Image Objects by Removing Them First (CVPR 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.