Research map / Image generation / Fast diffusion sampling
Amodal Completion via Progressive Mixed Context Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Training-free amodal completion that progressively inpaints occluders with a pre-trained Stable Diffusion v2 inpainting model, using mixed context diffusion sampling to avoid co-occurrence bias and a counterfactual outpainting rule to judge completeness.
Problem
Amodal completion (recovering hidden object pixels, including beyond image boundaries) is hard for generative models; off-the-shelf diffusion inpainting tends to regenerate co-occurring occluders instead of completing the object, and there is no reliable way to judge completion success or ground-tru
Approach
Instead of predicting amodal masks, expand to the object's bounding box context, remove occluders, and run diffusion inpainting progressively. Mixed context diffusion sampling swaps the background to a clean gray backdrop to break co-occurrence bias, extracts an intermediate object mask via clustering of UNet decoder features, and composites it onto an object-removed background. A training-free co
Key results
- Ours achieves the best CLIP/DreamSim/LPIPS on easy and hard cases (e.g., hard-case LPIPS 0.110 vs LaMa 0.130 on COCO) and wins 90.9% of user preferences on easy cases.
- Mixed context sampling raises successful completions on hard co-occurrence cases from 72% (w/o MC) to 90%, and user preference from 28% to 54%.
- Counterfactual curation rule reaches 0.70 accuracy on a 100-object test set versus 0.83 for human consensus.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Mixed Context Diffusion Sampling (ablation) | Progressive Occlusion-aware Completion · Natural-image occlusion, hard cases | Successful completions: 72% -> 90% | Helped |
| Remove Mixed Context Diffusion Sampling (ablation) | Progressive Occlusion-aware Completion · Natural-image occlusion, easy cases | Successful completions: 88% -> 90% | No clear effect |
| Use Mixed Context Diffusion Sampling (ablation) | Progressive Occlusion-aware Completion · Natural-image occlusion, hard cases | User preference: 28% -> 54% | Helped |
| Use a gray background when swapping the original context (ablation) | Mixed Context Diffusion Sampling · Amodal completion on 55 images | Successful completions: +20% | Helped |
| Use a forest background instead of the original context (ablation) | Mixed Context Diffusion Sampling · Amodal completion on 55 images | Successful completions: -16% | Hurt |
| Use a sky background instead of the original context (ablation) | Mixed Context Diffusion Sampling · Amodal completion on 55 images | Successful completions: -9% | Hurt |
| Composite at DDIM timestep 20 out of 50 (ablation) | Mixed Context Diffusion Sampling · Difficult co-occurrence cases | Successful completions | Helped |
| Use third UNet decoder layer features for clustering (ablation) | Mixed Context Diffusion Sampling · Amodal completion | Amodal mask quality | Helped |
| Use the counterfactual completion rule rather than random chance | Counterfactual Completion Curation System · Complete/incomplete object classification test set | Accuracy: 0.50 -> 0.70 | Helped |
Surprising findings
- LaMa scores similarly to the proposed method on automatic metrics despite not being designed for amodal completion, because it produces similarly colored blurry fills.
- Swapping to a forest or sky background hurts completion while a gray backdrop improves it by 20%.
Limitations
- Overextends small objects enclosed by larger occluders.
- Shadows and certain human poses can induce unintended occluders.
- Curation rule accuracy (0.70) lags human consensus (0.83).
Experimental setup
Datasets: COCO Open Images
Benchmarks: COCO pseudo-occlusion benchmark (3,000 objects) Open Images pseudo-occlusion subset
Metrics: CLIP similarity DreamSim LPIPS user preference successful completion rate accuracy/precision/recall of curation rule
Baselines: SSSD LaMa Inst-Inpaint Naive Outpainting
Builds on: Stable Diffusion v2 inpainting DDIM Grounded segmentation LaMa
Closest papers by meaning
- Using Diffusion Priors for Video Amodal Segmentation (CVPR 2025 · similarity 0.81)
- Erase Diffusion: Empowering Object Removal Through Calibrating Diffusion Pathways (CVPR 2025 · similarity 0.80)
- Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spaces (ICML 2025 · similarity 0.80)
- Scaling Properties of Diffusion Models For Perceptual Tasks (CVPR 2025 · similarity 0.80)
- Residual Learning in Diffusion Models (CVPR 2024 · similarity 0.79)
- MVDream: Multi-view Diffusion for 3D Generation (ICLR 2024 · similarity 0.79)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.