Research map / Detection and segmentation / Open-set and few-shot detection
Amodal Scene Analysis via Holistic Occlusion Relation Inference and Generative Mask Completion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HORI, a single-pass transformer that predicts a holistic occlusion relation matrix (handling mutual occlusion), and GMC, a latent-diffusion mask completion module, plus the ASW amodal dataset.
Problem
Amodal scene analysis: inferring occlusion ordering among objects (including mutual occlusion) efficiently and completing invisible object masks with plausible, high-fidelity shapes.
Approach
Replace pairwise de-occlusion ordering with a single-pass Occlusion Reasoning Block that uses masked cross-attention over visible instance masks and reads an occlusion relation matrix from self-attention maps, with two output channels for mutual occlusion; mask completion is cast as conditional latent diffusion on binary masks.
Key results
- HORI achieves 90.9% O-Acc on COCOA, versus 90.33% for ASBU, and 95.22% O-Acc on KINS versus 92.65% for ASBU.
- On the ASW dataset, HORI reaches 87.27% O-Acc, compared with 82.01% for ASBU and 80.93% for de-occlusion.
- HORI runs at about 5 FPS for occlusion reasoning on COCOA versus 1.68 FPS for ASBU on a single 3090Ti GPU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use holistic single-pass occlusion reasoning instead of ASBU pairwise reasoning | HORI · COCOA | O-Acc: 88.00 -> 90.90 | Helped |
| Use holistic single-pass occlusion reasoning instead of ASBU pairwise reasoning | HORI · KINS | O-Acc: 92.65 -> 95.22 | Helped |
| Use holistic single-pass occlusion reasoning instead of ASBU pairwise reasoning | HORI · ASW | O-Acc: 82.01 -> 87.27 | Helped |
| Use generative diffusion mask completion instead of ASBU | GMC · COCOA | FID: 8.816 -> 7.204 | Helped |
| Use generative diffusion mask completion instead of ASBU | GMC · ASW | FID: 4.438 -> 4.046 | Helped |
| Add visible instance masks to feature maps before amodal prediction (ablation) | HORI · COCOA | O-Acc: 85.6 -> 87.2 | Helped |
| Use predicted occlusion-relation matrix for relation inference (ablation) | HORI · COCOA | O-Acc: 87.2 -> 89.4 | Helped |
| Upsample inputs at inference (ablation) | HORI · COCOA | O-Acc: 89.4 -> 90.9 | Helped |
| Use masked attention to encode visible instance masks (ablation) | HORI · COCOA | O-Acc | Helped |
| Replace deterministic mask completion with GMC | amodal mask completion · COCOA | mIoU: ASBU: 84.22; GMC: 86.89; visual example: 82.4 vs 80.1 | Mixed |
Surprising findings
- HORI has the lowest FID/KID fidelity among methods on COCOA and ASW despite strong mIoU, so mIoU is argued to be unsuitable for non-deterministic amodal completion.
- On KINS, the HORI module's mIoU (93.79) is below the de-occlusion and ASBU baselines (about 94.8).
Limitations
- Assumes instance visible masks are given from an external instance or entity segmentation method.
- Mutual occlusion evaluation is limited to the newly introduced ASW dataset; the dataset evaluation split is 2,000 images.
- IoU does not reflect the plausibility of generated shapes, so the fidelity claims rely on FID/KID.
Experimental setup
Datasets: COCOA KINS ASW
Benchmarks: COCOA KINS ASW
Metrics: O-Acc mIoU FID KID
Baselines: CSDNet De-occlusion (Zhan et al. 2020) ASBU ASBU (retrained)
Builds on: Mask2Former Deformable DETR latent diffusion model ASBU
Closest papers by meaning
- Using Diffusion Priors for Video Amodal Segmentation (CVPR 2025 · similarity 0.89)
- Open-World Amodal Appearance Completion (CVPR 2025 · similarity 0.89)
- Amodal Ground Truth and Completion in the Wild (CVPR 2024 · similarity 0.89)
- Segment Anything, Even Occluded (CVPR 2025 · similarity 0.88)
- BLADE: Box-Level Supervised Amodal Segmentation through Directed Expansion (AAAI 2024 · similarity 0.87)
- Towards Efficient Foundation Model for Zero-shot Amodal Segmentation (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.