Research map / Architectures and efficiency / RNNs and normalization
OPUS: Occupancy Prediction Using a Sparse Set
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Frames occupancy prediction as direct set prediction with learnable sparse queries, trained end-to-end with a Chamfer-distance loss and nearest-neighbor class assignment, reaching state-of-the-art RayIoU on Occ3D-nuScenes.
Problem
Occupancy prediction treats dense voxel grids as classification, wasting computation on the vast majority of empty voxels; sparse alternatives need multi-stage filtering or explicit space modeling, and Hungarian matching is infeasible for ~10K-70K voxels.
Approach
Decouple set-to-set matching into two parallel steps: Chamfer distance aligns predicted point locations with ground-truth occupied voxels, and each predicted point takes the semantic class of its nearest ground-truth voxel. Combined with a transformer decoder over sparse queries, coarse-to-fine point counts, consistent point sampling, and adaptive loss re-weighting.
Key results
- OPUS-L (8f) reaches RayIoU 41.2 on Occ3D-nuScenes, surpassing the previous best (SparseOcc 16f, 35.1) by 6.1.
- OPUS-T (8f) achieves RayIoU 38.4 at 22.4 FPS, outperforming FB-Occ (16f, 33.5) and SparseOcc (16f, 35.1) while running ~1.3x faster than SparseOcc (8f).
- Chamfer-based matching for 10K points takes 1.25 ms versus 24,216 ms for the Hungarian algorithm (2,304 MB GPU memory).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use OPUS-T instead of SparseOcc (8f) | Occ3D-nuScenes occupancy prediction · Occ3D-nuScenes | RayIoU: 34.0 -> 38.4 | Helped |
| increase model size from OPUS-T to OPUS-L | OPUS · Occ3D-nuScenes | RayIoU: 38.4 -> 41.2 | Helped |
| replace the original Chamfer distance with CDR (ablation) | OPUS-M · Occ3D-nuScenes ablation | RayIoU: 29.2 -> 30.1 | Helped |
| add FocalLossR (ablation) | OPUS-M with CDR · Occ3D-nuScenes ablation | RayIoU: 30.1 -> 31.5 | Helped |
| add CPS using the σp term (ablation) | OPUS-M with CDR and FocalLossR · Occ3D-nuScenes ablation | RayIoU: 31.5 -> 32.3 | Helped |
| add coarse-to-fine query prediction (ablation) | OPUS-M with CDR, FocalLossR, and CPS · Occ3D-nuScenes ablation | RayIoU: 32.3 -> 32.6 | Helped |
| replace grid initialization with random 3D initialization (ablation) | OPUS · Occ3D-nuScenes initial-point ablation | RayIoU: 28.3 -> 29.9 | Helped |
| add CDR supervision to randomly initialized P0 (ablation) | OPUS · Occ3D-nuScenes initial-point ablation | RayIoU: 29.9 -> 30.1 | Helped |
| increase the last-layer point count from 8 to 16 (ablation) | OPUS-M · Occ3D-nuScenes point-count ablation | RayIoU: 33.9 -> 34.5 | Helped |
| increase the last-layer point count from 16 to 32 (ablation) | OPUS-M · Occ3D-nuScenes point-count ablation | RayIoU: 34.5 -> 32.6 | Hurt |
| increase the last-layer point count from 32 to 64 (ablation) | OPUS-M · Occ3D-nuScenes point-count ablation | RayIoU: 32.6 -> 28.8 | Hurt |
Surprising findings
- Hungarian matching on 10K points takes about 24 seconds, making it infeasible for occupancy-scale point sets, while the Chamfer-based assignment takes milliseconds.
- Increasing the number of predicted points per query beyond 16 lowers RayIoU even though mIoU keeps rising, since extra points add noise along rays.
Limitations
- Slow convergence compared with dense methods, the authors note, which may be addressed by DETR follow-up techniques.
- Lower mIoU than dense methods despite higher RayIoU; sparse approaches struggle with mIoU.
- Evaluated only on vision-only datasets; multi-modal extension left as future work.
Experimental setup
Datasets: Occ3D-nuScenes Waymo-Occ3D
Benchmarks: Occ3D-nuScenes Waymo-Occ3D
Metrics: RayIoU mIoU FPS RayIoU1m RayIoU2m RayIoU4m
Baselines: SparseOcc FB-Occ BEVFormer BEVDet-Occ RenderOcc PETRv2 TPVFormer CTF-Occ
Builds on: DETR SparseBEV SparseOcc PETRv2
Closest papers by meaning
- OctOcc: High-Resolution 3D Occupancy Prediction with Octree (AAAI 2024 · similarity 0.90)
- ODG: Occupancy Prediction Using Dual Gaussians (NeurIPS 2025 · similarity 0.90)
- Achieving Speed-Accuracy Balance in Vision-based 3D Occupancy Prediction via Geometric-Semantic Disentanglement (AAAI 2025 · similarity 0.90)
- OccMamba: Semantic Occupancy Prediction with State Space Models (CVPR 2025 · similarity 0.89)
- SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Prediction (CVPR 2024 · similarity 0.89)
- Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillation (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.