Research map / Detection and segmentation / 3D detection and driving
Towards Flexible 3D Perception: Object-Centric Occupancy Completion Augments 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces object-centric occupancy as a complement to 3D bboxes, builds an automated object-occupancy dataset from Waymo Open Dataset, and proposes a sequence-based implicit occupancy completion network that also refines detections.
Problem
Object bounding boxes miss fine-grained object geometry, while scene-level occupancy at high resolution is computationally infeasible for large driving scenes; noisy detections and tracks also make naive multi-frame object point aggregation blurry.
Approach
Defines occupancy per object in its local box coordinate system, and uses a causal transformer over dual local/global RoI latents with an implicit MLP decoder that predicts occupancy probability at arbitrary query positions, so the completed shape is then used to refine detection.
Key results
- On FSD tracklets, the method reaches 62.84% occupancy IoU versus 44.28% for the voxelized-accumulation baseline.
- Applied to 1-frame CenterPoint, the method improves vehicle L1 3D AP by +8.6 (73.2 to 81.8), versus +3.2 for MoDAR.
- With 7-frame FSD, the method reaches 83.3 L1 AP and 75.7 L2 APH, reported as new online state of the art on WOD validation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace history-point accumulation with learned sequence-based occupancy completion | Shape completion on WOD · GT, noisy GT, FSD, and CenterPoint tracklets | IoU: 61.35 -> 69.15; 50.39 -> 64.92; 44.28 -> 62.84; 40.45 -> 57.99 | Helped |
| add object-centric occupancy completion | CenterPoint · WOD val, Vehicle L1 3D | AP: 73.2 -> 81.8 (+8.6) | Helped |
| add object-centric occupancy completion | FSD with [-6, 0] input frames · WOD val, Vehicle L1 3D | AP: 80.9 -> 83.3 (+2.4) | Helped |
| add object-centric occupancy completion without retraining on FSDv2 tracklets | FSDv2 · WOD val, Vehicle L1 3D | AP: 79.8 -> 83.2 (+3.4) | Helped |
| query the implicit decoder at GT-box voxel centers outside the RoI | Ours shape completion · WOD val, GT track + noise, FSD track, and CP track | IoU: 64.92 -> 69.30; 62.84 -> 68.38; 57.99 -> 65.80 | Helped |
| add a global-coordinate RoI encoder alongside the local encoder (ablation) | Ours dual-branch model · WOD val, FSD track | IoU: 62.13 -> 62.84 | Helped |
| use implicit rather than explicit occupancy predictions for detection refinement (ablation) | Ours · WOD val, FSD track | AP L1: 80.20 -> 82.80 | Helped |
| add the occupancy decoder and occupancy completion task (ablation) | Ours detection model · WOD val, FSD track | AP L1: 81.10 -> 82.80 | Helped |
| train on 32-frame rather than 8-frame tracklets (ablation) | Ours · WOD val, FSD track | IoU: 62.43 -> 62.84 | Helped |
| train on 32-frame rather than 16-frame tracklets (ablation) | Ours · WOD val, FSD track | IoU: 62.61 -> 62.84 | Helped |
| allow attention to future timestamps for offline inference (ablation) | Ours · WOD val, FSD track | IoU: 62.84 -> 63.03 | Helped |
| limit inference to frames [-63, 0] rather than all history frames (ablation) | Ours · WOD val, FSD track | IoU: 62.84 -> 62.80 | No clear effect |
Surprising findings
- Explicit occupancy used for detection performs worse than the no-occupancy-decoder baseline (80.20 vs 81.10 L1 AP), because sampling occupied voxels is non-differentiable.
- Using the trained model on FSDv2 tracklets without retraining gives slightly lower shape completion IoU than on FSD tracklets, despite FSDv2 having better detection.
Limitations
- Automatic occupancy annotation assumes rigid bodies, so experiments are restricted to vehicles.
- The 0.2 m voxel size gives low-resolution objects (e.g., a sedan about 23x9x7 voxels), limiting high-quality rendering.
- Shape completion is evaluated only on the WOD validation set with the authors' own pseudo ground truth, which excludes unobserved voxels from IoU.
Experimental setup
Datasets: Waymo Open Dataset
Benchmarks: Waymo Open Dataset validation set
Metrics: IoU mIoU (track) mIoU (box) AP APH mAP by range
Baselines: Baseline voxelized point accumulation CenterPoint FSD FSDv2 SWFormer MoDAR MPPNet CenterFormer 3D-MAN VoxelNeXt HEDNet FSD++
Builds on: FSD DeepSDF PointNet LiDAR R-CNN MoDAR
Closest papers by meaning
- CSV-Occ: Fusing Multi-frame Alignment for Occupancy Prediction with Temporal Cross State Space Model and Central Voting Mechanism (ICML 2025 · similarity 0.89)
- OctOcc: High-Resolution 3D Occupancy Prediction with Octree (AAAI 2024 · similarity 0.88)
- COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Prediction (CVPR 2024 · similarity 0.88)
- ODG: Occupancy Prediction Using Dual Gaussians (NeurIPS 2025 · similarity 0.88)
- CorrBEV: Multi-View 3D Object Detection by Correlation Learning with Multi-modal Prototypes (CVPR 2025 · similarity 0.88)
- Achieving Speed-Accuracy Balance in Vision-based 3D Occupancy Prediction via Geometric-Semantic Disentanglement (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.