Research map / 3D vision / 3D generation
Spatiotemporal Decoupling for Efficient Vision-Based Occupancy Forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EfficientOCF, a vision-based occupancy forecasting network that decouples 3D occupancy into BEV occupancy plus height values and refines forecasts via flow-based instance association, plus a new C-IoU metric.
Problem
Vision-based 3D occupancy forecasting of movable objects is slow and produces poor shape details because dense 3D representations suffer from spatial bias toward empty voxels and temporal bias toward static voxels.
Approach
Spatially decouple 3D occupancy into a 2D BEV occupancy map plus per-column height values, and temporally decouple by propagating current instance segmentation into future frames using predicted backward centripetal flow, then lift the refined 2D forecast to 3D.
Key results
- EfficientOCF reaches 39.93 2D IoUc and 47.53 3D C-IoUc on nuScenes/nuScenes-Occupancy, outperforming OCFNet (35.50 and 39.71).
- Inference time of 82.33 ms (12.15 Hz) on a single GPU versus 124.38 ms for OCFNet.
- VPQbb improves from 18.53 (OCFNet) to 19.95 on nuScenes and from 28.35 to 36.75 on Lyft-Level5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use spatiotemporally decoupled EfficientOCF instead of OCFNet | 3D occupancy forecasting · nuScenes | 3D IoUf: 29.36 -> 32.73 | Helped |
| use spatiotemporally decoupled EfficientOCF instead of OCFNet | 3D occupancy forecasting · Lyft-Level5 | 3D IoUf: 38.14 -> 41.90 | Helped |
| use spatiotemporally decoupled EfficientOCF instead of OpenOccupancy | 3D occupancy forecasting · nuScenes & nuScenes-Occupancy | 3D IoUf: 13.06 -> 19.02 | Helped |
| use spatiotemporally decoupled EfficientOCF instead of OccFormer | 3D occupancy forecasting · nuScenes & nuScenes-Occupancy | 3D C-IoUf: 35.98 -> 45.57 | Helped |
| use EfficientOCF instead of OCFNet | 3D instance forecasting · nuScenes | VPQfg: 6.61 -> 10.69 | Helped |
| use EfficientOCF instead of OCFNet | 3D instance forecasting · Lyft-Level5 | VPQbb: 28.35 -> 36.75 | Helped |
| use EfficientOCF instead of OCFNet | 3D occupancy forecasting · inference | inference time (ms): 124.38 -> 82.33 | Helped |
| add instance-aware temporal refinement using flow-based association (ablation) | EfficientOCF · nuScenes & nuScenes-Occupancy | 3D IoUf: 17.31 -> 19.02 | Helped |
| add instance-aware temporal refinement using flow-based association (ablation) | EfficientOCF · nuScenes & nuScenes-Occupancy | 3D C-IoUf: 42.90 -> 45.57 | Helped |
| use adaptive dual pooling instead of average pooling (ablation) | EfficientOCF · nuScenes & nuScenes-Occupancy | 3D IoUf: 16.77 -> 19.02 | Helped |
| use adaptive dual pooling instead of max pooling (ablation) | EfficientOCF · nuScenes & nuScenes-Occupancy | 3D IoUf: 16.90 -> 19.02 | Helped |
| add the height head (ablation) | EfficientOCF segmentation-only variant · nuScenes & nuScenes-Occupancy | 3D IoUf: 16.89 -> 19.02 | Helped |
Surprising findings
- EfficientOCF trained on 1/6 of the training sequences outperforms OCFNet trained on the full set on all 3D fine-grained metrics.
Limitations
- Forecasting restricted to general movable objects; static objects are not forecast.
- Ground-truth fine-grained occupancy labels in nuScenes-Occupancy are incomplete due to LiDAR sparsity, motivating the proposed C-IoU metric.
Experimental setup
Datasets: nuScenes nuScenes-Occupancy Lyft-Level5
Benchmarks: nuScenes nuScenes-Occupancy Lyft-Level5
Metrics: IoU C-IoU VPQ inference time
Baselines: PowerBEV OpenOccupancy OccFormer OCFNet
Builds on: Lift-Splat-Shoot PowerBEV OCFNet Cam4DOcc
Closest papers by meaning
- OccProphet: Pushing the Efficiency Frontier of Camera-Only 4D Occupancy Forecasting with an Observer-Forecaster-Refiner Framework (ICLR 2025 · similarity 0.91)
- Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications (CVPR 2024 · similarity 0.91)
- ODG: Occupancy Prediction Using Dual Gaussians (NeurIPS 2025 · similarity 0.90)
- UnO: Unsupervised Occupancy Fields for Perception and Forecasting (CVPR 2024 · similarity 0.90)
- SDGOCC: Semantic and Depth-Guided Bird's-Eye View Transformation for 3D Multimodal Occupancy Prediction (CVPR 2025 · similarity 0.89)
- Achieving Speed-Accuracy Balance in Vision-based 3D Occupancy Prediction via Geometric-Semantic Disentanglement (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.