Research map / 3D vision / 3D generation
Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Cam4DOcc, a camera-only 4D occupancy forecasting benchmark built from nuScenes, nuScenes-Occupancy and Lyft-Level5 with four baselines, including the end-to-end OCFNet, and a standardized IoU-based evaluation protocol.
Problem
Camera-only occupancy estimation covers only the current (or past) 3D scene, and existing forecasting methods rely on LiDAR or are limited to 2D BEV, so there is no benchmark for camera-only 4D occupancy forecasting with semantics.
Approach
Reorganize nuScenes, nuScenes-Occupancy and Lyft-Level5 into sequences with sequential GMO/GSO occupancy labels and 3D backward centripetal flow, and propose OCFNet, an end-to-end spatiotemporal network that jointly forecasts future occupancy and flow from multi-view camera sequences.
Key results
- OCFNet beats PowerBEV-3D on inflated GMO forecasting on nuScenes by 12.4% in IoUf (23.89 vs 21.25) and reaches 26.82 with the full-data variant.
- On Lyft-Level5, OCFNet outperforms PowerBEV-3D by 20.8% in IoUf (29.56 vs 24.47).
- Removing the 3D flow prediction head reduces OCFNet's present and future IoU by about 4% (IoUc 26.84 vs 27.86).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OCFNet instead of PowerBEV-3D | camera-only 4D occupancy forecasting · nuScenes, inflated GMO | IoUf (2 s): 21.25 -> 23.89 | Helped |
| Train OCFNet on all training sequences rather than 1/6 of them | OCFNet · nuScenes, inflated GMO | IoUf (2 s): 23.89 -> 26.82 | Helped |
| Use OCFNet instead of PowerBEV-3D | camera-only 4D occupancy forecasting · Lyft-Level5, inflated GMO | IoUf (0.8 s): 24.47 -> 29.56 | Helped |
| Train OCFNet on all training sequences rather than 1/6 of them | OCFNet · nuScenes-Occupancy, fine-grained GMO | IoUf (2 s): 8.35 -> 9.68 | Helped |
| Use OCFNet instead of OpenOccupancy-C | 4D occupancy forecasting · nuScenes, inflated GMO, fine-grained GSO, and free space | IoUf (2 s), GMO: 12.67 -> 22.21 | Helped |
| Use OCFNet instead of OpenOccupancy-C | 4D occupancy forecasting · nuScenes, inflated GMO, fine-grained GSO, and free space | IoUf (2 s), GSO: 17.09 -> 17.14 | No clear effect |
| Train OCFNet on all training sequences rather than 1/6 of them | OCFNet · nuScenes, fine-grained GMO, fine-grained GSO, and free space | IoUf (2 s), GMO: 8.23 -> 9.20 | Helped |
| Remove the 3D flow prediction head (ablation) | OCFNet · nuScenes, inflated GMO | IoUf (2.0 s): 23.89 -> 22.99 | Hurt |
| Use fine-grained rather than inflated GMO annotations | occupancy forecasting baselines · nuScenes, GMO | IoUf (2 s): 21.25 -> 5.25 | Hurt |
Surprising findings
- The point-cloud-prediction voxelization baseline (SPC) fails to forecast future occupancy for inflated GMO, with near-zero IoU.
- Forecasting fine-grained GMO shape is much harder from camera images than inflated boxes, so inflated GMO labels are recommended for the benchmark.
Limitations
- Camera-only 4D forecasting remains difficult for longer horizons with many moving objects.
- Fine-grained GMO forecasting from camera images has low IoU, so the benchmark uses inflated GMO labels.
- The static-world, point-cloud voxelization and 2D-3D lifting baselines rely on strong hypotheses (static scene, flat ground, equal object heights).
Experimental setup
Datasets: nuScenes nuScenes-Occupancy Lyft-Level5
Benchmarks: Cam4DOcc (nuScenes, nuScenes-Occupancy, Lyft-Level5)
Metrics: IoU current (IoUc) IoU future (IoUf) time-weighted future IoU (~IoUf)
Baselines: OpenOccupancy-C SPC (SurroundDepth + PCPNet + Cylinder3D) PowerBEV-3D
Builds on: OpenOccupancy PowerBEV PCPNet SurroundDepth Cylinder3D
Closest papers by meaning
- OccProphet: Pushing the Efficiency Frontier of Camera-Only 4D Occupancy Forecasting with an Observer-Forecaster-Refiner Framework (ICLR 2025 · similarity 0.92)
- Spatiotemporal Decoupling for Efficient Vision-Based Occupancy Forecasting (CVPR 2025 · similarity 0.91)
- Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving (AAAI 2025 · similarity 0.90)
- GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Prediction (CVPR 2025 · similarity 0.90)
- Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving (ICLR 2025 · similarity 0.90)
- UnO: Unsupervised Occupancy Fields for Perception and Forecasting (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.