Research map / 3D vision / 3D generation
OccProphet: Pushing the Efficiency Frontier of Camera-Only 4D Occupancy Forecasting with an Observer-Forecaster-Refiner Framework
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OccProphet, a lightweight camera-only 4D occupancy forecasting framework (Observer-Forecaster-Refiner) that cuts compute and raises forecasting IoU relative to Cam4DOcc.
Problem
Camera-only 3D occupancy forecasting (predicting future occupied voxels from surround-view images) is computationally expensive for training and inference, which hinders deployment on edge autonomous-driving hardware.
Approach
An Observer that aggregates 4D voxel features via downsampling-based Efficient 4D Aggregation and Tripling-Attention Fusion (scene, height, BEV branches with temporal attention), a Forecaster that generates scene-conditioned adaptive weights to predict future frames, and a Refiner that reuses E4A to refine forecasts with cross-frame interactions.
Key results
- On nuScenes, OccProphet reaches 29.15 averaged future IoU vs 27.98 for Cam4DOcc (4% relative gain) on inflated GMO forecasting.
- Versus Cam4DOcc, OccProphet uses 82M params vs 370M, 24G training memory vs 57G, 1985G FLOPs vs 6434G, and runs at 4.5 FPS vs 1.7 FPS (2.6x speedup).
- On nuScenes-Occupancy fine-grained GMO, averaged future IoU is 11.98 vs 10.10 for Cam4DOcc.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace BEV representation with E4A (ablation) | Observer · nuScenes-Occupancy | tilde IoU_f (%): 26.50 -> 27.50 | Helped |
| replace TPV representation with E4A (ablation) | Observer · nuScenes-Occupancy | tilde IoU_f (%): 26.96 -> 27.50 | Helped |
| add the Observer (ablation) | OccProphet · nuScenes, inflated GMO | tilde IoU_f: 27.25 -> 28.24 | Helped |
| add the Forecaster (ablation) | OccProphet · nuScenes, inflated GMO | tilde IoU_f: 27.50 -> 28.24 | Helped |
| add the Refiner (ablation) | OccProphet · nuScenes, inflated GMO | tilde IoU_f: 27.44 -> 28.24 | Helped |
| add Observer, Forecaster, and Refiner instead of using only a linear layer (ablation) | OccProphet · nuScenes, inflated GMO | tilde IoU_f: 26.07 -> 28.24 | Helped |
| use OccProphet instead of OCFNet (Cam4DOcc) | camera-only occupancy forecasting · nuScenes, inflated GMO | tilde IoU_f: 27.98 -> 29.15 | Helped |
| use OccProphet instead of OCFNet (Cam4DOcc) | camera-only occupancy forecasting · Lyft-Level5, inflated GMO | tilde IoU_f: 34.60 -> 40.11 | Helped |
Surprising findings
- Cross-domain training (Lyft-Level5 to nuScenes) performs notably worse than intra-domain forecasting, attributed to sensor setting differences.
Limitations
- Performance drops notably in cross-domain evaluation (Lyft-Level5 to nuScenes).
- Occluded and dense scenarios remain challenging; fine-grained forecasting is still difficult.
- Evaluated only on camera-only forecasting with surround-view RGB; no on-vehicle deployment test is reported.
Experimental setup
Datasets: nuScenes nuScenes-Occupancy Lyft-Level5
Benchmarks: nuScenes Lyft-Level5 nuScenes-Occupancy
Metrics: IoU (%) current IoU (%) future 2s averaged future IoU FLOPs FPS parameters GPU memory
Baselines: Cam4DOcc OpenOccupancy-C SPC PowerBEV-3D BEVDet4D
Builds on: Cam4DOcc ResNet Swin-style window attention
Closest papers by meaning
- Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications (CVPR 2024 · similarity 0.92)
- Spatiotemporal Decoupling for Efficient Vision-Based Occupancy Forecasting (CVPR 2025 · similarity 0.91)
- Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving (AAAI 2025 · similarity 0.90)
- Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving (ICLR 2025 · similarity 0.89)
- UnO: Unsupervised Occupancy Fields for Perception and Forecasting (CVPR 2024 · similarity 0.89)
- COME: Adding Scene-Centric Forecasting Control to Occupancy World Model (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.