Research map / Video / Video segmentation and tracking
Exploiting Continuous Motion Clues for Vision-Based Occupancy Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CMOP, a lightweight plug-and-play module that continuously updates vision-based 3D occupancy predictions using a Propagator (historical occupancy forecasting), a Tracker (optical-flow-based per-step updates), and a Fuser.
Problem
Frame-by-frame occupancy networks ignore past occupancy states and continuous visual changes, so occlusions cause missing voxels and reduced temporal accuracy in dynamic driving scenes.
Approach
Treat occupancy prediction as continuous updating: forecast future occupancy from historical occupancy results (Propagator), refine it each step using explicit optical-flow motion clues conditioned on ego 6-DOF transforms (Tracker), and adaptively fuse both (Fuser), periodically refreshed by a base occupancy network.
Key results
- Adding CMOP to SurroundOcc raises IoU from 31.49 to 40.94 and mIoU from 20.30 to 22.71 at Ninfer=1, with about 0.03 s extra inference time.
- With Ninfer=6, SurroundOcc+CMOP reaches averaged IoU 31.56 and F1 46.71, versus 21.10 and 34.46 for the periodic SurroundOcc baseline (M0).
- On a Jetson AGX Orin, SurroundOcc takes 0.912 s while CMOP takes 0.068 s, enabling over 13 continuous updating steps per inference window.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CMOP to SurroundOcc | SurroundOcc · nuScenes | IoU: 31.49 -> 40.94 | Helped |
| Add CMOP to SurroundOcc | SurroundOcc · nuScenes | mIoU: +2.41 | Helped |
| Add CMOP to BEVDet4D | BEVDet4D · nuScenes | IoU: +7.91 | Helped |
| Add CMOP to BEVDet4D | BEVDet4D · nuScenes | mIoU: +0.10 | Helped |
| Add CMOP to SurroundOcc | SurroundOcc · nuScenes, Ninfer = 3 and 6 | IoU: +10.29; +10.46 | Helped |
| Add CMOP to SurroundOcc | SurroundOcc · nuScenes, Ninfer = 3 and 6 | F1: +11.02; +12.25 | Helped |
| Add Propagator, Tracker, and Fuser to the periodically invoked OccInfer baseline (ablation) | SurroundOcc, Ninfer = 6 · nuScenes, Ninfer = 6 | IoU: 21.10 -> 31.56 | Helped |
| Add Propagator, Tracker, and Fuser to the periodically invoked OccInfer baseline (ablation) | SurroundOcc, Ninfer = 6 · nuScenes, Ninfer = 6 | F1: 34.46 -> 46.71 | Helped |
| Add Propagator without Tracker or Fuser (ablation) | SurroundOcc, Ninfer = 6 · nuScenes, Ninfer = 6 | IoU: 21.10 -> 25.56 | Helped |
| Add Tracker without Propagator or Fuser (ablation) | SurroundOcc, Ninfer = 6 · nuScenes, Ninfer = 6 | IoU: 21.10 -> 26.38 | Helped |
| Add periodic OccInfer to prevent error accumulation during prolonged updating (ablation) | CMOP · nuScenes, Ninfer = 6 | IoU: 29.41 -> 31.56 | Helped |
| Use optical flow instead of intermediate frames as Tracker visual input (ablation) | CMOP · nuScenes, Ninfer = 6 | IoU: 21.49 -> 31.56 | Helped |
Surprising findings
- Plain RGB intermediate frames as visual input performed worst (IoU 21.49), below frame difference and HOG, suggesting raw frames are too cluttered for the compact encoder.
- Removing the periodic OccInfer refresh (feeding Fuser outputs back into Propagator) lowered IoU from 31.56 to 29.41, indicating error accumulation over long updating.
Limitations
- Relies on implicit correlation between occupied voxels and optical flow; instance-level motion is not modeled, leading to blank holes within object instances.
- Noisy voxels can appear after updating (e.g., on the right lane in the case study), so further optimization is needed.
Experimental setup
Datasets: nuScenes nuScenes dense occupancy GT (SurroundOcc)
Benchmarks: nuScenes occupancy (dense occupancy GT)
Metrics: IoU mIoU averaged IoU averaged F1 inference time
Baselines: MonoScene Atlas TPVFormer BEVFormer RenderOcc SurroundOcc IMVT3D Cam4DOcc BEVDet4D
Builds on: SurroundOcc BEVDet4D MonoScene Cam4DOcc
Closest papers by meaning
- Rethinking Temporal Fusion with a Unified Gradient Descent View for 3D Semantic Occupancy Prediction (CVPR 2025 · similarity 0.89)
- Spatiotemporal Decoupling for Efficient Vision-Based Occupancy Forecasting (CVPR 2025 · similarity 0.89)
- OccMamba: Semantic Occupancy Prediction with State Space Models (CVPR 2025 · similarity 0.88)
- AdaOcc: Adaptive 3D Occupancy Prediction for Embodied Tasks (NeurIPS 2026 · similarity 0.88)
- Achieving Speed-Accuracy Balance in Vision-based 3D Occupancy Prediction via Geometric-Semantic Disentanglement (AAAI 2025 · similarity 0.88)
- OPUS: Occupancy Prediction Using a Sparse Set (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.