Research map / Detection and segmentation / 3D detection and driving
UnO: Unsupervised Occupancy Fields for Perception and Forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UNO, an unsupervised 4D implicit occupancy world model trained on LiDAR-derived occupancy pseudo-labels, transferred to point cloud forecasting and BEV semantic occupancy forecasting.
Problem
Learning a self-driving world model that perceives and forecasts the environment without costly human annotations, since labeled object data is scarce and limited to predefined classes.
Approach
Self-supervise a continuous 4D occupancy field from future LiDAR rays (free space along rays and a buffer behind returns) with a balanced binary cross-entropy objective and an implicit BEV-based decoder, then transfer via a lightweight learned ray renderer or BEV fine-tuning.
Key results
- On point cloud forecasting, UNO reduces NFCD on Argoverse 2 to 0.71 m^2 from 2.20 for 4D-OCC and ranks first on the public Argoverse 2 leaderboard.
- Fine-tuned UNO outperforms MP3 and ImplicitO on BEV semantic occupancy across all label scales, and with roughly 10x less labeled data beats baselines trained on the full set.
- In class-wise unsupervised occupancy recall on Argoverse 2 Sensor, UNO achieves significantly higher recall than 4D-OCC for almost all classes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use continuous occupancy pre-training with balanced positive and negative query samples instead of depth rendering (ablation) | UNO · Argoverse 2 Sensor | NFCD (m2): 1.79 -> 0.83 | Helped |
| Use continuous occupancy pre-training with balanced positive and negative query samples instead of depth rendering (ablation) | UNO · Argoverse 2 Sensor | mAP: 20.7 -> 52.3 | Helped |
| Use continuous occupancy pre-training with balanced positive and negative query samples instead of free-space rendering (ablation) | UNO · Argoverse 2 Sensor | NFCD (m2): 1.16 -> 0.83 | Helped |
| Use continuous occupancy pre-training with balanced positive and negative query samples instead of free-space rendering (ablation) | UNO · Argoverse 2 Sensor | mAP: 39.6 -> 52.3 | Helped |
| Balance positive and negative occupancy query samples rather than sampling uniformly along rays (ablation) | UNO · Argoverse 2 Sensor | NFCD (m2): 0.84 -> 0.83 | Helped |
| Balance positive and negative occupancy query samples rather than sampling uniformly along rays (ablation) | UNO · Argoverse 2 Sensor | mAP: 43.6 -> 52.3 | Helped |
| Balance positive and negative occupancy query samples rather than sampling uniformly along rays (ablation) | UNO · Argoverse 2 Sensor | Soft-IoU: 17.0 -> 22.3 | Helped |
| Transfer UNO to point cloud forecasting with a learned renderer | UNO · Argoverse 2 LiDAR | NFCD (m2): 2.20 -> 0.71 | Helped |
| Transfer UNO to point cloud forecasting with a learned renderer | UNO · nuScenes | CD (m2): 3.47 -> 1.80 | Helped |
| Transfer UNO to point cloud forecasting with a learned renderer | UNO · KITTI | L1 (m): 1.45 -> 1.09 | Helped |
Surprising findings
- Point cloud forecasting metrics are dominated by static background, so UNO and the unbalanced variant score similarly there despite large differences in BEV semantic occupancy transfer.
- UNO implicitly learns the map layout even though map data is neither an input nor a supervision signal.
Limitations
- Relies on calibrated and localized vehicle with known sensor intrinsics and extrinsics for pseudo-label construction.
- Occupancy supervision is partial due to occlusion and non-reflective materials.
Experimental setup
Datasets: Argoverse 2 LiDAR Argoverse 2 Sensor nuScenes KITTI Odometry
Benchmarks: Argoverse 2 LiDAR nuScenes KITTI Odometry Argoverse 2 Sensor
Metrics: Chamfer distance NFCD AbsRel L1 depth error mAP Soft-IoU occupancy recall at 70% precision
Baselines: 4D-OCC RayTracing SPFNet S2Net ST3DCNN MP3 ImplicitO FREE-SPACE-RENDERING DEPTH-RENDERING
Builds on: ImplicitO 4D-OCC MP3 deformable attention NeRF-style depth rendering
Closest papers by meaning
- Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving (ICLR 2025 · similarity 0.92)
- Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving (AAAI 2025 · similarity 0.91)
- Spatiotemporal Decoupling for Efficient Vision-Based Occupancy Forecasting (CVPR 2025 · similarity 0.90)
- Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications (CVPR 2024 · similarity 0.89)
- Achieving Speed-Accuracy Balance in Vision-based 3D Occupancy Prediction via Geometric-Semantic Disentanglement (AAAI 2025 · similarity 0.89)
- SDGOCC: Semantic and Depth-Guided Bird's-Eye View Transformation for 3D Multimodal Occupancy Prediction (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.