Research map / 3D vision / 3D generation
VisionPAD: A Vision-Centric Pre-training Paradigm for Autonomous Driving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VisionPAD, a self-supervised vision-only pre-training paradigm for vision-centric driving that uses 3D Gaussian Splatting decoding, self-supervised voxel velocity estimation, and multi-frame photometric consistency, without LiDAR depth supervision.
Problem
Pre-training for vision-centric autonomous driving perception relies on explicit LiDAR depth supervision or labeled data; image-only pre-training with volume rendering (UniPAD) gives little gain.
Approach
Replace volume rendering with anchor-based 3D Gaussian Splatting to reconstruct multi-view images from voxel features, learn motion by warping voxels with a predicted velocity field and supervising adjacent-frame renders, and learn geometry via photometric consistency using rendered depths and relative poses, all using only images.
Key results
- On nuScenes val, UVTR+VisionPAD (image-only, no LiDAR) reaches 41.0 mAP vs 38.5 for UVTR+UniPAD in the same image-only setting.
- On Occ3D val, UVTR+VisionPAD achieves 35.4 mIoU vs 31.0 for UVTR+UniPAD and 30.1 for UVTR baseline.
- Map segmentation lane IoU improves from 15.0 (UVTR) to 20.4 with VisionPAD, versus 16.3 with UniPAD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply image-only VisionPAD pre-training | UVTR · nuScenes val | mAP (%): 39.2 -> 41.2 | Helped |
| apply VisionPAD pre-training | UVTR · Occ3D val | mIoU (%): 30.1 -> 35.4 | Helped |
| apply VisionPAD pre-training | UVTR · nuScenes map segmentation | Lanes IoU (%): 15.0 -> 20.4 | Helped |
| replace volume rendering with a 3D-GS decoder (ablation) | UniPAD-style pre-training · nuScenes val | NDS: 22.3 -> 22.8 | No clear effect |
| replace volume rendering with a 3D-GS decoder (ablation) | UniPAD-style pre-training · nuScenes val | mAP: 19.4 -> 18.2 | Hurt |
| add low-confidence Gaussian filtering (ablation) | 3D-GS pre-training (Model A) · nuScenes val | NDS: 22.8 -> 23.4 | Helped |
| add low-confidence Gaussian filtering (ablation) | 3D-GS pre-training (Model A) · nuScenes val | mAP: 18.2 -> 18.9 | Helped |
| add voxel velocity estimation (ablation) | filtered 3D-GS pre-training (Model B) · nuScenes val | mAP: 18.9 -> 20.1 | Helped |
| add photometric consistency loss (ablation) | voxel-velocity model (Model C) · nuScenes val | NDS: 23.6 -> 26.0 | Helped |
| add photometric consistency loss (ablation) | voxel-velocity model (Model C) · nuScenes val | mAP: 20.1 -> 24.5 | Helped |
| vary anchors per voxel from one to 2/3/4 (ablation) | VisionPAD · nuScenes val | NDS: NDS dropping by 0.1/0.5/0.4 | Hurt |
Surprising findings
- UniPAD pre-training with image-only supervision slightly decreases NDS on nuScenes (45.0 to 44.8) relative to the UVTR baseline.
Limitations
- Gains with image-only supervision are reported against a UniPAD reimplementation; the full model with LiDAR depth supervision also reported.
- Ablations use reduced settings (50% pre-training data, lighter backbone, 128x128x5 volume).
Experimental setup
Datasets: nuScenes Occ3D
Benchmarks: nuScenes val (3D object detection) Occ3D val (semantic occupancy) nuScenes map segmentation (lane IoU)
Metrics: NDS mAP mIoU IoU
Baselines: UniPAD ViDAR BEVFormer TPVFormer BEVDet-Occ OPUS UVTR
Builds on: 3D Gaussian Splatting UniPAD UVTR anchor-based Gaussian splatting (Scaffold-GS) photometric consistency self-supervised depth
Closest papers by meaning
- UniPAD: A Universal Pre-training Paradigm for Autonomous Driving (CVPR 2024 · similarity 0.87)
- Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving (ICLR 2025 · similarity 0.86)
- DriveGEN: Generalized and Robust 3D Detection in Driving via Controllable Text-to-Image Diffusion Generation (CVPR 2025 · similarity 0.86)
- Achieving Speed-Accuracy Balance in Vision-based 3D Occupancy Prediction via Geometric-Semantic Disentanglement (AAAI 2025 · similarity 0.85)
- CorrBEV: Multi-View 3D Object Detection by Correlation Learning with Multi-modal Prototypes (CVPR 2025 · similarity 0.85)
- ProtoCar: Learning 3D Vehicle Prototypes from Single-View and Unconstrained Driving Scene Images (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.