Research map / Detection and segmentation / 3D detection and driving
DiffBEV: Conditional Diffusion Model for Bird’s Eye View Perception
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiffBEV, an end-to-end framework that uses a conditional diffusion probability model with a cross-attention fusion module to refine LSS-style BEV features for semantic segmentation and 3D object detection.
Problem
Noisy and distorted BEV features (from inaccurate camera parameters and LiDAR scans, flat-world assumptions, sparse BEV distributions) degrade BEV semantic segmentation and 3D detection quality.
Approach
Treat the view transformer's BEV feature or depth-derived semantic feature as the condition of a conditional diffusion model that progressively denoises noisy BEV features, then fuse its output with the original BEV feature via cross-attention; trained end-to-end without a separate pre-training stage.
Key results
- DiffBEV-Dep reaches 25.9% mIoU on nuScenes BEV segmentation, 6.2 points above the best prior method (LSS, 19.7%).
- BEVDet with DiffBEV raises NDS from 38.7% to 39.8%; BEVDet4D with DiffBEV raises NDS from 47.6% to 48.6% on nuScenes val.
- On KITTI Raw, the depth-derived condition with cross-attention reaches 65.86% mIoU versus 63.38% for the LSS baseline when using self-attention encoding.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add conditional DiffBEV with depth-distribution condition | LSS · nuScenes val | mIoU: 19.7 -> 25.9 | Helped |
| add DiffBEV | BEVDet-Tiny · nuScenes val | NDS: 0.387 -> 0.398 | Helped |
| add DiffBEV | BEVDet4D-Tiny · nuScenes val | NDS: 0.476 -> 0.486 | Helped |
| use depth-distribution semantic features rather than original BEV features as the diffusion condition (ablation) | DiffBEV · KITTI Raw | mIoU: 64.33 -> 65.86 | Helped |
| use summed BEV and depth features rather than original BEV features as the diffusion condition (ablation) | DiffBEV · KITTI Raw | mIoU: 64.33 -> 65.16 | Helped |
| replace concatenation with cross-attention feature interaction (ablation) | DiffBEV · KITTI Raw | mIoU: 65.03 -> 65.86 | Helped |
| replace addition with cross-attention feature interaction (ablation) | DiffBEV · KITTI Raw | mIoU: 64.85 -> 65.86 | Helped |
| replace self-attention encoding of noisy samples with convolution (ablation) | DiffBEV · KITTI Raw | mIoU: 65.86 -> 64.23 | Hurt |
| add DiffBEV | VPN · KITTI 3D Object | mIoU: 27.02 -> 28.21 (+1.19) | Helped |
| add DiffBEV | PYVA · KITTI 3D Object | mIoU: 29.22 -> 30.83 (+1.61) | Helped |
| add DiffBEV | PON · KITTI 3D Object | mIoU: 36.49 -> 37.08 (+ 0.59) | Helped |
Surprising findings
- A simple convolutional encoder for the noisy BEV sample achieves 64.23% mIoU versus 65.86% with self-attention, at lower GFLOPs (433.72 vs 446.81).
Limitations
- Gains on rare dynamic classes (e.g., construction vehicle, motorcycle) remain low in absolute IoU.
- Adds a diffusion denoising module on top of the view transformer, increasing parameters (78.16M to 78.80M) and compute.
Experimental setup
Datasets: nuScenes KITTI Raw KITTI Odometry KITTI 3D Object
Benchmarks: nuScenes KITTI Raw KITTI Odometry KITTI 3D Object
Metrics: IoU mIoU mAP NDS mATE mASE mAOE mAVE mAAE
Baselines: LSS PYVA PON VPN VED IPM Unproj. OFT MonoOcc Mono3D BEVDet BEVDet4D
Builds on: LSS BEVDet BEVDet4D DDPM CenterPoint
Closest papers by meaning
- BEVDiffuser: Plug-and-Play Diffusion Model for BEV Denoising with Ground-Truth Guidance (CVPR 2025 · similarity 0.90)
- Diff-2-in-1: Bridging Generation and Dense Perception with Diffusion Models (ICLR 2025 · similarity 0.90)
- Toward Real-world BEV Perception: Depth Uncertainty Estimation via Gaussian Splatting (CVPR 2025 · similarity 0.87)
- DiffSF: Diffusion Models for Scene Flow Estimation (NeurIPS 2024 · similarity 0.87)
- M-BEV: Masked BEV Perception for Robust Autonomous Driving (AAAI 2024 · similarity 0.87)
- What Matters When Repurposing Diffusion Models for General Dense Perception Tasks? (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.