Research map / 3D vision / 3D generation
DV-3DLane: End-to-end Multi-modal 3D Lane Detection with Dual-view Representation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DV-3DLane, an end-to-end multi-modal (camera + LiDAR) 3D lane detector that learns features in both perspective-view and BEV spaces with bidirectional fusion, unified query generation, and 3D dual-view deformable attention.
Problem
Monocular 3D lane detection suffers from depth loss and lighting variation, IPM-based BEV methods suffer from flat-ground assumptions, and prior multi-modal lane methods fuse into a single view and underuse image features.
Approach
Keep modality features in both perspective view (PV) and BEV branches, fuse camera and LiDAR bidirectionally between the symmetric branches, generate lane-aware queries from both views and cluster them into a unified query set (lane-centric clustering with InfoNCE), and sample features consistently across views via 3D deformed reference points projected into each view.
Key results
- On OpenLane-1K at 0.5m threshold, DV-3DLane-Large reaches 65.2 F1 vs LATR 54.0 (+11.2), and Base reaches 63.5 F1.
- At 1.5m threshold, DV-3DLane-Large achieves 66.8 F1 and 93.3 accuracy, versus LATR 61.9 F1 and 92.0 accuracy.
- Ablation on OpenLane-300: bidirectional fusion improves F1 from 67.9 to 70.7; dual-view queries achieve 70.7 vs 69.7 for random queries.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use dual-view lane-aware queries instead of random query initialization (ablation) | DV-3DLane · OpenLane-300, 0.5m | F1: 69.7 -> 70.7 | Helped |
| Generate lane-aware queries from both PV and BEV instead of using PV queries alone (ablation) | DV-3DLane · OpenLane-300, 0.5m | F1: 69.6 -> 70.7 | Helped |
| Generate lane-aware queries from both PV and BEV instead of using BEV queries alone (ablation) | DV-3DLane · OpenLane-300, 0.5m | F1: 69.1 -> 70.7 | Helped |
| Use dual-view deformable attention rather than PV-space-only sampling (ablation) | DV-3DLane · OpenLane-300, 0.5m | F1: 63.6 -> 70.7 | Helped |
| Use dual-view deformable attention rather than BEV-space-only sampling (ablation) | DV-3DLane · OpenLane-300, 0.5m | F1: 68.5 -> 70.7 | Helped |
| Use consistent dual-view sampling instead of sequential feature fusion (ablation) | DV-3DLane · OpenLane-300, 0.5m | F1: +2.0 F1 | Helped |
| Use consistent sampling locations across views rather than modality-agnostic FUTR3D sampling (ablation) | DV-3DLane · OpenLane-300, 0.5m | F1: +0.5 | Helped |
| Add dual-view representation to image-only input (ablation) | Image branch · OpenLane-1K, 0.5m | F1: 52.9 -> 54.3 | Helped |
| Add dual-view representation to LiDAR-only input (ablation) | LiDAR branch · OpenLane-1K, 0.5m | F1: 54.1 -> 55.3 | Helped |
| Add bidirectional feature fusion instead of no fusion (ablation) | DV-3DLane · OpenLane-300, 0.5m | F1: +2.8% | Helped |
| Replace bidirectional fusion with one-way camera-to-LiDAR fusion (ablation) | DV-3DLane · OpenLane-300, 0.5m | F1: 70.7 -> 69.1 | Hurt |
| Replace bidirectional fusion with one-way LiDAR-to-camera fusion (ablation) | DV-3DLane · OpenLane-300, 0.5m | F1: 70.7 -> 69.0 | Hurt |
Surprising findings
- Naively adding LiDAR to LATR (LATR + LiDAR) gives only 57.4 F1 at 0.5m, while the dual-view design reaches 63.5 with the same modalities.
- Single-space PV-only queries (69.6) perform slightly worse than random query initialization (69.7) in the ablation.
Limitations
- Relies on LiDAR input, so it requires multi-sensor data; OpenLane is noted as the sole public multi-modal 3D lane dataset.
- Inference speed drops with larger variants (Base 8.82 FPS, Large 6.18 FPS on V100), though Tiny runs at 13.49 FPS.
- Robustness to calibration noise degrades sharply without noise-augmented training (63.5 to 32.4 F1 at Prob=0.7 under Noise).
Experimental setup
Datasets: OpenLane Apollo 3D Synthetic
Benchmarks: OpenLane Apollo 3D Synthetic
Metrics: F1 category accuracy X error (near/far) Z error (near/far)
Baselines: 3DLaneNet GenLaneNet PersFormer Anchor3DLane M^2-3DLaneNet LATR LATR + LiDAR DeepInteration FUTR3D
Builds on: LATR BEVDepth k-Max-DeepLab FUTR3D DeepInteration
Closest papers by meaning
- PVALane: Prior-Guided 3D Lane Detection with View-Agnostic Feature Alignment (AAAI 2024 · similarity 0.90)
- Rethinking Lanes and Points in Complex Scenarios for Monocular 3D Lane Detection (CVPR 2025 · similarity 0.90)
- GLane3D: Detecting Lanes with Graph of 3D Keypoints (CVPR 2025 · similarity 0.88)
- Sketch and Refine: Towards Fast and Accurate Lane Detection (AAAI 2024 · similarity 0.86)
- GEONet: Global Enhancement and Optimization Network for Lane Detection (AAAI 2025 · similarity 0.86)
- LaneCPP: Continuous 3D Lane Detection using Physical Priors (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.