Research map / Label-efficient and robust learning / Recognition and long-tail
Navigation-Guided Sparse Scene Representation for End-to-End Autonomous Driving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SSR, an end-to-end driving framework that compresses BEV features into 16 navigation-guided scene tokens and uses temporal future-BEV self-supervision instead of perception annotations.
Problem
End-to-end autonomous driving methods rely on expensive supervised perception sub-tasks (detection, mapping, occupancy) that limit real-time deployment and data scalability.
Approach
Extract only 16 sparse scene tokens from BEV features via a navigation-conditioned TokenLearner (SE-modulated by the high-level command), and supervise a future feature predictor by reconstructing the actual future BEV feature, so no perception labels are needed.
Key results
- On nuScenes, SSR reaches 0.75 m average L2 and 0.15% collision rate (MAX/AVG protocol as reported), 19.6 FPS, versus UniAD's 10.9x slower inference.
- On CARLA Town05 Long, SSR achieves 78.9 driving score vs 57.2 for TCP and 30.3 for VAD-Base.
- Reducing scene queries from 16 to 8 lowers average L2 (0.69 m) but raises collision rate, so 16 was chosen as default.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Scenes TokenLearner (16 navigation-guided scene tokens) instead of direct BEV interaction (ablation) | SSR · nuScenes open-loop (MAX metric) | collision rate Avg (%): 0.43 -> 0.20 | Helped |
| add Future Feature Predictor (temporal self-supervision) (ablation) | SSR with STL · nuScenes open-loop (MAX metric) | collision rate Avg (%): 0.20 -> 0.15 | Helped |
| add Future Feature Predictor (temporal self-supervision) (ablation) | SSR with STL · nuScenes open-loop (MAX metric) | L2 Avg (m): 0.75 -> 0.75 | No clear effect |
| use 8 scene queries instead of 16 (ablation) | SSR · nuScenes open-loop (MAX metric) | L2 Avg (m): 0.75 -> 0.69 | Helped |
| use 64 scene queries instead of 16 (ablation) | SSR · nuScenes open-loop (MAX metric) | collision rate Avg (%): 0.15 -> 0.41 | Hurt |
| add navigation command guidance (SE layer) (ablation) | SSR · nuScenes open-loop, go-straight and turn-left/right subsets | L2 Avg (m) on GS and LR: GS 0.72 -> 0.71; LR 1.09 -> 1.04 | Helped |
| add navigation command guidance (SE layer) (ablation) | SSR · nuScenes open-loop, turn left/right subset | collision rate Avg (%): LR 0.78 -> 0.66; GS 0.29 -> 0.10 | Helped |
| add pretrained perception weights (48 epochs) to SSR (ablation) | SSR · nuScenes open-loop (MAX metric) | L2 Avg (m): 0.75 -> 1.11 | Hurt |
| add map and/or observation perception auxiliary tasks to SSR (ablation) | SSR · nuScenes open-loop (MAX metric) | collision rate Avg (%): 0.15 -> 0.32 (map), 0.35 (obs), 0.26 (both) | Hurt |
| remove navigation command (turn right command used at test) (ablation) | SSR · nuScenes open-loop (MAX metric) | collision rate Avg (%): 0.15 -> 0.53 (turn right command) | Hurt |
| SSR built on VAD-Tiny with 16 scene tokens vs SparseDrive/UAD-type baselines | SSR · CARLA Town05 Long | Driving Score: 78.9 vs VAD-Base 30.3 (+48.6) | Helped |
Surprising findings
- Adding pretrained perception modules (48-epoch VAD-style detection/mapping pretraining) did not improve planning and degraded L2 and collision rate in the reported table.
- Using 64 scene queries performs worse than 16, suggesting too many queries reintroduce perception redundancy.
Limitations
- Relies on simple high-level navigation commands, which may limit effectiveness in complex scenarios requiring routing or natural language.
- Ambiguous navigation commands in nuScenes labels and noisy first frames in the temporal module cause failure cases.
Experimental setup
Datasets: nuScenes CARLA Town01/03/04/06 Roach-collected frames
Benchmarks: nuScenes open-loop planning (L2, collision rate) CARLA Town05 Long closed-loop
Metrics: L2 error collision rate curb collision rate FPS driving score route completion infraction score
Baselines: UniAD VAD-Base VAD-Tiny OccNet ST-P3 PARA-Drive GenAD UAD LAW SparseDrive BEV-Planner TCP
Builds on: VAD BEVFormer TokenLearner TokenFuser TCP
Closest papers by meaning
- DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving (ICLR 2025 · similarity 0.88)
- SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Driving (CVPR 2026 · similarity 0.87)
- DualAD: Disentangling the Dynamic and Static World for End-to-End Driving (CVPR 2024 · similarity 0.87)
- Enhancing End-to-End Autonomous Driving with Latent World Model (ICLR 2025 · similarity 0.87)
- S4-Driver: Scalable Self-Supervised Driving Multimodal Large Language Model with Spatio-Temporal Visual Representation (CVPR 2025 · similarity 0.86)
- Prune2Drive: A Plug-and-Play Framework for Accelerating Vision-Language Models in Autonomous Driving (CVPR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.