Research map / Vision-language and multimodal / Human and embodied modeling
DeepSight: Long-Horizon World Modeling via Latent States Prediction for End-to-End Autonomous Driving
Lingjun Zhang, Changjie Wu, Linzhe Shi, Jiangyang Li, Jiaxin Liu, Lei Yang, Hang Zhang, Mu Xu, Hong Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DINOv3 latent-feature world modeling instead of VAE reconstruction (ablation) | DeepSight · Dev 10 routes, one frame | Driving Score: 27.75 -> 74.79 | Improved |
| increase VAE world model from one to five frames (ablation) | VAE world model · Dev 10 routes | Driving Score: 27.75 -> 14.66 | Hurt |
| increase DINOv3 world model from one to five frames (ablation) | DeepSight · Dev 10 routes | Driving Score: 74.79 -> 86.57 | Improved |
| use BEV view instead of front view (ablation) | DeepSight · Dev 10 routes | Driving Score: 77.77 -> 86.57 | Improved |
| add world modeling (no adaptive CoT) (ablation) | DeepSight · Bench2Drive 220 routes | Driving Score: 58.16 -> 84.52 | Improved |
| add world modeling (no adaptive CoT) (ablation) | DeepSight · Bench2Drive 220 routes | Success Rate: 28.18 -> 65.91 | Improved |
| add adaptive CoT to model without world model (ablation) | DeepSight · Bench2Drive 220 routes | Success Rate: 28.18 -> 42.27 | Improved |
| add adaptive CoT to model without world model (ablation) | DeepSight · Bench2Drive 220 routes | Driving Score: 58.16 -> 69.87 | Improved |
| add adaptive CoT on top of world model (ablation) | DeepSight · Bench2Drive 220 routes | Driving Score: 84.52 -> 86.23 | Improved |
| add adaptive CoT on top of world model (ablation) | DeepSight · Bench2Drive 220 routes | Success Rate: 65.91 -> 71.36 | Improved |
| raise lambda_world from 1.0 to 2.0 (ablation) | DeepSight · Dev 10 routes | Driving Score: 86.57 -> 79.82 | Hurt |
| lower lambda_world from 1.0 to 0.5 (ablation) | DeepSight · Dev 10 routes | Driving Score: 86.57 -> 78.93 | Hurt |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 2 of 2 on most shared results.
- ORION — ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generation (ICCV 2025) · better on 3 of 3 shared results
- AutoVLA — AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning (NeurIPS 2025) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Driving (NeurIPS 2026 · similarity 0.91)
- OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation (NeurIPS 2026 · similarity 0.90)
- DriveSpatial: A Benchmark for Spatiotemporal Intelligence in VLMs for Autonomous Driving (NeurIPS 2026 · similarity 0.89)
- DriveFuture: Future-Aware Latent World Models for Autonomous Driving (NeurIPS 2026 · similarity 0.89)
- FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving (NeurIPS 2025 · similarity 0.88)
- Drive-R1: Bridging Reasoning and Planning in VLMs for Autonomous Driving with Reinforcement Learning (AAAI 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
