Research map / Reinforcement learning / Visual and representation RL

Distill to Think, Foresee to Act: Cognitive-Physical Reinforcement Learning for Autonomous Driving

NeurIPS 2026 · Reinforcement learning › Visual and representation RL · approach: Inferring decision-making from behavior · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CoPhy, which distills VLM (InternVL3-8B) cognitive priors into a BEV encoder, trains an auto-regressive BEV world model as a physical sandbox, and optimizes the trajectory refiner with GRPO under dual physical and cognitive rewards, reaching 91.4 PDMS on NAVSIM v1 and 86.1 EPDMS on v2.

Problem

End-to-end autonomous driving trained by imitation learning is limited by the behavioral cloning ceiling and lacks cognitive scene understanding and explicit physical foresight for safety; existing RL approaches do not unify soft cognitive preferences with hard physical constraints.

Approach

Distill VLM cognitive features into the BEV encoder so the VLM can be discarded at inference, use a BEV-space auto-regressive world model to roll out candidate trajectories and derive physical rewards from predicted semantic maps, and combine these with a CLIP-aligned cognitive reward in GRPO. The cognitive channel also serves as a pluggable interface for user language commands.

Key results

• Achieves 91.4 PDMS on NAVSIM v1 navtest, +1.4 over the second-best DriveDPO (90.0).
• Achieves 86.1 EPDMS on NAVSIM v2 navtest, the highest reported, with NC 99.3 and TL 100.
• Ablation: VLM distillation raises PDMS from 87.1 to 88.6; full dual-reward RL reaches 91.4 from 88.9 IL-only (A2).

Surprising findings

• DriveSuprim achieves higher EP (86.7) than CoPhy (85.3) on NAVSIM v1, which the authors attribute to IL fitting expert progress at the cost of safety.

Limitations

• Cognitive reward uses expert trajectories as proxy for VLM intent, since no human labels are available.
• EP and EC are lower than some baselines on NAVSIM v2, indicating a safety-progress trade-off.
• Only the trajectory refiner is updated by RL; the world model and encoder are frozen, which may limit adaptation.

Experimental setup

Datasets: NAVSIM nuPlan OpenScene driving VQA datasets (unspecified)

Benchmarks: NAVSIM v1 NAVSIM v2

Metrics: PDMS EPDMS NC DAC EP TTC comfort DDC TL LK

Baselines: DiffusionDrive WoTE DriveSuprim DriveDPO ResAD VADv2 HydraMDP++ ARTEMIS

Builds on: GRPO DriveDPO LoRA InternVL3 CLIP

Closest papers by meaning

• Drive-R1: Bridging Reasoning and Planning in VLMs for Autonomous Driving with Reinforcement Learning (AAAI 2026 · similarity 0.90)
• AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning (NeurIPS 2025 · similarity 0.89)
• DriveAgent-R1: Advancing VLM-based Autonomous Driving with Active Perception and Hybrid Thinking (ICLR 2026 · similarity 0.88)
• $AutoDrive\text{-}P^3$: Unified Chain of Perception–Prediction–Planning Thought via Reinforcement Fine-Tuning (ICLR 2026 · similarity 0.88)
• DriveDPO: Policy Learning via Safety DPO For End-to-End Autonomous Driving (NeurIPS 2025 · similarity 0.88)
• Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving (NeurIPS 2026 · similarity 0.88)