Research map / Reinforcement learning / Offline RL

ReflectDrive-2: Reinforcement-Learning-Aligned Self-Editing for Discrete Diffusion Driving

NeurIPS 2026 · Reinforcement learning › Offline RL · approach: Diffusion models for offline decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ReflectDrive-2, a masked discrete diffusion driving VLA whose in-place token-to-token AutoEdit is pretrained on structure-aware perturbations and then co-trained with the drafter via RL on the full draft-and-edit rollout, reaching 91.0 PDMS camera-only on NAVSIM at 30.2 ms on Thor.

Problem

Imitation-learned driving planners accumulate errors along longitudinal speed and lateral heading axes and typically cannot revise emitted trajectories efficiently; existing editing for discrete diffusion is heuristic or trained separately from the drafter.

Approach

Use a masked discrete diffusion planner whose token space allows the same model to rewrite its own trajectory tokens (AutoEdit), trained on longitudinal/lateral perturbations and then optimized with a single terminal driving reward over the full draft-then-edit rollout so drafter and editor co-adapt.

Key results

• ReflectDrive-2 reaches 91.0 PDMS camera-only on NAVSIM, above ReCogDrive (90.8) and GoalFlow (90.3, camera+LiDAR).
• Inference-time AutoEdit adds only +0.3 PDMS after supervised training but +1.9 PDMS after RL over the draft-and-edit rollout (89.1 to 91.0).
• Best-of-6 oracle reaches 94.8 PDMS and the full stack runs at 30.2 ms average latency on NVIDIA Thor.

Surprising findings

• Inference-time AutoEdit gives almost no gain (at most +0.3 PDMS) under supervised training alone, and only becomes useful after RL over the composed rollout.
• Replacing alternating full-step frames with lite-step temporal AutoEdit costs only -0.20 on the in-house overall score.

Limitations

• Fixed-resolution BEV coordinate tokens bound waypoint spatial precision.
• RL reward is a lightweight closed-loop PDMS proxy rather than full real-world objectives.
• AutoEdit perturbations cover only longitudinal and lateral errors, not interaction-level failures.

Experimental setup

Datasets: NAVSIM navtrain

Benchmarks: NAVSIM navtest in-house deployment benchmark

Metrics: PDMS NC DAC TTC Comfort EP minSADE meanSADE path-minADE path-meanFDE

Baselines: UniAD TransFuser Hydra-MDP DiffusionDrive GoalFlow AutoVLA DriveVLA-W0 ReCogDrive DriveFine HDP

Builds on: LLaDA 2.1 token-to-token editing MaskGIT d2 diffusion RL DriveFine GRPO

Closest papers by meaning

• Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous and Instruction-guided Driving (CVPR 2024 · similarity 0.88)
• DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving (CVPR 2025 · similarity 0.87)
• Drive-R1: Bridging Reasoning and Planning in VLMs for Autonomous Driving with Reinforcement Learning (AAAI 2026 · similarity 0.87)
• $AutoDrive\text{-}P^3$: Unified Chain of Perception–Prediction–Planning Thought via Reinforcement Fine-Tuning (ICLR 2026 · similarity 0.86)
• Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving (NeurIPS 2026 · similarity 0.86)
• Diffusion-Based Planning for Autonomous Driving with Flexible Guidance (ICLR 2025 · similarity 0.86)