Research map / Reinforcement learning / Offline RL

AdaWM: Adaptive World Model based Planning for Autonomous Driving

ICLR 2025 · Reinforcement learning › Offline RL · approach: Simple practical offline RL · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AdaWM, which identifies whether dynamics-model or policy mismatch dominates during finetuning of a pretrained world-model RL driving agent and then updates only the dominating component using low-rank (LoRA/NoLa-style) or sub-unit weight updates, evaluated in CARLA.

Problem

Finetuning pretrained world-model based RL planners for autonomous driving suffers dramatic performance degradation under distribution shift between pretraining and new tasks, and it is unclear which component (dynamics model or policy) to finetune and when.

Approach

Derives a performance-gap bound decomposing degradation into dynamics-model mismatch (TV distance of transitions) and policy mismatch (TV distance of policies), then at each finetuning step updates whichever mismatch dominates, using low-rank updates for the model and convex sub-unit weighting for the policy.

Key results

• On LTD03 (left turn, dense traffic), AdaWM reaches TTC 1.92 and SR 0.70, versus 0.35 SR for the best baseline DreamerV3 (no finetuning).
• On ROM03, AdaWM achieves TTC 2.05 and SR 0.82, outperforming model-only (SR 0.60) and policy-only (SR 0.72) finetuning.
• Performance is robust across threshold C between 2 and 50; very large C (100) degrades to policy-only behavior and drops SR substantially.

Measured effects reported in the paper

Surprising findings

• Naive alternating model+policy finetuning often performed worse than single-component finetuning in the reported CARLA tasks (e.g., ROM03 SR 0.52 vs 0.60 for model-only).

Limitations

• Theoretical bound relies on several assumptions (Lipschitz activations, bounded weights, TV distance estimates) and the practical criterion is a simplified heuristic.
• Evaluated only on CARLA driving tasks with a single pretraining task and a discrete action space.
• Baselines VAD and UniAD were not finetuned, so the comparison is asymmetric with the finetuned AdaWM.

Experimental setup

Datasets: CARLA leaderboard v2 tasks Bench2Drive

Benchmarks: CARLA ROM03 CARLA RTD12 CARLA LTM03 CARLA LTD03

Metrics: success rate time-to-collision

Baselines: UniAD VAD DreamerV3 Model-only finetuning Policy-only finetuning Model+Policy finetuning

Builds on: DreamerV3 RSSM LoRA Think2Drive

Closest papers by meaning

• PWM: Policy Learning with Multi-Task World Models (ICLR 2025 · similarity 0.88)
• Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving (NeurIPS 2026 · similarity 0.88)
• AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning (NeurIPS 2025 · similarity 0.88)
• Raw2Drive: Reinforcement Learning with Aligned World Models for End-to-End Autonomous Driving (in CARLA v2) (NeurIPS 2025 · similarity 0.87)
• Model-Based Policy Adaptation for Closed-Loop End-to-end Autonomous Driving (NeurIPS 2025 · similarity 0.87)
• CarPlanner: Consistent Auto-regressive Trajectory Planning for Large-Scale Reinforcement Learning in Autonomous Driving (CVPR 2025 · similarity 0.87)