Research map / Reinforcement learning / Visual and representation RL

HCRMP: An LLM-Hinted Contextual Reinforcement Learning Framework for Autonomous Driving

NeurIPS 2025 · Reinforcement learning › Visual and representation RL · approach: RL for combinatorial routing · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HCRMP, an LLM-Hinted RL motion planner in which an LLM supplies semantic state features and multi-critic weight hints (anchored by RAG and cached by a semantic memory) to a PPO agent in CARLA, rather than dictating actions.

Problem

LLM-dominated RL methods for autonomous driving motion planning over-rely on LLM outputs, which are prone to hallucinations (reported non-hallucination rate ~57.95% for Gemini-2.5-Pro) and degrade driving policy performance.

Approach

Keep the RL agent autonomous by having the LLM provide semantic hints only as auxiliary state features and multi-critic weights for the advantage function, stabilized by retrieval over a knowledge base and a historical semantic cache that handles low-frequency LLM latency.

Key results

• HCRMP reaches 99.0% SR in low-density overtaking and 93.0% in medium-density overtaking, with an average SR of 89.5% in medium and high density conditions reported in text.
• In safety-critical conditions at high density, HCRMP achieves 62.5% SR and 37.5% CR, with AV of 1.72 m/s^2, versus VLM-RL with 43.7% average CR.
• Ablation in medium-density trilemma: SR drops to 40% without ASR and 48% without CSA, versus 54% for HCRMP with ASR only.

Measured effects reported in the paper

Surprising findings

• VLM-RL achieves very high total distance in safety-critical conditions (up to 283 m) while its collision rate is high, and HCRMP trades distance for safety.

Limitations

• Ablation and main results are limited to CARLA Town 2 with a small set of driving conditions.
• Low-density gains over baselines are marginal, as acknowledged by the authors.
• The LLM used in the final system and its hallucination reduction are not quantified beyond the motivating evaluation of Gemini-2.5-Pro.

Experimental setup

Datasets: CARLA Town 2 simulated driving scenarios curated semantic reference corpus (regulations, standards, guidelines)

Benchmarks: CARLA Town 2 overtaking and merging CARLA occluded pedestrian and trilemma safety-critical scenarios

Metrics: success rate collision rate average speed total distance time steps speed variance acceleration variance

Baselines: Vanilla PPO E2ECLA AutoReward VLM-RL

Builds on: PPO Generalized Advantage Estimation retrieval-augmented generation ShuffleNetV2 FAISS

Closest papers by meaning

• Drive-R1: Bridging Reasoning and Planning in VLMs for Autonomous Driving with Reinforcement Learning (AAAI 2026 · similarity 0.89)
• Hybrid-Driving: An Autonomous Driving Decision Framework Integrating Large Language Models, Knowledge Graphs and Driving Rules (AAAI 2025 · similarity 0.89)
• AdaWM: Adaptive World Model based Planning for Autonomous Driving (ICLR 2025 · similarity 0.87)
• AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning (NeurIPS 2025 · similarity 0.87)
• DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models (ICLR 2024 · similarity 0.87)
• Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving (NeurIPS 2026 · similarity 0.87)