Research map / Vision-language and multimodal / Multimodal LLMs

COVR: Collaborative Optimization of VLMs and RL Agent for Visual-Based Control

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes COVR, a collaborative framework where RL-generated trajectories fine-tune a VLM (via LoRA with filtered, return-weighted NLL) and the tuned VLM provides action priors that regularize SAC/RAD policy learning on CARLA and DMControl.

Problem

Visual RL is sample-inefficient under high-dimensional observations; prior VLM-assisted methods distill from a frozen VLM, which limits gains when the VLM lacks domain knowledge, and they do not feed RL-generated data back to improve the VLM.

Approach

Close the loop between VLM and RL: fine-tune the VLM on RL interaction data selected by an exploration-driven dynamic threshold and weighted by normalized returns, then use the improved VLM's action outputs as a regularization prior for the policy, with progressively less frequent fine-tuning.

Key results

• On CARLA #HW, COVR achieves ER 248 ± 81 and DD 259 ± 85, above the best baseline DGC (208 ± 13 ER) and ResAct (227 ± 36 ER).
• On DMControl, RAD+COVR reaches 872 on Cartpole Swingup, 969 on Reacher Easy and 960 on Ball in cup Catch, with COVR's reported results listed as best among compared methods.
• Ablations show removing EDDF (random filtering) drops #HW ER to 144 ± 74 and removing RALW gives 204 ± 111 versus 248 ± 81 for the full model.

Surprising findings

• Cross-scenario tests show the VBE with a basic VLM is negative in CARLA (ER -11), so the raw VLM alone is poor at driving control.
• Qwen2.5-VL-3B outperforms the larger LLaVA-1.5-7B in the fine-tuned setting, which the authors attribute to better driving understanding.

Limitations

• Does not explicitly model continuous observation sequences; fine-tuned VLM struggles with some dynamic inputs.
• Sensitive to lambda and psi_0 hyperparameters.
• Large standard deviations in several reported results.

Experimental setup

Datasets: CARLA highway (#HW) RL-generated trajectories CARLA ghost pedestrian (#GP) RL-generated trajectories DMControl tasks CarRacing

Benchmarks: CARLA #HW CARLA #GP DMControl CarRacing DMControl hard tasks

Metrics: episode reward (ER) driving distance (DD) episode return

Baselines: SAC DeepMDP CURL DrQ SPR MLR PER ERE ResAct VBE DPL APL

Builds on: Soft Actor-Critic RAD DeepMDP DPL LoRA

Closest papers by meaning

• VLMs-Guided Representation Distillation for Efficient Vision-Based Reinforcement Learning (CVPR 2025 · similarity 0.91)
• Code as Reward: Empowering Reinforcement Learning with VLMs (ICML 2024 · similarity 0.91)
• Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning (NeurIPS 2024 · similarity 0.90)
• Empowering Small VLMs to Think with Dynamic Memorization and Exploration (ICLR 2026 · similarity 0.89)
• What Can RL Bring to VLA Generalization? An Empirical Study (NeurIPS 2025 · similarity 0.89)
• NoRD: A Data-Efficient Vision-Language-Action Model that Drives without Reasoning (CVPR 2026 · similarity 0.89)