Research map / Vision-language and multimodal / Cross-modal learning
Hierarchical Adaptive Value Estimation for Multi-modal Visual Reinforcement Learning
Yangru Huang, Peixi Peng, Yifan Zhao, Haoran Xu, Mengyue Geng, Yonghong Tian
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace GVE (global critic on concatenated features) with LVE (per-modality critics with learned contribution weights) (ablation) | HAVE on RGB+event · CARLA ClearNight | episode return: 225 -> 274 | Improved |
| replace GVE with LVE (ablation) | HAVE on RGB+event · CARLA WetNoon | episode return: 226 -> 296 | Improved |
| replace GVE with LVE (ablation) | HAVE on RGB+event · CARLA CloudyNoon | episode return: 226 -> 293 | Improved |
| add task-contextual re-fusion of GVE and LVE on top of LVE alone (ablation) | HAVE on RGB+event · CARLA ClearNight | episode return: 274 -> 319 | Improved |
| add task-contextual re-fusion on top of LVE (ablation) | HAVE on RGB+event · CARLA WetNoon | episode return: 296 -> 341 | Improved |
| add task-contextual re-fusion on top of LVE (ablation) | HAVE on RGB+event · CARLA WetNight | driving distance (m): 196 -> 222 | Improved |
| fuse RGB and event modalities with GVE instead of RGB alone (ablation) | SAC encoder · CARLA ClearNight | episode return: 75 -> 225 | Improved |
| fuse RGB and event modalities with GVE instead of events alone (ablation) | SAC encoder · CARLA ClearNight | episode return: 158 -> 225 | Improved |
| fuse RGB and event modalities with GVE instead of events alone (ablation) | SAC encoder · CARLA WetNight | episode return: 175 -> 241 | No clear effect |
| use RGB frames only under adverse weather (ablation) | SAC encoder · CARLA ClearNight | episode return: 75 | Hurt |
| HAVE vs best baseline (TransFuser) with learned fusion modules | HAVE · CARLA ClearNight | episode return: 260 -> 319 | Improved |
| HAVE vs best baseline (EFNet) | HAVE · CARLA CloudyNight | episode return: 289 -> 322 | Improved |
Closest papers by meaning
- DMR: Decomposed Multi-Modality Representations for Frames and Events Fusion in Visual Reinforcement Learning (CVPR 2024 · similarity 0.91)
- Seek Commonality but Preserve Differences: Dissected Dynamics Modeling for Multi-modal Visual RL (NeurIPS 2024 · similarity 0.89)
- Vision Language Models are In-Context Value Learners (ICLR 2025 · similarity 0.86)
- COVR: Collaborative Optimization of VLMs and RL Agent for Visual-Based Control (AAAI 2026 · similarity 0.86)
- A Simple Framework for Generalization in Visual RL under Dynamic Scene Perturbations (NeurIPS 2024 · similarity 0.85)
- The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learning (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
