Research map / Vision-language and multimodal / Robot action and navigation
Training-free Generation of Temporally Consistent Rewards from VLMs
Yinuo Zhao, Jiale Yuan, Zhiyuan Xu, Xiaoshuai Hao, Xinyi Zhang, Kun Wu, Zhengping Che, Chi Harold Liu, Jian Tang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add VLM initialization of particle filter instead of random initialization (ablation) | T2-VLM · Place-same-color reward accuracy | reward accuracy (%): 81.72 -> 94.60 | Improved |
| add particle filter instead of querying VLM at every step (ablation) | T2-VLM · first three tasks reward accuracy | reward accuracy (%): below 70% | Improved |
| use T2-VLM instead of VLM-score with Claude-3.5-sonnet | VLM-score · Place-same-color | reward accuracy (%): 54.77 -> 90.56 | Improved |
| use T2-VLM instead of VLM-score with Qwen2-VL-7B | VLM-score · Place-same-color | reward accuracy (%): 42.08 -> 83.25 | Improved |
| use T2-VLM instead of VLM-score with Claude-3.5-sonnet | VLM-score · Make-line | inference time (s): 166.20 -> 65.59 | Improved |
| use T2-VLM instead of VLM-score with Claude-3.5-sonnet | VLM-score · Stack-tower | VLM query times: 1620 -> 200 | Improved |
| use T2-VLM instead of VLM-score with Qwen2-VL-72B | VLM-score · Make-line | inference time (s): 42.39 -> 42.10 | No clear effect |
| use larger VLM (Qwen2-VL-72B instead of Qwen2-VL-7B) in VLM-score | VLM-score · Stack-tower | reward accuracy (%): 76.96 -> 68.12 | Hurt |
| use RL with T2-VLM rewards instead of REFLECT | REFLECT · recovery average | recovery success rate: 0.53 -> 0.93 | Improved |
| use RL with T2-VLM rewards instead of SayCan | SayCan · recovery average | recovery meta steps: 18.53 -> 9.73 | Improved |
| reduce PF hidden length below 25 (ablation) | T2-VLM · Place-same-color | reward accuracy (%) | Hurt |
Closest papers by meaning
- Code as Reward: Empowering Reinforcement Learning with VLMs (ICML 2024 · similarity 0.90)
- GoalLadder: Incremental Goal Discovery with Vision-Language Models (NeurIPS 2025 · similarity 0.90)
- Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning (ICLR 2024 · similarity 0.90)
- SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot Reinforcement Learning (NeurIPS 2026 · similarity 0.89)
- VL-RewardBench: A Challenging Benchmark for Vision-Language Generative Reward Models (CVPR 2025 · similarity 0.89)
- Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learning (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
