Research map / Reinforcement learning / RL from feedback
CoRe: Combined Rewards with Vision-Language Model Feedback for Preference-Aligned Reinforcement Learning
Hexian Ni, Tao Lu, Yinghao Cai
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Reward-Preference Alignment iterations to the formal reward (FRM0 to FRM2) (ablation) | Formal Reward Module · MetaWorld Sweep Into | success rate (%): 36.0 -> 93.3 | Improved |
| more alignment iterations (FRM0 to FRM4) (ablation) | Formal Reward Module · MetaWorld Hammer | success rate (%): 34.4 -> 54.0 | Improved |
| add VLM importance weights (KL prior) to residual reward (ablation) | Residual Reward Module · MetaWorld Hammer | success rate (%): 12.7 -> 49.3 | Improved |
| add importance weights to residual reward (ablation) | RRM · reward vs task progress (10 task avg) | Spearman rho: 0.56 -> 0.77 | Improved |
| combine FRM with RRM (ablation) | RRM alone · MetaWorld Button Press | success rate (%): 8.7 -> 100.0 | Improved |
| combine RRM with FRM (ablation) | FRM4 alone · MetaWorld Hammer | success rate (%): 54.0 -> 98.0 | Improved |
| video-level instead of image-level VLM preference labels (ablation) | RRM preference labeling · 10 tasks average | correct preference label rate (%): 41.6 -> 51.5 | Improved |
| use stronger LLM (GPT-5.4) for FRM (ablation) | FRM · MetaWorld Hammer | success rate (%): 54.0 -> 54.7 (9x cost) | No clear effect |
| use stronger VLM (Gemini 3 Flash) for RRM (ablation) | RRM · MetaWorld Hammer | success rate (%): 49.3 -> 62.6 (6x cost) | Improved |
| replace Gemini 2.0 Flash with open-source Qwen3-VL-8B (ablation) | CoRe · MetaWorld Hammer | success rate (%): 98.0 -> 88.7 | Hurt |
| replace Gemini 2.0 Flash with Video-LLaVA-7B (ablation) | CoRe · Sweep Into, Hammer | success rate | Hurt |
| add moving red cube visual disturbance (ablation) | CoRe · MetaWorld Sweep Into | success rate: 97.3% -> 91.7% | Hurt |
Closest papers by meaning
- Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning (NeurIPS 2024 · similarity 0.91)
- Reward Model Learning vs. Direct Policy Optimization: A Comparative Analysis of Learning from Human Preferences (ICML 2024 · similarity 0.91)
- Policy-labeled Preference Learning: Is Preference Enough for RLHF? (ICML 2025 · similarity 0.91)
- Tool-Augmented Reward Modeling (ICLR 2024 · similarity 0.90)
- Capturing Individual Human Preferences with Reward Features (NeurIPS 2025 · similarity 0.90)
- PILAF: Optimal Human Preference Sampling for Reward Modeling (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
