Research map / Vision-language and multimodal / Multimodal LLMs
PRIMT: Preference-based Reinforcement Learning with Multimodal Feedback and Trajectory Synthesis from Foundation Models
Ruiqi Wang, Dezhong Zhao, Ziqin Yuan, Tianyu Shao, Guohua Chen, Dominic Kao, Sungeun Hong, Byung-Cheol Min
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
placeholder
Problem
placeholder
Approach
placeholder
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PRIMT's multimodal feedback fusion and bidirectional trajectory synthesis instead of single-modality FM feedback | PbRL on the evaluated robotics tasks · 8 DMC, MetaWorld, and ManiSkill tasks | task performance | Improved |
| use PRIMT instead of the scripted-teacher PrefGT baseline | PbRL · Sweep Into and PegInsertionSide | task performance | Improved |
| use PRIMT's hierarchical neuro-symbolic fusion instead of naive multimodal evaluation | synthetic preference feedback · Evaluated tasks | preference-label accuracy | Improved |
| remove the intra-modal crowd-check and preference-fusion module (ablation) | PRIMT · Door Open and PickSingleYCB | task performance | Hurt |
| remove PSL-based inter-modal preference fusion (ablation) | PRIMT · Door Open and PickSingleYCB | task performance | Hurt |
| remove foresight trajectory generation (ablation) | PRIMT · Door Open and PickSingleYCB | early-stage learning speed | Hurt |
| add foresight trajectory generation to initialize the trajectory buffer (ablation) | PRIMT · Evaluated tasks | preference indecision rate | Improved |
| remove hindsight trajectory augmentation (ablation) | PRIMT · Door Open and PickSingleYCB | final task performance | Hurt |
| remove the causal auxiliary loss while retaining counterfactual trajectory augmentation (ablation) | PRIMT · Door Open and PickSingleYCB | final task performance | Hurt |
| replace hierarchical multimodal fusion with direct multimodal FM evaluation | PbRL · Evaluated tasks | task performance | Hurt |
| replace GPT-4o with GPT-4o-mini (ablation) | PRIMT · Evaluated tasks | performance: 16–26% | Hurt |
Closest papers by meaning
- Hindsight PRIORs for Reward Learning from Human Preferences (ICLR 2024 · similarity 0.88)
- DecisionNCE: Embodied Multimodal Representations via Implicit Preference Learning (ICML 2024 · similarity 0.87)
- CLARIFY: Contrastive Preference Reinforcement Learning for Untangling Ambiguous Queries (ICML 2025 · similarity 0.87)
- Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning (NeurIPS 2024 · similarity 0.87)
- Efficient Language-instructed Skill Acquisition via Reward-Policy Co-Evolution (AAAI 2025 · similarity 0.87)
- LIBERO-PeRM: Benchmarking Personalized Robotic Manipulation (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
