Research map / Reinforcement learning / Visual and representation RL
SARM: Stage-Aware Reward Modeling for Long Horizon Robot Manipulation
Qianzhong Chen, Justin Yu, Mac Schwager, Pieter Abbeel, Fred Shentu, Philipp Wu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| stage-aware progress labels from subtask annotations (SARM) vs ReWiND | reward model · T-shirt rollout classification | rho: 0.50 -> 0.94 | Improved |
| add rewind augmentation (ablation) | SARM · T-shirt rollouts | rho: 0.67 -> 0.94 | Improved |
| add rewind augmentation (ablation) | SARM · T-shirt demo validation | MSE: 0.008 -> 0.009 | No clear effect |
| train on both dense and sparse annotations instead of dense only (ablation) | SARM · T-shirt rollouts | rho: 0.11 -> 0.94 | Improved |
| train on both annotation schemes instead of sparse only (ablation) | SARM · T-shirt rollouts | rho: 0.78 -> 0.94 | Improved |
| perturb frame brightness by up to 0.3 (ablation) | SARM · T-shirt rollouts | rho: 0.94 -> 0.78 | Hurt |
| filter demonstrations to under 2 minutes (BC-2min vs BC-All) (ablation) | Pi0 BC · T-shirt medium task, 40K | success: 1/12 -> 7/12 | Improved |
| duration filtering only, on the hard task (ablation) | Pi0 BC · T-shirt hard task, 40K | success: 0/12 -> 0/12 | No clear effect |
| apply RA-BC with SARM instead of BC-2min | Pi0 · T-shirt hard task, 40K | success: 0/12 -> 8/12 | Improved |
| use SARM rather than ReWiND reward model in RA-BC (ablation) | RA-BC · T-shirt medium task, 40K | success: 6/12 -> 10/12 | Improved |
| RA-BC with ReWiND reward model instead of plain BC-2min (ablation) | Pi0 · T-shirt medium task, 40K | success: 7/12 -> 6/12 | No clear effect |
| increase reward model from 4 to 8 layers (ablation) | SARM · T-shirt rollouts | rho: 0.72 -> 0.94 | Improved |
Closest papers by meaning
- Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learning (ICML 2025 · similarity 0.90)
- VICtoR: Learning Hierarchical Vision-Instruction Correlation Rewards for Long-horizon Manipulation (ICLR 2025 · similarity 0.89)
- Subtask-Aware Visual Reward Learning from Segmented Demonstrations (ICLR 2025 · similarity 0.88)
- Text2Reward: Reward Shaping with Language Models for Reinforcement Learning (ICLR 2024 · similarity 0.88)
- RoboAlign-R1: Distilled Multimodal Reward Alignment for Robot Video World Models (NeurIPS 2026 · similarity 0.88)
- DrS: Learning Reusable Dense Rewards for Multi-Stage Tasks (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
