Research map / Reinforcement learning / RL from feedback
SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SURF reward learning, including pseudo-labeling and temporal cropping | PEBBLE · Meta-world manipulation tasks | success rate: ~50% -> ~100% success rate on Window Open with 400 queries | Helped |
| Add SURF reward learning | PEBBLE · DMControl locomotion tasks | ground-truth return | Helped |
| Add SURF reward learning | PEBBLE with DrQ-v2 backbone · DMControl visual control tasks | ground-truth return | Helped |
| Add pseudo-labeling on confident unlabeled samples (ablation) | PEBBLE reward learning · Walker-walk, 100 queries | ground-truth return | Helped |
| Add temporal cropping to supervised reward learning (ablation) | PEBBLE reward learning · Walker-walk, 100 queries | ground-truth return | Helped |
| Add temporal cropping to semi-supervised reward learning (ablation) | PEBBLE with pseudo-labeling · Walker-walk, 100 queries | ground-truth return | Helped |
| Replace no augmentation with random amplitude scaling (RAS) (ablation) | PEBBLE reward learning · Walker-walk, 100 queries | ground-truth return | Helped |
| Replace no augmentation with Gaussian noise (GN) (ablation) | PEBBLE reward learning · Walker-walk, 100 queries | ground-truth return | Hurt |
| Use temporal cropping instead of random amplitude scaling or Gaussian noise (ablation) | PEBBLE reward learning · Walker-walk, 100 queries | ground-truth return | Helped |
| Combine temporal cropping with random amplitude scaling (ablation) | PEBBLE reward learning · Walker-walk, 100 queries | ground-truth return | No clear effect |
| Combine temporal cropping with Gaussian noise (ablation) | PEBBLE reward learning · Walker-walk, 100 queries | ground-truth return | No clear effect |
| Increase the unlabeled batch ratio to 7 (ablation) | SURF · Walker-walk, 100 queries | ground-truth return | Hurt |
Closest papers by meaning
- Reinforcement Learning Guided Semi-Supervised Learning (NeurIPS 2024 · similarity 0.89)
- Leveraging Sub-Optimal Data for Human-in-the-Loop Reinforcement Learning (ICLR 2025 · similarity 0.89)
- Policy-labeled Preference Learning: Is Preference Enough for RLHF? (ICML 2025 · similarity 0.88)
- PEARL: Zero-shot Cross-task Preference Alignment and Robust Reward Learning for Robotic Manipulation (ICML 2024 · similarity 0.88)
- Improving Reward Model Generalization from Adversarial Process Enhanced Preferences (ICML 2025 · similarity 0.88)
- Active Reward Modeling: Adaptive Preference Labeling for Large Language Model Alignment (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.