Research map / Reinforcement learning / Policy gradient and actor-critic
Learning from Active Human Involvement through Proxy Value Propagation
Zhenghao Peng, Wenjie Mo, Chenda Duan, Quanyi Li, Bolei Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| disable TD learning (J_TD(Q)=0) (ablation) | PVP · MetaDrive-Keyboard | success rate: 0.857 -> 0.220 | Hurt |
| add environmental reward (ablation) | PVP · MetaDrive-Keyboard | success rate: 0.857 -> 0.755 | Hurt |
| add environmental reward (ablation) | PVP · CARLA | route completion: 0.92 -> 0.793 | Hurt |
| remove balanced buffer (ablation) | PVP · CARLA | route completion: 0.92 -> 0.62 | Hurt |
| remove novice buffer (ablation) | PVP · CARLA | route completion: 0.92 -> 0.708 | Hurt |
| use stochastic policy for novice (ablation) | PVP · CARLA | route completion: 0.92 -> 0.40 | Hurt |
| use stochastic policy for novice (ablation) | PVP · CARLA | human data usage: 6.6K -> 12.3K | Hurt |
| replace PVP objective with vanilla CQL objective (ablation) | PVP · CARLA | success rate: 0.73 -> 0.266 | Hurt |
| make HACO policy deterministic (ablation) | HACO · CARLA | success rate: 0.40 -> 0.20 | No clear effect |
| replace HACO with PVP | PVP · MetaDrive-Keyboard | episodic return: 143.287 -> 353.636 | Improved |
| replace HACO with PVP | PVP · CARLA | route completion: 0.52 -> 0.92 | Improved |
| replace HACO with PVP in human data usage | PVP · MetaDrive-Keyboard | human data usage: 19.2K -> 14.6K | Improved |
Closest papers by meaning
- Reinforcement Learning from Imperfect Corrective Actions and Proxy Rewards (ICLR 2025 · similarity 0.87)
- Predictive Preference Learning from Human Interventions (NeurIPS 2025 · similarity 0.87)
- GUIDE: Real-Time Human-Shaped Agents (NeurIPS 2024 · similarity 0.86)
- Robot-Gated Interactive Imitation Learning with Adaptive Intervention Mechanism (ICML 2025 · similarity 0.86)
- Learning Human-Like RL Agents Through Trajectory Optimization With Action Quantization (NeurIPS 2025 · similarity 0.86)
- When Does Interaction Help? Representational Tradeoffs in Value-Based Imitation Learning (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
