Research map / Reinforcement learning / Policy gradient and actor-critic
CUP: Critic-Guided Policy Reuse
Jin Zhang, Siyuan Li, Chongjie Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CUP’s critic-guided source-policy selection and adaptive imitation regularization instead of SAC alone | SAC · Meta-World: Hammer, Peg-Insert-Side, Push-Wall, Pick-Place-Wall, Push-Back, Shelf-Place | cumulative return | Improved |
| Use CUP instead of HAAR, PTF, MULTIPOLAR, or MAMBA for policy reuse | Meta-World policy reuse · Meta-World target tasks | cumulative return | Improved |
| Add up to 3 random source policies (ablation) | CUP · Meta-World target tasks | cumulative return | No clear effect |
| Add 4 or 5 random source policies (ablation) | CUP · Meta-World target tasks | cumulative return | Hurt |
| Use 3 random source policies with no useful source policy (ablation) | CUP · Meta-World transfer with useless sources | sample efficiency | No clear effect |
| Add Reach as a useful source policy alongside 3 random policies (ablation) | CUP · Meta-World transfer with 3 Random Sources+Reach | learning performance | Improved |
| Increase the source-policy set from 3 to 6 policies (ablation) | CUP · Meta-World, especially Pick-Place-Wall and Peg-Insert-Side | performance | Improved |
| Increase the source-policy set from 3 to 6 policies (ablation) | MULTIPOLAR · Meta-World target tasks | performance | Hurt |
| Vary CUP’s additional regularization hyperparameters (ablation) | CUP · Meta-World target tasks | performance | No clear effect |
| Use a source-policy set containing only unrelated or less-related policies (ablation) | CUP · Bin-Picking and Stick-Pull | performance improvement over SAC | Mixed |
Closest papers by meaning
- Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RL (ICML 2024 · similarity 0.85)
- RICE: Breaking Through the Training Bottlenecks of Reinforcement Learning with Explanation (ICML 2024 · similarity 0.85)
- Novelty-Guided Data Reuse for Efficient and Diversified Multi-Agent Reinforcement Learning (AAAI 2025 · similarity 0.84)
- ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantages (ICML 2024 · similarity 0.84)
- The Ladder in Chaos: Improving Policy Learning by Harnessing the Parameter Evolving Path in A Low-dimensional Space (NeurIPS 2024 · similarity 0.84)
- Diffusion Actor-Critic: Formulating Constrained Policy Iteration as Diffusion Noise Regression for Offline Reinforcement Learning (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
