Research map / Reinforcement learning / Visual and representation RL

Hard Tasks First: Multi-Task Reinforcement Learning Through Task Scheduling

ICML 2024 · Reinforcement learning › Visual and representation RL · approach: In-context and adaptive RL

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Scheduled Multi-Task Training (SMT), which schedules the hardest Meta-World tasks first using an SAC-based evaluation metric, with periodic network resets and budget-based stages to counter simplicity bias.

Problem

Multi-task RL with tasks of varying difficulty suffers negative transfer and simplicity bias, where easy tasks dominate gradient updates and hard tasks go unlearned.

Approach

Dynamically prioritize the K hardest tasks (ranked by a probabilistic-inference evaluation metric of return minus log-policy) in a training pool, periodically reset policy and Q networks while keeping replay buffers, and use a two-stage budget to revisit unsolvable tasks.

Key results

• SMT reaches 86.8% average success on Meta-World MT10, versus 82.4% for the next-best baseline PaCo and 69.4% for SAC-MT.
• On MT50, SMT attains 8.0% average bottom-20 success versus 4.6% for PaCo.
• Ablation: removing the reset mechanism drops MT10 average success from 86.8% to 70.7%; hard-tasks-first scheduling beats easy-tasks-first (62.7%) and random (62.6%).

Measured effects reported in the paper

Surprising findings

• Difficult 'push' task is learned when paired with another difficult task (peg-insert-side) but fails when paired with easy 'reach', showing simplicity bias.

Limitations

• Evaluated only on Meta-World with SAC backbone; generalization to other RL algorithms and domains not shown.
• Performance variance across tasks remains high (e.g., std of 31.9 on push-related tasks).

Experimental setup

Datasets: Meta-World MT10 Meta-World MT50

Benchmarks: Meta-World MT10 Meta-World MT50

Metrics: average success ratio (%) average bottom-k success ratio

Baselines: SAC-MT SAC-MT-MH SAC-soft-modular PCGrad PaCo

Builds on: SAC VariBAD Resetting deep RL (Nikishin et al., 2022) PCGrad

Closest papers by meaning

• ARS: Adaptive Reward Scaling for Multi-Task Reinforcement Learning (ICML 2025 · similarity 0.89)
• Task-Aware Virtual Training: Enhancing Generalization in Meta-Reinforcement Learning for Out-of-Distribution Tasks (ICML 2025 · similarity 0.88)
• Mastering Massive Multi-Task Reinforcement Learning via Mixture-of-Expert Decision Transformer (ICML 2025 · similarity 0.88)
• AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers (NeurIPS 2024 · similarity 0.87)
• Quantifying Task Priority for Multi-Task Optimization (CVPR 2024 · similarity 0.86)
• Prioritized Soft Q-Decomposition for Lexicographic Reinforcement Learning (ICLR 2024 · similarity 0.86)