Research map / Reinforcement learning / Visual and representation RL

Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environments

ICML 2025 · Reinforcement learning › Visual and representation RL · approach: In-context and adaptive RL · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PDTS, a robust active task sampling method that combines posterior sampling over a learned risk predictor with a diversity regularizer to select task batches for DR and Meta-RL, improving CVaR robustness and sometimes learning speed.

Problem

Robust adaptation of policies in randomized environments (domain randomization and meta-RL) requires costly evaluation of task difficulty to prioritize hard tasks; prior robust active task sampling (MPTS) degrades as the pseudo candidate batch grows due to concentration of selected tasks.

Approach

Frames robust active task sampling as an infinitely many-armed bandit over task subsets, replaces UCB acquisition with posterior sampling from a risk predictive model, and adds a pairwise-distance diversity regularizer to the subset selection so that large candidate pools can be used without concentration.

Key results

• PDTS outperforms ERM by more than 8% on all DR benchmarks in CVaR_0.9 and by as much as 73% on LunarLander.
• PDTS reaches ERM's best average return with 2.4x fewer training steps on Pusher and 1.3x on LunarLander.
• With PEARL backbone on HalfCheetah, PDTS achieves CVaR_0.95 return 993 vs 945 for MPTS and 855 for RoML.

Measured effects reported in the paper

Surprising findings

• MPTS performance collapses as the pseudo batch size grows (e.g., 8B) without remedy, while PDTS remains stable with 64B candidates.
• PDTS sometimes accelerates learning, not only improving robustness.

Limitations

• Relies on the risk predictive model being accurate for roughly scoring task difficulty.
• Assumes smoothness (Lipschitz) of adaptation risk over task identifiers and parameters, which may not hold in restricted scenarios.

Experimental setup

Datasets: MuJoCo Reacher MuJoCo Walker2d MuJoCo HalfCheetah LunarLander Pusher ErgoReacher ManiSkill3 LiftPegUpright_Light ManiSkill3 AnymalCReach_Goal

Benchmarks: MuJoCo Reacher MuJoCo Walker2d MuJoCo HalfCheetah LunarLander Pusher ErgoReacher ManiSkill3 LiftPegUpright_Light ManiSkill3 AnymalCReach_Goal

Metrics: CVaR return average return success ratio Pearson correlation coefficient wall-clock time memory cost

Baselines: ERM DRM GDRM MPTS RoML PEARL

Builds on: MPTS MAML PEARL TD3 PPO

Closest papers by meaning

• Bridging Risk Approximation Gaps in Model Predictive Task Sampling via In-Context Modeling (NeurIPS 2026 · similarity 0.90)
• Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learning (NeurIPS 2024 · similarity 0.88)
• Distributionally Robust Reinforcement Learning with Interactive Data Collection: Fundamental Hardness and Near-Optimal Algorithms (NeurIPS 2024 · similarity 0.88)
• Sample Complexity of Distributionally Robust Average-Reward Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
• Finite-Sample Analysis of Policy Evaluation for Robust Average Reward Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
• Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning (ICML 2024 · similarity 0.87)