Research map / Reinforcement learning / Offline RL

Pessimistic Latent Task-aware Optimization for Robust Offline Meta-Reinforcement Learning

NeurIPS 2026 · Reinforcement learning › Offline RL · approach: Offline RL sample complexity theory

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PLATO, an offline context-based meta-RL method that perturbs inferred task latents outward from the training centroid, rolls out a decoder ensemble at the perturbed latents, and updates the policy on those imagined transitions with an ensemble-disagreement pessimism weight.

Problem

Context-based offline meta-RL policies are never trained on latents outside the training latent support, so out-of-support (parameter-extrapolation) tasks produce inputs the policy has never seen, and better encoders alone do not close this gap.

Approach

Synthesize off-support task latents by pushing inferred latents radially outward from the batch centroid (scaled by edge weight and batch spread), roll out a task-conditioned decoder ensemble there, and weight each imagined step by 1/(1+λ·u) where u is ensemble variance, while keeping the critic trained on real data only.

Key results

• PLATO attains the highest out-of-support return on all eight MuJoCo task families, e.g. Walker-Friction 649±27 vs 454±74 for the best baseline ER-TRL.
• On Hopper-Friction, off-support ensemble disagreement averages 8.8e-4 vs 6.7e-5 on ID tasks (about 13x).
• On MetaWorld Reach, PLATO reaches 0.53 out-of-support success rate vs 0.43 for the strongest baseline.

Surprising findings

• Ablating the centroid-outward direction (random direction) still beats the default on Humanoid-Dir, and the Q-synth variant (imagined data in the critic) underperforms the default on all six environments.

Limitations

• Relies on an outwardness assumption on average latent position that is not guaranteed; the paper notes PLATO retains advantage where it is not confirmed.
• Targets extrapolation within a task family; transfer to qualitatively different task families is out of scope.
• Theoretical guarantees are conditional or worst-case, with persistence depending on an assumed parameter-transport budget.

Experimental setup

Datasets: MuJoCo offline meta-RL task datasets (Ant-Goal, Ant-Dir, Cheetah-Vel, Humanoid-Dir, Hopper-Mass, Hopper-Friction, Walker

Benchmarks: Ant-Dir Ant-Goal Cheetah-Vel Humanoid-Dir Hopper-Mass Hopper-Friction Walker-Mass Walker-Friction MetaWorld Reach

Metrics: normalized return success rate

Baselines: Offline PEARL FOCAL CSRO UNICORN ER-TRL

Builds on: BRAC CSRO FOCAL UNICORN MOPO

Closest papers by meaning

• Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning (NeurIPS 2024 · similarity 0.91)
• Reasoning with Latent Diffusion in Offline Reinforcement Learning (ICLR 2024 · similarity 0.91)
• Entropy Regularized Task Representation Learning for Offline Meta-Reinforcement Learning (AAAI 2025 · similarity 0.91)
• Scrutinize What We Ignore: Reining In Task Representation Shift Of Context-Based Offline Meta Reinforcement Learning (ICLR 2025 · similarity 0.90)
• Generalizable Task Representation Learning for Offline Meta-Reinforcement Learning with Data Limitations (AAAI 2024 · similarity 0.89)
• MetaCARD: Meta-Reinforcement Learning with Task Uncertainty Feedback via Decoupled Context-Aware Reward and Dynamics Components (AAAI 2024 · similarity 0.89)