Research map / Reinforcement learning / Visual and representation RL

MetaCARD: Meta-Reinforcement Learning with Task Uncertainty Feedback via Decoupled Context-Aware Reward and Dynamics Components

AAAI 2024 · Reinforcement learning › Visual and representation RL · approach: Task-inference meta-RL · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MetaCARD, a context-based meta-RL method that linearly decomposes rewards into context-aware dynamics features and reward weights, and feeds task-uncertainty (reward and CSF loss) feedback into context encoder training.

Problem

Context-based meta-RL suffers from MDP ambiguity: biased trajectories cause the task inference module to form spurious correlations, and entangled latent contexts fail to separate dynamics changes from reward changes, hurting adaptation to new tasks.

Approach

Decompose the reward into context-aware features (dynamics, conditioned on latent context) and context-aware reward weights, build successor-feature-style critics on these components within SAC, and use the reward and critic losses as uncertainty feedback to regularize the context encoder.

Key results

• On Meta-World ML1, MetaCARD achieves the highest test success rate in most environments, e.g., Door-Unlock 99.93% vs PEARL 90.40% and variBAD 95.30%.
• MetaCARD outperforms baselines in Ant-Goal and Half-Cheetah-Mass/Walker-Rand-Params, and shows more efficient adaptation on tasks with both reward and dynamics changes (Half-Cheetah-Dir-Mass, Half-Cheetah-Dir-Damp).
• Ablation shows removing the L_CARD uncertainty feedback reduces adaptation efficiency and final performance on Plate-Slide and Ant-Goal, though MetaCARD without it still beats PEARL in most environments.

Measured effects reported in the paper

Surprising findings

• Without the uncertainty feedback term, MetaCARD still outperforms PEARL in most environments, suggesting the decomposed policy learning alone provides gains.

Limitations

• Gains on some Meta-World environments are not uniform; PEARL matches or beats MetaCARD on Faucet-Close and Handle-Press asymptotically.
• Evaluation limited to ML1 Meta-World and classical MuJoCo; no ML45/MT settings reported in the main text.
• Hyperparameter beta fixed to 0.01 across experiments with limited sensitivity analysis.

Experimental setup

Datasets: Meta-World ML1 tasks MuJoCo Half-Cheetah-Dir MuJoCo Half-Cheetah-Mass MuJoCo Half-Cheetah-Dir-Mass MuJoCo Half-Cheetah-Dir-Damp MuJoCo Ant-Goal MuJoCo Walker-Rand-Params

Benchmarks: Meta-World ML1 MuJoCo Half-Cheetah-Dir MuJoCo Ant-Goal MuJoCo Half-Cheetah-Mass MuJoCo Walker-Rand-Params MuJoCo Half-Cheetah-Dir-Mass MuJoCo Half-Cheetah-Dir-Damp

Metrics: average success rate average return

Baselines: PEARL SFC RL2 variBAD CCM

Builds on: PEARL successor features variBAD soft actor-critic CCM

Closest papers by meaning

• Decoupling Meta-Reinforcement Learning with Gaussian Task Contexts and Skills (AAAI 2024 · similarity 0.92)
• AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers (NeurIPS 2024 · similarity 0.91)
• Meta-Reinforcement Learning with Universal Policy Adaptation: Provable Near-Optimality under All-task Optimum Comparator (NeurIPS 2024 · similarity 0.91)
• Meta-Reinforcement Learning via Exploratory Task Clustering (AAAI 2024 · similarity 0.91)
• Scrutinize What We Ignore: Reining In Task Representation Shift Of Context-Based Offline Meta Reinforcement Learning (ICLR 2025 · similarity 0.90)
• Meta-World+: An Improved, Standardized, RL Benchmark (NeurIPS 2025 · similarity 0.90)