Research map / Reinforcement learning / Visual and representation RL

Representation Learning Enables Scalable Multitask Deep Reinforcement Learning

NeurIPS 2026 · Reinforcement learning › Visual and representation RL · approach: Learned state and action representations · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that a model-free agent (MR.Q) with predictive latent auxiliary objectives, without planning, matches or beats the world-model method Newt in multitask continuous control at lower compute cost.

Problem

Whether scalable multitask RL requires model-based planning, or whether the gains come mainly from predictive representation learning; prior MTRL evaluations often run to 100M+ steps, obscuring sample efficiency.

Approach

Argue that predictive, model-based representations used only as auxiliary losses (no planning) combined with high-capacity model-free actor-critic (MR.Q over TD3) is sufficient for scalable multitask RL; ablate against an encoder-free TD3 to isolate representation effects.

Key results

• On the 200-task setting, MR.Q reaches a normalized score of 0.11 at 2M steps versus 0.08 for Newt (+37% relative).
• In few-shot finetuning on 28 held-out tasks, MR.Q achieves 0.13 zero-shot vs 0.09 for Newt, and 0.62 vs 0.55 at 200k steps.
• Without predictive representations, the 95%-dimensionality of encoder-free TD3 drops from 89 to 21 on DMControl-Ext and from 66 to 15 on MuJoCo.

Surprising findings

• The encoder-free TD3 baseline consistently matches or outperforms Newt across all four evaluated domains.
• Predictive latent objectives help PPO at every network size, and reward-only scaling yields little benefit on HalfCheetah.

Limitations

• Evaluation focuses on continuous-control multitask benchmarks; generalization to long-horizon or other domains is unclear.
• Mechanisms behind the representation benefits are not fully understood and lack theory.
• Primary experiments use a 10M-step budget, unlike much prior large-scale MTRL work.

Experimental setup

Datasets: MMBench MuJoCo DMControl DMControl-Ext MetaWorld ManiSkill3 Pygame Box2D RoboDesk Atari OGBench

Benchmarks: MMBench 200-task multitask suite 28 held-out tasks for few-shot finetuning MMBench visual variant

Metrics: normalized score SRank 95% effective dimensionality dormant neuron fraction value loss wall-clock time

Baselines: Newt TD3 PPO SPR BBF

Builds on: MR.Q TD3 TD-MPC2 MMBench

Closest papers by meaning

• Rethinking Model-based, Policy-based, and Value-based Reinforcement Learning via the Lens of Representation Complexity (NeurIPS 2024 · similarity 0.89)
• Bigger, Regularized, Categorical: High-Capacity Value Functions are Efficient Multi-Task Learners (NeurIPS 2025 · similarity 0.89)
• Mastering Massive Multi-Task Reinforcement Learning via Mixture-of-Expert Decision Transformer (ICML 2025 · similarity 0.89)
• Towards General-Purpose Model-Free Reinforcement Learning (ICLR 2025 · similarity 0.88)
• AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers (NeurIPS 2024 · similarity 0.88)
• Overcoming Slow Decision Frequencies in Continuous Control: Model-Based Sequence Reinforcement Learning for Model-Free Control (ICLR 2025 · similarity 0.88)