Research map / Reinforcement learning / Visual and representation RL
DMC-VB: A Benchmark for Representation Learning for Control with Visual Distractors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DMC-VB, an offline visual-control dataset built on DeepMind Control with visual distractors, multiple policy skill levels, paired states and pixels, and hidden-goal navigation, plus three benchmarks for pretrained representation learning.
Problem
Offline visual control agents are brittle to control-irrelevant visual variations (background, camera, agent color), and existing offline datasets lack the combination of task diversity, distractors, demonstration quality diversity, paired states/pixels, scale, and hidden goals needed to evaluate re
Approach
A dataset satisfying six desiderata (task diversity, distractor diversity, mixed policy skill levels, paired state and pixel observations, large size, hidden goals) with three benchmarks that test whether pretrained visual encoders (ID, LFD, AE, DINO) help BC/TD3-BC policy learning.
Key results
- NULL + BC is the best overall method; pretrained representations (ID, LFD, AE, DINO) give no advantage on locomotion or navigation with BC on expert/medium data.
- On walker-expert with dynamic distractors, NULL + BC reaches 913 ± 10 reward versus 33 ± 2 for LFD + BC, while BC (state) reaches 960 ± 2 without distractors.
- Pretraining on mixed-quality data improves BC with only 1% expert data (20 trajectories), and pretraining on stochastic hidden-goal data helps few-shot policy learning on fixed-goal tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pretrain the encoder with inverse dynamics before BC instead of training end-to-end with BC | BC on DMC-VB locomotion tasks · walker expert, no distractor | reward: 948 ± 4 -> 947 ± 3 | No clear effect |
| Pretrain the encoder with latent forward dynamics before BC instead of training end-to-end with BC | BC on DMC-VB locomotion tasks · walker expert, dynamic distractor | reward: 913 ± 10 -> 33 ± 2 | Hurt |
| Replace BC with TD3-BC | pixel-based policy learning on DMC-VB · walker expert, no distractor | reward: 948 ± 4 -> 950 ± 68 | No clear effect |
| Pretrain representations on mixed-quality data, then train BC on 1% expert data | few-shot policy learning on DMC-VB locomotion tasks · locomotion tasks with 1% expert data | proportion of reward relative to BC on full expert data | Helped |
| Pretrain representations on mixed-quality data, then train BC on 1% expert data | few-shot policy learning on DMC-VB locomotion tasks · locomotion tasks with 1% expert data | proportion of reward relative to BC on full expert data | No clear effect |
| Pretrain representations on stochastic hidden-goal tasks before learning a policy for a fixed hidden goal | few-shot ant-maze policy learning · ant maze tasks with fixed hidden goals | reward | Helped |
| Use BC rather than inverse-dynamics pretraining for the stochastic hidden-goal representation | few-shot ant-maze policy learning · ant maze tasks with fixed hidden goals | reward | Helped |
| Add frame stacking of three consecutive observations (ablation) | pixel-based policy learning on DMC-VB · DMC-VB policy learning | reward | Helped |
Surprising findings
- Pretrained visual representations (ID, LFD, AE, DINO) do not improve BC policy learning on DMC-VB, contrary to expectations from prior work.
- BC pretraining outperforms ID pretraining on the hidden-goal benchmark, contrary to prior findings favoring ID.
Limitations
- Environments are limited to simple locomotion and navigation without sparse rewards, multi-agent settings, manipulation, or stochastic dynamics.
- Visual distractors are synthetic, which may limit transfer of findings to real-world robotics.
- Authors do not explain why pretrained representations fail to help policy learning.
Experimental setup
Datasets: DMC-VB locomotion datasets (walker, cheetah, humanoid) DMC-VB ant maze datasets Kubric video backgrounds
Benchmarks: DMC-VB locomotion tasks DMC-VB ant maze tasks
Metrics: average reward fraction of successes average velocity least-squares reconstruction error
Baselines: NULL + BC NULL + TD3-BC ID + BC LFD + BC AE + BC DINO + BC BC (state) State + BC
Builds on: TD3-BC MPO Distracting Control Suite D4RL VD4RL
Closest papers by meaning
- The Unsurprising Effectiveness of Pre-Trained Vision Models for Control (ICML 2022 · similarity 0.91)
- DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization (ICLR 2024 · similarity 0.89)
- The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learning (NeurIPS 2024 · similarity 0.89)
- Sliding Puzzles Gym: A Scalable Benchmark for State Representation in Visual Reinforcement Learning (ICML 2025 · similarity 0.88)
- VLMs-Guided Representation Distillation for Efficient Vision-Based Reinforcement Learning (CVPR 2025 · similarity 0.88)
- DMR: Decomposed Multi-Modality Representations for Frames and Events Fusion in Visual Reinforcement Learning (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.