Research map / Vision-language and multimodal / Human and embodied modeling
The Unsurprising Effectiveness of Pre-Trained Vision Models for Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune the PVR instead of freezing it (ablation) | Superv. (RN50) · Adroit Domain | normalized performance: 72.3 \pm 4.0 -> 68.2 \pm 8.4 | Hurt |
| Fine-tune the PVR instead of freezing it (ablation) | Superv. (RN50) · DMC Domain | normalized performance: 61.0 \pm 1.5 -> 55.4 \pm 1.7 | Hurt |
| Fine-tune the PVR instead of freezing it (ablation) | MAE (ViT-B) · Adroit Domain | normalized performance: 74.2 \pm 6.2 -> 36.7 \pm 4.2 | Hurt |
| Fine-tune the PVR instead of freezing it (ablation) | MAE (ViT-B) · DMC Domain | normalized performance: 64.1 \pm 2.1 -> 17.2 \pm 8.2 | Hurt |
| Fine-tune the PVR instead of freezing it (ablation) | MoCo (RN50, L5) · Adroit Domain | normalized performance: 73.1 \pm 2.5 -> 59.3 \pm 7.9 | Hurt |
| Fine-tune the PVR instead of freezing it (ablation) | MoCo (RN50, L5) · DMC Domain | normalized performance: 69.6 \pm 1.0 -> 58.2 \pm 2.2 | Hurt |
| Fine-tune the PVR instead of freezing it (ablation) | MoCo (RN50, L3) · Adroit Domain | normalized performance: 77.0 \pm 2.0 -> 64.7 \pm 8.6 | Hurt |
| Fine-tune the PVR instead of freezing it (ablation) | MoCo (RN50, L3) · DMC Domain | normalized performance: 77.8 \pm 1.0 -> 68.6 \pm 2.1 | Hurt |
| Use intermediate-layer features rather than final-layer features | Pre-trained vision representations for continuous control · MuJoCo / DMC | task performance | Helped |
| Use final-layer features rather than intermediate-layer features | Pre-trained vision representations for navigation · Habitat | task performance | Helped |
| Use a full hierarchy of features | Pre-trained vision representations for control · 4 domains, 17 environments/tasks | task performance | Helped |
| Use layer-1 or layer-2 features (ablation) | Pre-trained vision representations for control · Control tasks | task performance | Hurt |
Closest papers by meaning
- DMC-VB: A Benchmark for Representation Learning for Control with Visual Distractors (NeurIPS 2024 · similarity 0.91)
- Pre-trained Text-to-Image Diffusion Models Are Versatile Representation Learners for Control (NeurIPS 2024 · similarity 0.89)
- The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learning (NeurIPS 2024 · similarity 0.89)
- Control-oriented Clustering of Visual Latent Representation (ICLR 2025 · similarity 0.87)
- Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers (NeurIPS 2024 · similarity 0.87)
- From Representational Complementarity to Dual Systems: Synergizing VLM and Vision-Only Backbones for End-to-End Driving (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.