Research map / Reinforcement learning / Visual and representation RL
Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?
Xiang Li, Jinghuan Shang, Srijan Das, Michael S Ryoo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Balance the two constituent prediction losses instead of averaging their concatenated outputs (ablation) | two-loss RL context prediction methods · DMControl | score: Extract-AR: ↑75.7, ↓-10.6, ↑7.1, ↑98.5, ↑41.8, ↑75.4; Guess-AF: ↑588.6, ↑395.4, ↑190.3, ↓-37.1, ↑79.1, ↑12.3; Predict-FR | Mixed |
| Use a ResNet encoder backbone instead of the baseline CNN (ablation) | RL methods evaluated in the backbone ablation · DMControl | score | Improved |
| Tune random-crop augmentation strength rather than using an arbitrary magnitude (ablation) | SAC-based pixel RL · DMControl cheetah, run | score | Mixed |
| Add a self-supervised loss alongside image augmentation | SAC-Aug(100) · DMControl | score | Mixed |
| Use SimSiam or ShuffleCLS self-supervision | SAC with image augmentation · DMControl | score | Hurt |
| Search combinations of multiple SSL losses and augmentation magnitudes | SAC · DMControl | relative score | Mixed |
| Add random-crop augmentation to Efficient Rainbow (ablation) | Efficient Rainbow · Atari | relative score | Hurt |
| Use DrQ-style image augmentation with Efficient Rainbow (ablation) | Efficient Rainbow · Atari | relative score | Hurt |
| Add autoencoder reconstruction loss | Efficient Rainbow · Atari | relative score | Improved |
| Use image augmentation (ablation) | SAC · uArm reacher | score | Improved |
| Remove image augmentation (ablation) | SAC · uArm reacher | score | Hurt |
| Balance losses when manually combining two context-prediction objectives (ablation) | Extract-AR, Guess-AF, and Predict-FR · DMControl | score | Mixed |
Closest papers by meaning
- Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data (ICLR 2024 · similarity 0.91)
- Accelerating Goal-Conditioned Reinforcement Learning Algorithms and Research (ICLR 2025 · similarity 0.89)
- Self-Supervised Reinforcement Learning that Transfers using Random Features (NeurIPS 2023 · similarity 0.89)
- 1000 Layer Networks for Self-Supervised RL: Scaling Depth Can Enable New Goal-Reaching Capabilities (NeurIPS 2025 · similarity 0.86)
- Foundation Policies with Hilbert Representations (ICML 2024 · similarity 0.86)
- Reinforcement Learning Guided Semi-Supervised Learning (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
