Research map / Reinforcement learning / Visual and representation RL
Reinforcement Learning with Neural Radiance Fields
Danny Driess, Ingmar Schubert, Pete Florence, Yunzhu Li, Marc Toussaint
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use compositional NeRF supervision instead of other learned representations or expert keypoints | NeRF-RL · Mug on Hook, Planar Pushing, Door Opening | success rate | Improved |
| Use a compositional NeRF decoder rather than a 2D CNN decoder | NeRF-RL state representation · Mug on Hook, Planar Pushing, Door Opening | success rate | Improved |
| Use the 3D neural field encoder instead of the 2D CNN image encoder (ablation) | NeRF-RL with compositional NeRF decoder · Door Opening | success rate | Improved |
| Use the 2D CNN image encoder instead of the 3D neural field encoder (ablation) | NeRF-RL with compositional NeRF decoder · Mug on Hook, Planar Pushing | success rate | No clear effect |
| Replace compositional NeRF supervision with multi-view contrastive learning | NeRF-RL representation learning · Mug on Hook, Planar Pushing, Door Opening | success rate | Hurt |
| Use CURL with image-cropping augmentation instead of NeRF supervision | RL state representation · Door Opening | success rate | Improved |
| Use CURL with image-cropping augmentation instead of NeRF supervision | RL state representation · Planar Pushing | success rate | Hurt |
| Use CURL with image-cropping augmentation instead of NeRF supervision | RL state representation · Mug on Hook | success rate: 59% | Mixed |
| Use a global NeRF decoder instead of a compositional NeRF decoder (ablation) | NeRF-RL with image encoder · Planar Pushing | success rate | Hurt |
| Use a global NeRF decoder instead of a compositional NeRF decoder (ablation) | NeRF-RL with image encoder · Door Opening | success rate | Hurt |
| Use compositionality in a convolutional autoencoder without NeRF supervision (ablation) | Convolutional autoencoder · Mug on Hook, Planar Pushing, Door Opening | success rate | Hurt |
| Train the NeRF-RL representation for more epochs to improve reconstruction quality (ablation) | NeRF-RL compositional + image · Mug on Hook | success rate | Improved |
Closest papers by meaning
- SNeRL: Semantic-aware Neural Radiance Fields for Reinforcement Learning (ICML 2023 · similarity 0.90)
- RRL: Resnet as representation for Reinforcement Learning (ICML 2021 · similarity 0.87)
- Entity-Centric Reinforcement Learning for Object Manipulation from Pixels (ICLR 2024 · similarity 0.87)
- Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data (ICLR 2024 · similarity 0.85)
- Foundation Policies with Hilbert Representations (ICML 2024 · similarity 0.85)
- Robot-R1: Reinforcement Learning for Enhanced Embodied Reasoning in Robotics (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
