Research map / Vision-language and multimodal / Human and embodied modeling
The Curious Robot: Learning Visual Representations via Physical Interactions
Lerrel Pinto, Dhiraj Gandhi, Yuanfeng Han, Yong-Lae Park, Abhinav Gupta
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the grasp task (ablation) | robot multi-task ConvNet · ImageNet Household | accuracy: 0.309 -> 0.354 | Improved |
| add the grasp task (ablation) | robot multi-task ConvNet · UW RGB-D | accuracy: 0.632 -> 0.693 | Improved |
| add the grasp task (ablation) | robot multi-task ConvNet · Caltech-256 | accuracy: 0.263 -> 0.317 | Improved |
| add the push task (ablation) | robot multi-task ConvNet · UW RGB-D | accuracy: 0.710 -> 0.693 | Hurt |
| add the push task (ablation) | robot multi-task ConvNet · Caltech-256 | accuracy: 0.279 -> 0.317 | Improved |
| add the push task (ablation) | robot multi-task ConvNet · ImageNet Household | accuracy: 0.356 -> 0.354 | No clear effect |
| add the poke task (ablation) | robot multi-task ConvNet · ImageNet Household | accuracy: 0.342 -> 0.354 | Improved |
| add the poke task (ablation) | robot multi-task ConvNet · Caltech-256 | accuracy: 0.289 -> 0.317 | Improved |
| add the identity task (ablation) | robot multi-task ConvNet · ImageNet Household | accuracy: 0.324 -> 0.354 | Improved |
| add the identity task (ablation) | robot multi-task ConvNet · UW RGB-D | accuracy: 0.711 -> 0.693 | Hurt |
| train on robot interaction data instead of random init | AlexNet-style root network · ImageNet Household | accuracy: 0.250 -> 0.354 | Improved |
| train on robot interaction data instead of random init | AlexNet-style root network · UW RGBD | accuracy: 0.468 -> 0.693 | Improved |
Closest papers by meaning
- Time to augment self-supervised visual representation learning (ICLR 2023 · similarity 0.87)
- Self-supervised visual learning from interactions with objects (ECCV 2024 · similarity 0.87)
- Learning About Objects by Learning to Interact with Them (NeurIPS 2020 · similarity 0.87)
- Beyond the Doors of Perception: Vision Transformers Represent Relations Between Objects (NeurIPS 2024 · similarity 0.86)
- Towards flexible perception with visual memory (ICML 2025 · similarity 0.86)
- Object Concepts Emerge from Motion (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
