Research map / Vision-language and multimodal / Human and embodied modeling
Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| scale from ViT-B to ViT-L (ablation) | MAE on Ego4D+MNI · CortexBench mean success | mean success: 66.2 -> 68.7 | Helped |
| add indoor navigation data (Ego4D+N) to Ego4D (ablation) | MAE ViT-L · CortexBench mean success | mean success: 63.5 -> 67.1 | Helped |
| add manipulation data (Ego4D+M) to Ego4D (ablation) | MAE ViT-L · CortexBench mean success | mean success: 63.5 -> 64.4 | Helped |
| add ~800K more frames (Ego4D+MN vs Ego4D+N) (ablation) | MAE ViT-L · CortexBench mean success | mean success: 67.1 -> 67.2 | No clear effect |
| add ImageNet to pre-training data (Ego4D+MNI vs MN) (ablation) | MAE ViT-L · CortexBench mean success | mean success: 67.2 -> 68.7 | Helped |
| scale data and model (VC-1 vs per-task best PVR) | VC-1 · CortexBench per-benchmark | success rate | Mixed |
| end-to-end fine-tune VC-1 in few-shot IL domain (ablation) | VC-1 · Adroit | success rate: 59.3 -> 15.9 | Hurt |
| end-to-end fine-tune VC-1 in large-scale RL/IL domain (ablation) | VC-1 · ImageNav | success rate: 70.3 -> 81.6 | Helped |
| end-to-end fine-tune VC-1 (ablation) | VC-1 · ObjectNav | success rate: 60.3 -> 67.7 | Helped |
| MAE adaptation on task-specific data (ablation) | VC-1 · DMControl | success rate: 66.9 -> 80.9 | Helped |
| MAE adaptation on task-specific data (ablation) | VC-1 · Adroit | success rate: 59.3 -> 72.0 | Helped |
| MAE adaptation in large-scale RL/IL domains (ablation) | VC-1 · ObjectNav | success rate: 60.3 -> 57.4 | Hurt |
Closest papers by meaning
- Selective Visual Representations Improve Convergence and Generalization for Embodied AI (ICLR 2024 · similarity 0.90)
- SPA: 3D Spatial-Awareness Enables Effective Embodied Representation (ICLR 2025 · similarity 0.89)
- Pre-trained Text-to-Image Diffusion Models Are Versatile Representation Learners for Control (NeurIPS 2024 · similarity 0.88)
- DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Control (NeurIPS 2024 · similarity 0.88)
- EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI (CVPR 2024 · similarity 0.88)
- Robots Pre-train Robots: Manipulation-Centric Robotic Representation from Large-Scale Robot Datasets (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.