Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Distilling Internet-Scale Vision-Language Models into Embodied Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| retrain agent on zero-shot VLM relabels | original agent · Lift object names | task success (%): 7.1 -> 64.4 | Helped |
| retrain on VLM name relabels | original agent · Lift names (colored objects) | task success (%): 19.6 -> 83.0 | Helped |
| retrain on VLM color relabels using a color prompt | original agent · Lift color | task success (%): 18.4 -> 80.6 | Helped |
| add 6 in-context examples (Fewshot-1) to prompt | zero-shot relabeling · Lift food or toy | task success (%): 60.8 -> 73.3 | Helped |
| add 12 in-context examples (Fewshot-2) | zero-shot relabeling · Lift food or toy | task success (%): 60.8 -> 82.8 | Helped |
| add more in-context examples beyond Fewshot-2 | Fewshot-2 relabeling · Lift food or toy | task success (%): 82.8 -> 84.7 | No clear effect |
| use arbitrary instead of aligned preference categories | Fewshot-5 relabeling · John Doe preferences | task success (%): 76.7 -> 64.2 | Hurt |
| use few-shot instead of zero-shot prompt | relabeling · Lift object names | task success (%): 64.4 -> 52.9 | Hurt |
| few-shot prompt raises label accuracy | relabeling · Lift object names | labels with canonical name (%): 54.7 -> 77.2 | Helped |
| filter out least-confident 50% of labels | Fewshot relabeling · Lift object names | task success (%): 52.9 -> 78.6 | Helped |
| filter out least-confident 50% of labels | Zeroshot relabeling · Lift object names | task success (%): 64.4 -> 66.1 | No clear effect |
| recolor toys at test time | VLM preference relabeling · Aligned preferences | VLM accuracy: 0.91 -> 0.92 | No clear effect |
Closest papers by meaning
- VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought (NeurIPS 2024 · similarity 0.90)
- EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents (ICML 2025 · similarity 0.89)
- Ground-Compose-Reinforce: Grounding Language in Agentic Behaviours using Limited Data (NeurIPS 2025 · similarity 0.89)
- UP-VLA: A Unified Understanding and Prediction Model for Embodied Agent (ICML 2025 · similarity 0.88)
- GenRL: Multimodal-foundation world models for generalization in embodied agents (NeurIPS 2024 · similarity 0.88)
- Embodied Multi-Modal Agent trained by an LLM from a Parallel TextWorld (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.