Research map / Vision-language and multimodal / Robot action and navigation
GEM: Generative Supervision Helps Embodied Intelligence
Ruowen Zhao, Bangguo Li, Zuyan Liu, Yinan Liang, junliang ye, FANGFU LIU, Diankun Wu, Zhengyi Wang, Xumin Yu, Yongming Rao, Han Hu, Jun Zhu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add depth generative supervision during pretraining (ablation) | Qwen3-VL-8B SFT · VSI-Bench | accuracy: 68.6 -> 70.6 | Improved |
| add depth generative supervision during pretraining (ablation) | Qwen3-VL-2B SFT · VSI-Bench | accuracy: 60.0 -> 62.8 | Improved |
| add depth generative supervision during pretraining (ablation) | Qwen3-VL-8B SFT · MMSI-Bench | accuracy: 32.8 -> 35.3 | Improved |
| add depth generative supervision during pretraining (ablation) | Qwen3-VL-2B SFT · RoboSpatial | accuracy: 44.6 -> 47.4 | Improved |
| add depth generative supervision during pretraining (ablation) | Qwen3-VL-8B SFT · RefSpatial | accuracy: 45.8 -> 44.4 | Hurt |
| add depth generative supervision during pretraining (ablation) | Qwen3-VL-2B SFT · Where2Place | accuracy: 53.0 -> 53.0 | No clear effect |
| add GEM-4M SFT plus depth supervision over base model | Qwen3-VL-8B · VSI-Bench | accuracy: 57.9 -> 70.6 | Improved |
| use depth instead of RGB reconstruction as generative target (ablation) | GEM · VSI-Bench | accuracy: 60.0 -> 63.0 | Improved |
| use progressive three-stage training instead of direct end-to-end (ablation) | GEM · VSI-Bench | accuracy: 57.6 -> 63.0 | Improved |
| add depth generative supervision to VLA (ablation) | Qwen3-VL-2B VLA · LIBERO | average success rate: 94.9 -> 96.1 | Improved |
| add depth generative supervision to VLA (ablation) | Qwen3-VL-2B VLA · Simpler WidowX | average success rate: 61.5% -> 67.0% | Improved |
| keep depth head active during VLA finetuning (ablation) | GEM-VLA · real-world UR5 tasks | success rate | Improved |
Closest papers by meaning
- EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents (ICML 2025 · similarity 0.91)
- EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI (CVPR 2024 · similarity 0.91)
- ManiTaskGen: A Comprehensive Task Generator for Benchmarking and Improving Vision-Language Agents on Embodied Decision-Making (NeurIPS 2026 · similarity 0.90)
- UP-VLA: A Unified Understanding and Prediction Model for Embodied Agent (ICML 2025 · similarity 0.90)
- PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding (ICLR 2025 · similarity 0.90)
- 3D-VLA: A 3D Vision-Language-Action Generative World Model (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
