Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Demo2Tutorial: From Human Experience to Multimodal Software Tutorials
Zechen Bai, Zhiheng Chen, Yiqi Lin, Kevin Qinghong Lin, Difei Gao, Xiangwu Guo, Xin Wang, Mike Zheng Shou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add adaptive visual highlighting (ablation) | Demo2Tutorial · TutorialBench subset | annotation quality: 3.8 -> 85.2 | Improved |
| add adaptive visual highlighting (ablation) | Demo2Tutorial · TutorialBench subset | overall score: 66.2 -> 90.2 | Improved |
| add task hierarchy (ablation) | Demo2Tutorial · TutorialBench subset | conciseness: 64.8 -> 84.8 | Improved |
| add task hierarchy (ablation) | Demo2Tutorial · TutorialBench subset | overall score: 84.5 -> 90.2 | Improved |
| add actor-critic iterative refinement (ablation) | Demo2Tutorial · TutorialBench subset | overall score: 87.7 -> 90.2 | Improved |
| add score-based key-frame selection (ablation) | Demo2Tutorial · TutorialBench subset | overall score: 87.5 -> 90.2 | Improved |
| use action-aligned key-frames and agentic components over vanilla multi-agent | Demo2Tutorial · TutorialBench | overall score: 70.3 -> 86.2 | Improved |
| add visual input to text-only generation | Text-based Generation vs Vision-based · TutorialBench | overall score: 59.2 -> 64.3 | Improved |
| use GPT-4o instead of Qwen-VL-32B backbone (ablation) | Demo2Tutorial · TutorialBench | overall score: 79.5 -> 86.2 | Improved |
| add tutorial to GUI agent planner | GPT-5 in Agent-S3 · OSWorld Chrome | success rate: 52.9 -> 70.6 | Improved |
| add tutorial to GUI agent planner | o4-mini in Agent-S3 · OSWorld VLC | success rate: 53.4 -> 56.1 | Improved |
| add tutorial to GUI agent planner | o4-mini in Agent-S3 · OSWorld Chrome | success rate: 47.1 -> 58.8 | Improved |
Closest papers by meaning
- VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought (NeurIPS 2024 · similarity 0.88)
- Synatra: Turning Indirect Knowledge into Direct Demonstrations for Digital Agents at Scale (NeurIPS 2024 · similarity 0.87)
- SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learning (NeurIPS 2026 · similarity 0.87)
- GameDevBench: Evaluating Agentic Capabilities Through Game Development (ICML 2026 · similarity 0.86)
- XSkill: Continual Learning from Experience and Skills in Multimodal Agents (ICML 2026 · similarity 0.86)
- DeepEyesV2: Toward Agentic Multimodal Model (ICLR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
