Research map / Vision-language and multimodal / Human and embodied modeling
ProTo: Program-Guided Transformer for Program-Guided Tasks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add structure guidance (transition mask) (ablation) | ProTo · GQA val | accuracy: 55.16 -> 64.47 | Helped |
| add semantic guidance (routine embeddings) (ablation) | ProTo · GQA val | accuracy: 33.82 -> 64.47 | Helped |
| use masked-attention transformer instead of GAT encoding (ablation) | ProTo · GQA val | accuracy: 58.58 -> 64.47 | Helped |
| use dense supervision instead of partial supervision (ablation) | ProTo · GQA val | accuracy: 60.28 -> 64.47 | Helped |
| use ProTo instead of MMN on human-written programs | ProTo · GQA human programs | accuracy: 59.47 -> 70.33 | Helped |
| use ProTo instead of MMN on unseen programs | ProTo · GQA unseen programs | accuracy: 61.88 -> 65.34 | Helped |
| use ProTo instead of MMN with restricted data | ProTo · GQA restricted data | accuracy: 52.39 -> 58.31 | Helped |
| use ProTo instead of PGA under RL target | ProTo · 2D Minecraft standard | task completion rate: 94.2 -> 97.3 | Helped |
| use ProTo instead of PGA with dense supervision on longer programs | ProTo · 2D Minecraft longer programs | task completion rate: 92.1 -> 94.4 | Helped |
| use program structure guidance instead of vanilla end-to-end transformer | ProTo · 2D Minecraft standard (RL) | task completion rate: 50.1 -> 97.3 | Helped |
Closest papers by meaning
- Synthesizing Visual Concepts as Vision-Language Programs (CVPR 2026 · similarity 0.86)
- Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks (NeurIPS 2024 · similarity 0.85)
- OmniJARVIS: Unified Vision-Language-Action Tokenization Enables Open-World Instruction Following Agents (NeurIPS 2024 · similarity 0.85)
- Lifting Traces to Logic: Programmatic Skill Induction with Neuro-Symbolic Learning for Long-Horizon Agentic Tasks (ICML 2026 · similarity 0.85)
- GENOME: Generative Neuro-Symbolic Visual Reasoning by Growing and Reusing Modules (ICLR 2024 · similarity 0.85)
- Grokking of Implicit Reasoning in Transformers: A Mechanistic Journey to the Edge of Generalization (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.