Research map / Vision-language and multimodal / Multimodal LLMs
STEP: Spatial Thinking and Egocentric Pointing for Embodied Instruction Following
Hanxuan Li, Bin Fu, Zeyuan Lin, Ruiping Wang, Xilin CHEN
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add RGB action data fine-tuning (ablation) | Qwen2.5-VL-7B · ALFRED Val Unseen | SR: 7.87 -> 13.92 | Improved |
| add Set-of-Mark traversable-region markers (ablation) | Qwen2.5-VL-7B · ALFRED Val Unseen | SR: 13.92 -> 18.43 | Improved |
| add BEV map modality via simple concatenation (ablation) | Qwen2.5-VL-7B · ALFRED Val Unseen | SR: 18.43 -> 19.22 | Improved |
| add map modality (ablation) | Qwen2.5-VL-7B · ALFRED Val Unseen | GC: 30.61 -> 29.85 | No clear effect |
| add STEP-CoT reasoning traces (ablation) | STEP · ALFRED Val Unseen | SR: 19.22 -> 23.72 | Improved |
| add STEP-CoT reasoning traces (ablation) | STEP · ALFRED Val Seen | SR: 21.12 -> 28.40 | Improved |
| add primitive-skill data (ablation) | STEP · ALFRED Val Unseen | GC: 33.33 -> 39.36 | Improved |
| add primitive-skill data (ablation) | STEP · ALFRED Val Seen | GC: 40.35 -> 39.69 | Hurt |
| add primitive-skill data (ablation) | STEP · ALFRED Val Unseen | SR: 23.72 -> 26.77 | Improved |
| provide oracle segmentation (ablation) | STEP-7B · ALFRED Val Unseen | SR: 26.77 -> 41.57 | Improved |
| provide oracle goal decomposition on top of oracle segmentation (ablation) | STEP-7B · ALFRED Val Unseen | SR: 41.57 -> 55.69 | Improved |
| fine-tune on STEP-CoT vs zero-shot GPT-4o | STEP-7B · ALFRED Val Unseen | SR: 16.00 -> 26.77 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- FILM — FILM: Following Instructions in Language with Modular Methods (ICLR 2022) · better on 3 of 3 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- ThinkBot: Embodied Instruction Following with Thought Chain Reasoning (ICLR 2025 · similarity 0.90)
- EgoProx: Evaluating MLLMs on Egocentric 3D Proximity Reasoning Across a Cognitive Hierarchy (CVPR 2026 · similarity 0.88)
- Thinking with Imagination: Agentic Visual Spatial Reasoning with World Simulators (NeurIPS 2026 · similarity 0.88)
- Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations (ICLR 2026 · similarity 0.88)
- SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning (NeurIPS 2026 · similarity 0.88)
- EgoMind: Activating Spatial Cognition through Linguistic Reasoning in MLLMs (CVPR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
