Research map / Vision-language and multimodal / Robot action and navigation
FILM: Following Instructions in Language with Modular Methods
So Yeon Min, Devendra Singh Chaplot, Pradeep Kumar Ravikumar, Yonatan Bisk, Ruslan Salakhutdinov
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the semantic search policy (ablation) | FILM · ALFRED Tests Unseen, high-level instruction only | SR: 22.56 -> 24.46 | Improved |
| Add the semantic search policy (ablation) | FILM · ALFRED Tests Seen, high-level instruction only | SR: 24.72 -> 25.77 | Improved |
| Add low-level instructions as additional input (ablation) | FILM · ALFRED Tests Unseen | SR: 24.46 -> 26.49 | Improved |
| Use a direct instruction-to-subtask BART language module instead of the task-type templates (ablation) | FILM · ALFRED Valid Unseen | SR: 20.10% -> 18.03% | Hurt |
| Use ground-truth depth instead of predicted depth (ablation) | FILM · ALFRED development scenes, seen | SR: +1.48% | Improved |
| Use ground-truth depth instead of predicted depth (ablation) | FILM · ALFRED development scenes, unseen | SR: +10.64% | Improved |
| Use ground-truth segmentation instead of predicted segmentation (ablation) | FILM · ALFRED development scenes, seen and unseen | SR: 9.26% / 9.26% | Improved |
| Use ground-truth language instead of predicted language (ablation) | FILM · ALFRED development scenes, seen and unseen | SR: 7.43% / 4.22% | Improved |
Later papers that compared with this method
7 later accepted papers report FILM as a baseline in their results tables; 7 of them report a better number on most of the results they share with it.
- STEP: Spatial Thinking and Egocentric Pointing for Embodied Instruction Following (ECCV 2026 · better on 3 of 3 shared results)
- Every SAM Drop Counts: Embracing Semantic Priors for Multi-Modality Image Fusion and Beyond (CVPR 2025 · better on 1 of 1 shared result)
- On-the-Fly Modulation for Balanced Multimodal Learning (TPAMI 2024 · better on 2 of 2 shared results)
- 3D-Aware Visual Question Answering about Parts, Poses and Occlusions (NeurIPS 2023 · better on 5 of 5 shared results)
- Super-CLEVR: A Virtual Benchmark To Diagnose Domain Robustness in Visual Reasoning (CVPR 2023 · better on 3 of 4 shared results)
- PMR: Prototypical Modal Rebalance for Multimodal Learning (CVPR 2023 · better on 1 of 1 shared result)
- DecAF: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge Bases (ICLR 2023 · better on 1 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- ThinkBot: Embodied Instruction Following with Thought Chain Reasoning (ICLR 2025 · similarity 0.89)
- EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents (ICML 2025 · similarity 0.87)
- GROOT-2: Weakly Supervised Multimodal Instruction Following Agents (ICLR 2025 · similarity 0.86)
- Policy Improvement using Language Feedback Models (NeurIPS 2024 · similarity 0.86)
- Do LLMs ``know'' internally when they follow instructions? (ICLR 2025 · similarity 0.86)
- Pre-trained Text-to-Image Diffusion Models Are Versatile Representation Learners for Control (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
