Research map / Vision-language and multimodal / Robot action and navigation
History Aware Multimodal Transformer for Vision-and-Language Navigation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace recurrent state with temporal-only transformer history (ablation) | HAMT (ResNet152) · R2R val unseen | SPL: 47.0±0.5 -> 48.0±0.4 | Helped |
| add panoramic views hierarchically on top of temporal-only history (ablation) | HAMT (ResNet152) · R2R val unseen | SPL: 48.0±0.4 -> 48.7±0.4 | Helped |
| add proxy-task training before RL+IL (ablation) | HAMT (ResNet152) · R2R val unseen | SPL: 48.7±0.4 -> 57.5±0.5 | Helped |
| swap ResNet152 features for ViT features (ablation) | HAMT · R2R val unseen | SPL: 57.5±0.5 -> 58.8±0.0 | Helped |
| train ViT end-to-end (ablation) | HAMT · R2R val unseen | SPL: 58.8±0.0 -> 60.9±0.7 | Helped |
| add SAP(R) proxy task (ablation) | HAMT (ViT frozen) · R2R val unseen | SPL: 57.7±1.0 -> 58.1±0.4 | Helped |
| add SPREL proxy task (ablation) | HAMT (ViT frozen) · R2R val unseen | SPL: 58.1±0.4 -> 58.8±0.0 | Helped |
| add IL fine-tuning to proxy-trained model (ablation) | HAMT · R2R val unseen | SPL: 48.9 -> 54.7±0.3 | Helped |
| use RL instead of IL for fine-tuning (ablation) | HAMT · R2R val unseen | SPL: 54.7±0.3 -> 57.5±1.1 | Helped |
| mix RL with IL (ablation) | HAMT · R2R val unseen | SPL: 57.5±1.1 -> 60.9±0.7 | Helped |
| HAMT with long history vs recurrent RecBERT on R2R-Back | HAMT · R2R-Back val unseen | SPL: 37.7 -> 53.1 | Helped |
| HAMT vs RecBERT on long-horizon R4R | HAMT · R4R val unseen | nDTW: 45.1 -> 50.3 | Helped |
Closest papers by meaning
- HOP: History-and-Order Aware Pre-Training for Vision-and-Language Navigation (CVPR 2022 · similarity 0.89)
- UNeMo: Collaborative Visual-Language Reasoning and Navigation via a Multimodal World Model (AAAI 2026 · similarity 0.89)
- Vision-Dialog Navigation by Exploring Cross-Modal Memory (CVPR 2020 · similarity 0.89)
- NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models (AAAI 2024 · similarity 0.88)
- Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Method (CVPR 2025 · similarity 0.87)
- Vision-and-Language Navigation via Causal Learning (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.