AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Otter: A Multi-Modal Model With In-Context Instruction Tuning

Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Fanyi Pu, Joshua Adrian Cahyono, Jingkang Yang, Chunyuan Li, Ziwei Liu

TPAMI 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Visual instruction tuning · 574 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add answer format to the task description prefix (ablation)Otter-Image (LLaVA-Instruct ablation) · MME Cognitionscore: 1212.9 -> 1414.5Improved
add question coverage to the task description (ablation)Otter-Image (LLaVA-Instruct ablation) · MME Cognitionscore: 1212.9 -> 1315.2Improved
add model ability to the task description (ablation)Otter-Image (LLaVA-Instruct ablation) · MM-VetGPT-4 score: 23.7 -> 25.8Improved
add answer format to the task description (ablation)Otter-Image (LLaVA-Instruct ablation) · MM-VetGPT-4 score: 23.7 -> 22.3Hurt
add the full task description at evaluation (ablation)Otter-Image (LLaVA-Instruct ablation) · MM-VetGPT-4 score: 23.7 -> 27.0Improved
train with 2-shot in-context examples (ICIT), evaluate 2-shot (ablation)Otter-Image · MME Cognitionscore: 1310.7 -> 1430.3Improved
train with 4-shot in-context examples (ICIT), evaluate 4-shot (ablation)Otter-Image · MME Cognitionscore: 1173.5 -> 1440.8Improved
increase evaluation shots from 2 to 4 without in-context instruction tuning (ablation)Otter-Image trained w/o ICIT · MME Cognitionscore: 1310.7 -> 1173.5Hurt
add in-context instruction tuning for few-shot COCO Caption (ablation)Otter-Image · COCO Caption few-shotcaptioning scoreImproved
add LLAVAR data to MIMIC-IT subsets (ablation)Otter-Image · MMBenchaccuracy: 41.5 -> 43.2Improved
add academic datasets to MIMIC-IT subsets (ablation)Otter-Image · MMBenchaccuracy: 41.5 -> 62.1Improved
add academic datasets to MIMIC-IT subsets (ablation)Otter-Image · MM-VetGPT-4 score: 19.7 -> 30.6Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy