AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

VisualCOMET: Reasoning about the Dynamic Context of a Still Image

Jae Sung Park, Chandra Bhagavatula, Roozbeh Mottaghi, Ali Farhadi, Yejin Choi

ECCV 2020 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 104 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add person grounding (PG) tags (ablation)Image + Event + Place · VisualCOMET val (text given)CIDEr: 17.12 -> 18.27Improved
add person grounding (PG) tags (ablation)Image · VisualCOMET val (no text given)CIDEr: 5.63 -> 7.61Improved
add image input to event and place text (ablation)Event + Place · VisualCOMET val (text given)CIDEr: 15.73 -> 18.27Improved
add image input to event and place text (ablation)Event + Place · VisualCOMET val (text given)Acc@50: 33.06 -> 38.72Improved
add place input to image and event (ablation)Image + Event · VisualCOMET val (text given)CIDEr: 16.49 -> 17.12Improved
add event text to image input (ablation)Image + Place · VisualCOMET val (text given)CIDEr: 6.69 -> 17.12Improved
add EP loss (generate event and place) when text is given (ablation)Image + Event + Place + PG · VisualCOMET val (text given)CIDEr: 18.27 -> 15.00Hurt
add EP loss when no text is given at test time (ablation)Image + Event + Place + PG · VisualCOMET val (no text given)CIDEr: 8.77 -> 11.86Improved
add EP loss when no text is given at test time (ablation)Image + Event + Place + PG · VisualCOMET val (no text given)Acc@50: 17.35 -> 22.7Improved
add image to text-only model, human evaluationEvent + Place · VisualCOMET test, human eval (200 images)human avg correct %: 50.1 -> 61.0Improved
add event/place text training to image-only model, human evaluationImage + PG · VisualCOMET test, human eval (200 images)human avg correct %: 34.4 -> 38.2Improved
use nucleus sampling instead of beam search (ablation)Image + Event + Place + PG · VisualCOMET valunique inferences (UI): 26.49 -> 51.99Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy