AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

Learning to Learn Words from Visual Scenes

Dídac Surís, Dave Epstein, Heng Ji, Shih-Fu Chang, Carl Vondrick

ECCV 2020 · Vision-language and multimodal › Human and embodied modeling · approach: Language-grounded visual concept learning · 8 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add via-vision attention instead of isolated attention (ablation)EXPERT · EPIC-Kitchens new-word acquisition 1:1top-1 accuracy: 69.0 -> 72.7Improved
add via-vision attention instead of isolated attention (ablation)EXPERT · EPIC-Kitchens new-word acquisition 2:1top-1 accuracy: 57.8 -> 64.5Improved
add input pointing on top of via-vision attention (ablation)EXPERT · EPIC-Kitchens new-word acquisition 1:1top-1 accuracy: 72.7 -> 75.0Improved
add input pointing on top of via-vision attention (ablation)EXPERT · EPIC-Kitchens familiar words (all)top-5 accuracy: 74.9 -> 67.0Hurt
use full quadratic attention instead of via-vision attention (ablation)EXPERT · EPIC-Kitchens new-word acquisition 1:1top-1 accuracy: 72.7 -> 76.6Mixed
use full attention instead of via-vision attention (ablation)EXPERT · EPIC-Kitchens familiar words (all)top-5 accuracy: 74.9 -> 67.3Hurt
add visual grounding (BERT with Vision vs BERT scratch)BERT · EPIC-Kitchens new-word acquisition 1:1top-1 accuracy: 36.5 -> 63.4Improved
add meta-learned episodic training (EXPERT vs BERT with Vision)BERT with Vision · Flickr30k new-word acquisition 1:1top-1 accuracy: 32.1 -> 69.3Improved
add meta-learned episodic training (EXPERT vs BERT with Vision)BERT with Vision · EPIC-Kitchens new compositionstop-5 accuracy: 37.6 -> 53.0Improved
add language pretraining (BERT pretrained vs scratch)BERT · Flickr30k new-word acquisition 1:1top-1 accuracy: 31.6 -> 69.4Improved
remove bounding boxes at test time, keep full image only (ablation)EXPERT · EPIC-Kitchens new-word acquisitiontop-1 accuracy: 4% worse with 1:1, 2% worse with 2:1Hurt
remove region proposals at test time (ablation)EXPERT · Flickr30kaccuracy: decreases by at most 7%Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy