AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

TextCaps: a Dataset for Image Captioning with Reading Comprehension

Oleksii Sidorov, Ronghang Hu, Marcus Rohrbach, Amanpreet Singh

ECCV 2020 · Vision-language and multimodal › Human and embodied modeling · approach: Concept-guided image captioning

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add OCR token inputs (ablation)M4C-Captioner · TextCaps valCIDEr: 35.1 -> 89.6Improved
add pointer-network copying of OCR tokens (ablation)M4C-Captioner · TextCaps valCIDEr: 49.2 -> 89.6Improved
add spatial and visual OCR features to semantic-only features (ablation)M4C-Captioner · TextCaps valCIDEr: 69.0 -> 89.6Improved
add semantic and visual OCR features to spatial-only features (ablation)M4C-Captioner · TextCaps valCIDEr: 72.0 -> 89.6Improved
add semantic and spatial OCR features to visual-only features (ablation)M4C-Captioner · TextCaps valCIDEr: 84.0 -> 89.6Improved
add spatial OCR features to semantic and visual features (ablation)M4C-Captioner · TextCaps valCIDEr: 86.0 -> 89.6Improved
train on TextCaps instead of COCOBUTD · TextCaps valCIDEr: 24.2 -> 41.9Improved
train on TextCaps instead of COCOAoANet · TextCaps valCIDEr: 32.3 -> 42.7Improved
train on TextCaps instead of COCOM4C-Captioner · TextCaps valCIDEr: 30.3 -> 89.6Improved
use ground-truth OCR tokens for training and predictionM4C-Captioner · TextCaps valCIDEr: 89.6 -> 104.3Improved
train with GT OCRs but test with Rosetta OCRsM4C-Captioner · TextCaps valCIDEr: 89.6 -> 88.6No clear effect
train on COCO plus TextCaps instead of COCO aloneM4C-Captioner · COCO val (Karpathy split)CIDEr: 112.2 -> 87.5Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy