AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

Efficient Image Captioning for Edge Devices

Ning Wang, Jiangrong Xie, Hang Luo, Qinglin Cheng, Jihao Wu, Mingbo Jia, Linlin Li

AAAI 2023 · Vision-language and multimodal › Cross-modal learning · approach: Contrastive vision-language alignment · 28 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add pre-training (ablation)LightCap student · COCO Karpathy test splitCIDEr: 103.6 -> 111.6Improved
Add the visual concept extractor (ablation)LightCap with pre-training · COCO Karpathy test splitCIDEr: 111.6 -> 115.0Improved
Add the cross-modal modulator (ablation)LightCap with pre-training and visual concept extractor · COCO Karpathy test splitCIDEr: 115.0 -> 116.8Improved
Add pre-training attention and representation distillation (ablation)LightCap student · COCO testCIDEr: 116.8 -> 117.6Improved
Add pre-training token-probability distillation (ablation)LightCap student · COCO testCIDEr: 117.6 -> 119.6Improved
Add pre-training visual-concept probability distillation (ablation)LightCap student · COCO testCIDEr: 119.6 -> 120.8Improved
Add fine-tuning attention and representation distillation (ablation)LightCap student with pre-training distillation · COCO testCIDEr: 120.8 -> 121.9Improved
Add fine-tuning token-probability distillation (ablation)LightCap student with pre-training distillation · COCO testCIDEr: 120.8 -> 123.4Improved
Add fine-tuning attention, representation, and caption distillation (ablation)LightCap student with pre-training distillation · COCO testCIDEr: 120.8 -> 124.1Improved
Add a three-branch ensemble head with ensemble distillation (ablation)LightCap student with sequential distillation · COCO testCIDEr: +1.7 CIDErImproved
Use constrained beam search decodingLightCap · nocaps validation overallCIDEr: 85.1 -> 90.8Improved
Fine-tune the CLIP ResNet-50 visual backbone end-to-end instead of freezing itLightCap · image captioningcaptioning performanceHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy