Research map / Vision-language and multimodal / Cross-modal learning
Efficient Image Captioning for Edge Devices
Ning Wang, Jiangrong Xie, Hang Luo, Qinglin Cheng, Jihao Wu, Mingbo Jia, Linlin Li
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add pre-training (ablation) | LightCap student · COCO Karpathy test split | CIDEr: 103.6 -> 111.6 | Improved |
| Add the visual concept extractor (ablation) | LightCap with pre-training · COCO Karpathy test split | CIDEr: 111.6 -> 115.0 | Improved |
| Add the cross-modal modulator (ablation) | LightCap with pre-training and visual concept extractor · COCO Karpathy test split | CIDEr: 115.0 -> 116.8 | Improved |
| Add pre-training attention and representation distillation (ablation) | LightCap student · COCO test | CIDEr: 116.8 -> 117.6 | Improved |
| Add pre-training token-probability distillation (ablation) | LightCap student · COCO test | CIDEr: 117.6 -> 119.6 | Improved |
| Add pre-training visual-concept probability distillation (ablation) | LightCap student · COCO test | CIDEr: 119.6 -> 120.8 | Improved |
| Add fine-tuning attention and representation distillation (ablation) | LightCap student with pre-training distillation · COCO test | CIDEr: 120.8 -> 121.9 | Improved |
| Add fine-tuning token-probability distillation (ablation) | LightCap student with pre-training distillation · COCO test | CIDEr: 120.8 -> 123.4 | Improved |
| Add fine-tuning attention, representation, and caption distillation (ablation) | LightCap student with pre-training distillation · COCO test | CIDEr: 120.8 -> 124.1 | Improved |
| Add a three-branch ensemble head with ensemble distillation (ablation) | LightCap student with sequential distillation · COCO test | CIDEr: +1.7 CIDEr | Improved |
| Use constrained beam search decoding | LightCap · nocaps validation overall | CIDEr: 85.1 -> 90.8 | Improved |
| Fine-tune the CLIP ResNet-50 visual backbone end-to-end instead of freezing it | LightCap · image captioning | captioning performance | Hurt |
Closest papers by meaning
- ViPCap: Retrieval Text-Based Visual Prompts for Lightweight Image Captioning (AAAI 2025 · similarity 0.87)
- MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training (CVPR 2024 · similarity 0.87)
- MM-SeR: Multimodal Self-Refinement for Lightweight Image Captioning (CVPR 2026 · similarity 0.86)
- Modeling Caption Diversity in Contrastive Vision-Language Pretraining (ICML 2024 · similarity 0.86)
- Panoptic Captioning: An Equivalence Bridge for Image and Text (NeurIPS 2025 · similarity 0.86)
- CAPability: A Comprehensive Visual Caption Benchmark for Evaluating Both Correctness and Thoroughness (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
