Research map / Vision-language and multimodal / Human and embodied modeling
Learning Distinct and Representative Modes for Image Captioning
Qi Chen, Chaorui Deng, Qi Wu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Hungarian mode assignment (vs DML w/o Hungarian) (ablation) | Transformer-DML k=64 · COCO | oracle CIDEr: 1.211 -> 1.871 | Improved |
| add fully non-autoregressive objective (vs w/o NAT) (ablation) | Transformer-DML k=64 · COCO | oracle CIDEr: 1.207 -> 1.871 | Improved |
| increase sampled captions for MIC branch from 1 to 5 at batch 64 (ablation) | Transformer-DML · COCO | oracle CIDEr: 1.871 -> 1.779 | Hurt |
| increase sampled captions from 1 to 2 at batch 64 (ablation) | Transformer-DML · COCO | oracle CIDEr: 1.871 -> 1.833 | Hurt |
| reduce total batch size from 64 to 16 (2 captions) (ablation) | Transformer-DML · COCO | oracle CIDEr: 1.833 -> 1.760 | Hurt |
| increase sampled captions from 2 to 5 at batch 16 (ablation) | Transformer-DML · COCO | oracle CIDEr: 1.760 -> 1.799 | Improved |
| mask probability 0.5 instead of 1.0 in masked decoder (ablation) | Transformer-DML · COCO | oracle CIDEr: 1.871 -> 1.384 | Hurt |
| decay mask probability 1.0 -> 0.0 instead of fixed 1.0 (ablation) | Transformer-DML · COCO | oracle CIDEr: 1.871 -> 1.579 | Hurt |
| mask probability 0.0 instead of 1.0 (ablation) | Transformer-DML · COCO | oracle CIDEr: 1.871 -> 1.207 | Hurt |
| replace Transformer-BS with Transformer-DML modes | Transformer · COCO diversity | Div-1: 0.21 -> 0.43 | Improved |
| replace Transformer-BS with Transformer-DML | Transformer · COCO diversity | mBLEU (lower better): 0.78 -> 0.54 | Improved |
| use DML vs COS-CVAE | Transformer-DML · COCO oracle 20 samples | oracle CIDEr: 1.624 -> 1.704 | Improved |
Closest papers by meaning
- Guiding Image Captioning Models Toward More Specific Captions (ICCV 2023 · similarity 0.87)
- Length-Controllable Image Captioning (ECCV 2020 · similarity 0.87)
- Unveiling Effective In-Context Configurations for Image Captioning: An External & Internal Analysis (TPAMI 2026 · similarity 0.87)
- Controllable Image Captioning via Prompting (AAAI 2023 · similarity 0.87)
- CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning (ICLR 2026 · similarity 0.86)
- Revisit Large-Scale Image-Caption Data in Pre-training Multimodal Foundation Models (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
