Research map / Detection and segmentation / CNN features
Dual-level Collaborative Transformer for Image Captioning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DLCT rather than X-Transformer | MS-COCO captioning with ResNeXt-101 · MS-COCO Karpathy test split | CIDEr: 132.8 -> 133.8 | Helped |
| Use DLCT rather than X-Transformer | MS-COCO captioning, 4-model ensemble · MS-COCO Karpathy test split | CIDEr: 135.3 -> 137.5 | Helped |
| Use DLCT rather than X-Transformer | MS-COCO official test with c40 evaluation · MS-COCO official test, c40 | CIDEr: 133.5 -> 135.4 | Helped |
| Use grid and region features together instead of grid features only (ablation) | Standard Transformer · MS-COCO Karpathy test split | CIDEr: 131.2 -> 131.6 | Helped |
| Use grid and region features together instead of region features only (ablation) | Standard Transformer · MS-COCO Karpathy test split | CIDEr: 130.1 -> 131.6 | Helped |
| Add DLCT collaborative processing to the concatenated grid and region features (ablation) | Grid + region feature captioning · MS-COCO Karpathy test split | CIDEr: 131.6 -> 133.8 | Helped |
| Add CRA to the grid- and region-feature DLCT (ablation) | DLCT · MS-COCO Karpathy test split | CIDEr: 133.0 -> 133.8 | Helped |
| Add CRA rather than traditional positional encoding (ablation) | Grid-feature Standard Transformer · MS-COCO Karpathy test split | CIDEr: 131.7 -> 132.5 | Helped |
| Add CRA rather than traditional positional encoding (ablation) | Region-feature Standard Transformer · MS-COCO Karpathy test split | CIDEr: 129.7 -> 131.0 | Helped |
| Add LCCA with the geometric alignment graph (ablation) | DLCT · MS-COCO Karpathy test split | CIDEr: 132.6 -> 133.8 | Helped |
| Constrain cross-attention using geometric alignment rather than a complete bipartite graph (ablation) | LCCA · MS-COCO Karpathy test split | CIDEr: 130.8 -> 133.8 | Helped |
Closest papers by meaning
- Image Captioning: Transforming Objects into Words (NeurIPS 2019 · similarity 0.87)
- Normalized and Geometry-Aware Self-Attention Network for Image Captioning (CVPR 2020 · similarity 0.87)
- Injecting Semantic Concepts Into End-to-End Image Captioning (CVPR 2022 · similarity 0.86)
- Improving Image Captioning by Leveraging Intra- and Inter-layer Global Representation in Transformer Network (AAAI 2021 · similarity 0.86)
- Attention-Aligned Transformer for Image Captioning (AAAI 2022 · similarity 0.86)
- Composing Object Relations and Attributes for Image-Text Matching (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.