Research map / Vision-language and multimodal / Human and embodied modeling
End-to-End Transformer Based Model for Image Captioning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use shifted-window attention together with window attention instead of global MSA or unshifted window attention (ablation) | PureT refining encoder · MSCOCO Karpathy test | CIDEr: 137.5 -> 138.2; 136.8 -> 138.2 | Helped |
| Add pre-fusion of refined global image features and generated words (ablation) | PureT decoder · MSCOCO Karpathy test | CIDEr: 137.9 -> 138.2 | Helped |
| Add pre-fusion (ablation) | Standard Transformer · MSCOCO Karpathy test | CIDEr: 136.4 -> 137.5 | Helped |
| Increase encoder and decoder depth from 1 to 2 blocks (ablation) | PureT · MSCOCO Karpathy test | CIDEr: 135.8 -> 138.2 | Helped |
| Increase encoder and decoder depth from 3 to 4 blocks (ablation) | PureT · MSCOCO Karpathy test | CIDEr: 138.2 -> 138.2 | No clear effect |
| Replace SwinTransformer with ResNeXt-101 (ablation) | PureT · MSCOCO Karpathy test | CIDEr: 138.2 -> 131.8 | Hurt |
| Replace SwinTransformer with ViT (ablation) | PureT · MSCOCO Karpathy test | CIDEr: 138.2 -> 135.2 | Hurt |
| Add the refining encoder (ablation) | PureT · MSCOCO Karpathy test | CIDEr: 134.3 -> 138.2 | Helped |
| Use the proposed refining encoder instead of the M² Transformer encoder (ablation) | PureT · MSCOCO Karpathy test | CIDEr: 135.9 -> 138.2 | Helped |
| Use the proposed refining encoder instead of the X-Transformer encoder (ablation) | PureT · MSCOCO Karpathy test | CIDEr: 135.5 -> 138.2 | Helped |
| Use PureT instead of RSTNet | Image captioning model · MSCOCO Karpathy test, single model | CIDEr: 135.6 -> 138.2 | Helped |
| Use PureT instead of DLCT | Image captioning model · MSCOCO online test c5 | CIDEr: 133.3 -> 136.0 | Helped |
Closest papers by meaning
- Convolutional Image Captioning (CVPR 2018 · similarity 0.89)
- Injecting Semantic Concepts Into End-to-End Image Captioning (CVPR 2022 · similarity 0.89)
- RSTNet: Captioning With Adaptive Attention on Visual and Non-Visual Words (CVPR 2021 · similarity 0.88)
- Meshed-Memory Transformer for Image Captioning (CVPR 2020 · similarity 0.88)
- Vision‑Language‑Vision Auto‑Encoder: Scalable Knowledge Distillation from Diffusion Models (NeurIPS 2025 · similarity 0.85)
- A Comprehensive Study of Decoder-Only LLMs for Text-to-Image Generation (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.