Research map / Vision-language and multimodal / Human and embodied modeling
Entangled Transformer for Image Captioning
Guang Li, Linchao Zhu, Ping Liu, Yi Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the transformer encoder (self-attention over semantic inputs) (ablation) | Transformer semantic · COCO VGG token-level | CIDEr: 93.1 -> 96.2 | Improved |
| add the transformer encoder (self-attention over visual inputs) (ablation) | Transformer visual · COCO VGG token-level | CIDEr: 91.8 -> 93.8 | Improved |
| replace late fusion of Tv and Ts with ETA feature-level fusion (ablation) | ETA-Transformer · COCO SCST | CIDEr: 118.7 -> 126.6 | Improved |
| add semantic modality to visual Transformer through ETA+GBC | Transformer visual · COCO SCST | CIDEr: 124.1 -> 126.6 | Improved |
| add preliminary information injection (ETA) to Parallel Attention (ablation) | Parallel Attention · COCO SCST | CIDEr: 124.9 -> 126.6 | Improved |
| add GBC to Stacked Attention (visual first) (ablation) | Stacked Attention visual · COCO SCST | CIDEr: 125.0 -> 126.6 | Improved |
| add GBC to Stacked Attention (semantic first) (ablation) | Stacked Attention semantic · COCO SCST | CIDEr: 124.5 -> 126.6 | Improved |
| use identity function instead of Sigmoid in GBC (ablation) | GBC · COCO token-level | CIDEr: 104.9 -> 115.2 | Improved |
| use identity function instead of Tanh in GBC (ablation) | GBC · COCO token-level | CIDEr: 104.1 -> 115.2 | Improved |
| use identity function instead of ReLU in GBC (ablation) | GBC · COCO token-level | CIDEr: 114.0 -> 115.2 | No clear effect |
| add semantic attributes alongside visual features (ETA vs Transformer visual only) | Transformer · COCO cross-entropy | CIDEr: 112.2 -> 117.9 | Improved |
| replace LSTM-based Up-Down with ETA-Transformer under SCST | Up-Down · COCO SCST | CIDEr: 120.1 -> 126.6 | Improved |
Closest papers by meaning
- Recurrent Fusion Network for Image captioning (ECCV 2018 · similarity 0.90)
- With a Little Help from Your Own Past: Prototypical Memory Networks for Image Captioning (ICCV 2023 · similarity 0.88)
- RSTNet: Captioning With Adaptive Attention on Visual and Non-Visual Words (CVPR 2021 · similarity 0.88)
- Long-Short Transformer: Efficient Transformers for Language and Vision (NeurIPS 2021 · similarity 0.87)
- Meshed-Memory Transformer for Image Captioning (CVPR 2020 · similarity 0.87)
- Attention-Aligned Transformer for Image Captioning (AAAI 2022 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
