Research map / Graph learning / Knowledge and temporal graphs
Say As You Wish: Fine-Grained Control of Image Caption Generation With Abstract Scene Graphs
Shizhe Chen, Qin Jin, Peng Wang, Qi Wu
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ASG2Caption instead of C-BUTD | Controllable image captioning · VisualGenome | CIDEr: 159.5 -> 202.4 | Improved |
| Use ASG2Caption instead of C-BUTD | Controllable image captioning · MSCOCO | CIDEr: 143.8 -> 204.2 | Improved |
| Use ASG2Caption instead of C-BUTD | Controllable image captioning · VisualGenome | G_a: 0.7 -> 0.3 | Improved |
| Use ASG2Caption instead of C-BUTD | Controllable image captioning · MSCOCO | G_a: 1.0 -> 0.3 | Improved |
| Generate captions using sampled ASGs rather than the comparison baseline | Diverse caption generation · VisualGenome | SelfCIDEr: 0.47 -> 0.75 | Improved |
| Generate captions using sampled ASGs rather than BUTD beam search | Diverse caption generation · MSCOCO | SelfCIDEr: 0.58 -> 0.76 | Improved |
| Add role-aware node embeddings (ablation) | C-BUTD baseline · VisualGenome | CIDEr: 146.3 -> 176.9 | Improved |
| Add role-aware node embeddings (ablation) | C-BUTD baseline · MSCOCO | CIDEr: 135.7 -> 166.9 | Improved |
| Add MR-GCN contextual graph encoding (ablation) | Role-aware encoder with graph content attention · VisualGenome | CIDEr: 176.9 -> 191.7 | Improved |
| Add MR-GCN contextual graph encoding (ablation) | Role-aware encoder with graph content attention · MSCOCO | CIDEr: 166.9 -> 190.5 | Improved |
| Add graph flow attention (ablation) | Model with role embeddings, MR-GCN, and graph content attention · VisualGenome | CIDEr: 191.7 -> 193.1 | Improved |
| Add graph flow attention (ablation) | Model with role embeddings, MR-GCN, and graph content attention · MSCOCO | CIDEr: 190.5 -> 196.2 | Improved |
Closest papers by meaning
- Show, Control and Tell: A Framework for Generating Controllable and Grounded Captions (CVPR 2019 · similarity 0.89)
- Comprehensive Image Captioning via Scene Graph Decomposition (ECCV 2020 · similarity 0.89)
- Benchmarking Large Vision-Language Models via Directed Scene Graph for Comprehensive Image Captioning (CVPR 2025 · similarity 0.89)
- SGDiff: Scene Graph Guided Diffusion Model for Image Collaborative SegCaptioning (AAAI 2025 · similarity 0.88)
- Fine-Grained Captioning of Long Videos through Scene Graph Consolidation (ICML 2025 · similarity 0.87)
- Scene Graph Generation with Role-Playing Large Language Models (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
