Research map / Vision-language and multimodal / Human and embodied modeling
Matching Visual Features to Hierarchical Semantic Topics for Image Paragraph Captioning
Dandan Guo, Ruiying Lu, Bo Chen, Zequn Zeng, Mingyuan Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add VTCM hierarchical topics to LSTM-ATT | LSTM-ATT · Stanford image-paragraph | CIDEr: 20.11 -> 22.82 | Improved |
| add VTCM hierarchical topics to M2 Transformer | M2 Transformer · Stanford image-paragraph | CIDEr: 16.1 -> 26.15 | Improved |
| add VTCM topics to M2 Transformer | M2 Transformer · Stanford image-paragraph | METEOR: 15.4 -> 16.88 | Improved |
| add self-critical training to VTCM-LSTM | VTCM-LSTM · Stanford image-paragraph | CIDEr: 22.82 -> 25.50 | Improved |
| increase topic layers from 1 to 3 (ablation) | VTCM-LSTM · Stanford image-paragraph | CIDEr: 19.10 -> 22.82 | Improved |
| increase topic layers from 1 to 3 (ablation) | VTCM-Transformer · Stanford image-paragraph | CIDEr: 22.71 -> 26.15 | Improved |
| use variational topic encoder (VTCM) instead of two-stage PGBN (ablation) | VTCM-LSTM · Stanford image-paragraph | CIDEr: 20.49 -> 22.82 | Improved |
| use variational topic encoder (VTCM) instead of two-stage PGBN (ablation) | VTCM-Transformer · Stanford image-paragraph | CIDEr: 24.83 -> 26.15 | Improved |
| use multi-layer VTCM instead of ProdLDA shallow topics (ablation) | LSTM LM · Stanford image-paragraph | CIDEr: 18.52 -> 22.82 | Improved |
| use VTCM coupling instead of concatenating topics before softmax (ablation) | LSTM LM · Stanford image-paragraph | CIDEr: 18.02 -> 22.82 | Improved |
| use VTCM coupling instead of concatenating topics before softmax (ablation) | M2 Transformer · Stanford image-paragraph | CIDEr: 23.45 -> 26.15 | Improved |
| replace M2 Transformer with pretrained original Transformer (ablation) | VTCM-Transformer · Stanford image-paragraph | overall | Hurt |
Closest papers by meaning
- A Hierarchical Approach for Generating Descriptive Image Paragraphs (CVPR 2017 · similarity 0.93)
- Diverse and Coherent Paragraph Generation from Images (ECCV 2018 · similarity 0.91)
- Visual Clues: Bridging Vision and Language Foundations for Image Paragraph Captioning (NeurIPS 2022 · similarity 0.89)
- Recurrent Topic-Transition GAN for Visual Paragraph Generation (ICCV 2017 · similarity 0.89)
- Object Relation Attention for Image Paragraph Captioning (AAAI 2021 · similarity 0.89)
- Top-Down Semantic Refinement for Image Captioning (AAAI 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
