Research map / Vision-language and multimodal / Human and embodied modeling
Context-Aware Group Captioning via Self-Attention and Contrastive Features
Zhuowan Li, Quan Tran, Long Mai, Zhe Lin, Alan L. Yuille
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add self-attention for group feature aggregation (ablation) | Average feature aggregation · Conceptual and Stock | WordAcc: 36.7329 -> 37.9916; 37.9428 -> 39.2410 | Improved |
| Add contrastive features based on shared group information (ablation) | Self-attention group captioning · Conceptual and Stock | WordAcc: 37.9916 -> 39.4496; 39.2410 -> 40.6113 | Improved |
| Add contrastive features to average-pooled representations (ablation) | Average feature aggregation · Conceptual and Stock | WordAcc: 36.7329 -> 37.8450; 37.9428 -> 39.1985 | Improved |
| Apply self-attention jointly across target and reference images (ablation) | Group captioning · Conceptual and Stock | WordAcc: 37.9916 -> 36.1231; 39.2410 -> 38.9215 | Hurt |
| Add cross-attention between groups after within-group self-attention (ablation) | Self-attention group captioning · Conceptual and Stock | WordAcc: 37.9916 -> 36.2892; 39.2410 -> 38.6316 | Hurt |
| Use negative cross-attention to contrast the groups (ablation) | Self-attention group captioning · Conceptual and Stock | WordAcc: 37.9916 -> 37.6046; 39.2410 -> 39.3278 | Mixed |
| Represent contrast as target-group features minus reference-group features (ablation) | Self-attention group captioning · Conceptual and Stock | WordAcc: 37.9916 -> 38.2574; 39.2410 -> 39.9114 | Improved |
| Use only target-group features minus joint features as contrast (ablation) | Self-attention group captioning · Conceptual and Stock | WordAcc: 37.9916 -> 38.5916; 39.2410 -> 40.2068 | Improved |
| Include both target and reference residual features relative to joint information (ablation) | SA+Contrast2 · Conceptual and Stock | WordAcc: 38.5916 -> 39.4496; 40.2068 -> 40.6113 | Improved |
| Use five target images instead of three (ablation) | SA+Contrast · Stock | WordAcc: 34.6574 -> 40.6113 | Improved |
| Add reference images to the target group (ablation) | SA+Contrast · Stock | WordAcc: 31.8061 -> 37.1283 -> 39.4072 -> 40.6113 | Improved |
| Train with more random noise images in the target group (ablation) | Group captioning model · Conceptual | WordAcc | Mixed |
Closest papers by meaning
- GroupCap: Group-Based Image Captioning With Structured Relevance and Diversity Constraints (CVPR 2018 · similarity 0.92)
- Group-Based Distinctive Image Captioning with Memory Difference Encoding and Attention (IJCV 2024 · similarity 0.89)
- Exploring Group Video Captioning with Efficient Relational Approximation (ICCV 2023 · similarity 0.88)
- Fine-Grained Captioning of Long Videos through Scene Graph Consolidation (ICML 2025 · similarity 0.84)
- CAPability: A Comprehensive Visual Caption Benchmark for Evaluating Both Correctness and Thoroughness (NeurIPS 2025 · similarity 0.84)
- Benchmarking Large Vision-Language Models via Directed Scene Graph for Comprehensive Image Captioning (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
