Research map / Vision-language and multimodal / Multimodal LLMs
Multimodal Summarization with Guidance of Multimodal Reference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add image-selection cross-entropy loss (multimodal objective) with RR references | ATG multimodal attention model · MSMO test | image precision: 59.28 -> 65.45 | Helped |
| add multimodal objective function | ATG · MSMO test | ROUGE-L: 37.53 -> 37.80 | Helped |
| add multimodal objective function | ATG · MSMO test | human score: 3.45 -> 3.67 | Helped |
| use decoder last state instead of encoder state in image discriminator (ablation) | MOF-RR · MSMO test | image precision: 62.63 -> 65.45 | Helped |
| use ROUGE-ranking instead of Order-ranking references (ablation) | MOF-dec · MSMO test | image precision: 64.00 -> 65.45 | Helped |
| set loss weight lambda to 1.0 vs 0.5/1.5/2.0 (ablation) | MOF-RR-dec · MSMO test | image precision: 63.44 -> 65.45 | Helped |
| set target image number K=3 vs 1/2/4 (ablation) | MOF-RR-dec · MSMO test | image precision: 63.55 -> 65.45 | Helped |
| add joint multimodal representation metric MRmax to MMAE | MMAE evaluation · human correlation test | Pearson r: .6646 -> .6902 | Helped |
Closest papers by meaning
- MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos (CVPR 2024 · similarity 0.89)
- DIUSum: Dynamic Image Utilization for Multimodal Summarization (AAAI 2024 · similarity 0.86)
- MEGA-Bench: Scaling Multimodal Evaluation to over 500 Real-World Tasks (ICLR 2025 · similarity 0.82)
- A High Quality Dataset and Reliable Evaluation for Interleaved Image-Text Generation (ICLR 2026 · similarity 0.82)
- MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation? (NeurIPS 2025 · similarity 0.82)
- Deep Submodular Optimization and LLM for Multimodal Content Extraction and Automatic Poster Generation from Long Document (AAAI 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.