Research map / Vision-language and multimodal / Multimodal LLMs
Towards Building Large Scale Multimodal Domain-Aware Conversation Systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add image context to text-only HRED | HRED · MMD V1 text response | BLEU: 14.58 -> 20.42 | Helped |
| Add image context to text-only HRED | HRED · MMD V2 text response | BLEU: 35.9 -> 56.67 | Helped |
| Add image context to text-only HRED | HRED · MMD V1 best image selection (m=5) | Recall@1: 0.46 -> 0.72 | Helped |
| Add image context to text-only HRED | HRED · MMD V2 best image selection (m=5) | Recall@1: 0.44 -> 0.69 | Helped |
| Add attention to Multimodal HRED (ablation) | Multimodal HRED · MMD V1 text response | BLEU: 20.42 -> 19.58 | Hurt |
| Add attention to Multimodal HRED (ablation) | Multimodal HRED · MMD V2 text response | BLEU: 56.67 -> 50.20 | Hurt |
| Add attention to Multimodal HRED (ablation) | Multimodal HRED · MMD V1 best image selection (m=5) | Recall@1: 0.72 -> 0.79 | Helped |
| Add attention to Multimodal HRED (ablation) | Multimodal HRED · MMD V2 best image selection (m=5) | Recall@1: 0.69 -> 0.78 | Helped |
| Use context size 5 instead of 2 (ablation) | Multimodal HRED · MMD V1 text response | BLEU: 20.42 -> 19.73 | Hurt |
| Use context size 5 instead of 2 (ablation) | Multimodal HRED · MMD V2 text response | BLEU: 56.67 -> 56.15 | Hurt |
| Include image descriptions in the dataset context (V1 instead of V2) (ablation) | Multimodal HRED · MMD best image selection (m=5) | Recall@1: 0.69 -> 0.72; 0.78 -> 0.79 | Mixed |
Closest papers by meaning
- Structure-Aware Multimodal Sequential Learning for Visual Dialog (AAAI 2024 · similarity 0.88)
- MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMs (NeurIPS 2024 · similarity 0.87)
- V^2Dial: Unification of Video and Visual Dialog via Multimodal Experts (CVPR 2025 · similarity 0.86)
- ConvBench: A Multi-Turn Conversation Evaluation Benchmark with Hierarchical Ablation Capability for Large Vision-Language Models (NeurIPS 2024 · similarity 0.86)
- F-LMM: Grounding Frozen Large Multimodal Models (CVPR 2025 · similarity 0.86)
- Can Large Language Models Help Multimodal Language Analysis? MMLA: A Comprehensive Benchmark (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.