Research map / Vision-language and multimodal / Multimodal LLMs
Generalize LMMs to Versatile Visual Modalities via Fabricated Modality Synthesis
Shihao Yuan, Yuanze Li, Ruyi Zhang, Ming LIU, Wangmeng Zuo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply VVM-Tuning (modality synthesis + modality contexts) | LLaVA-1.5 7B · VVM-Bench perception average | accuracy: 60.4% -> 74.1% | Improved |
| apply VVM-Tuning | Qwen-3-VL 4B · VVM-Bench perception average | accuracy: 77.6% -> 82.8% | Improved |
| apply VVM-Tuning | LLaVA-1.5 7B · VVM-Bench understanding average | accuracy: 79.4% -> 84.9% | Improved |
| apply VVM-Tuning | Qwen-3-VL 4B · VVM-Bench understanding average | accuracy: 87.9% -> 89.9% | Improved |
| apply VVM-Tuning | LLaVA-1.5 7B · VVM-Bench perception, Optical Flow | accuracy: 53.2% -> 75.4% | Improved |
| add modality context in the prompt (ablation) | Base LMMs and VVM-Tuned models · VVM-Bench understanding, Thermal/Depth/X-Ray | accuracy: 15.37% (Thermal), 32.75% (Depth), 3.05% (X-Ray) average gain | Improved |
| add modality-unaware perception tasks (ablation) | VVM-Tuning · VVM-Bench perception average | accuracy: 78.1% -> 81.7% | Improved |
| add modality-unaware perception tasks (ablation) | VVM-Tuning · VVM-Bench understanding average | accuracy: 86.2% -> 88.3% | Improved |
| add modality-aware understanding tasks without perception tasks (ablation) | VVM-Tuning · VVM-Bench perception average | accuracy: 78.1% -> 78.0% | No clear effect |
| add modality-aware understanding tasks on top of perception tasks (ablation) | VVM-Tuning · VVM-Bench perception average | accuracy: 81.7% -> 84.1% | Improved |
| add modality-aware understanding tasks on top of perception tasks (ablation) | VVM-Tuning · VVM-Bench understanding average | accuracy: 88.3% -> 90.6% | Improved |
| increase training data from 25% to 100% (ablation) | VVM-Tuning · VVM-Bench perception average | accuracy: 81.9% -> 84.1% | Improved |
Closest papers by meaning
- Vision as a Dialect: Unifying Visual Understanding and Generation via Text-Aligned Representations (NeurIPS 2025 · similarity 0.88)
- LoMo: Local Modality Substitution for Deeper Vision-Language Fusion (NeurIPS 2026 · similarity 0.88)
- CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Models (CVPR 2026 · similarity 0.88)
- Seeing is Understanding: Unlocking Causal Attention into Modality-Mutual Attention for Multimodal LLMs (ICML 2026 · similarity 0.88)
- Keep the Balance: A Parameter-Efficient Symmetrical Framework for RGB+X Semantic Segmentation (CVPR 2025 · similarity 0.88)
- UNIALIGN: Scaling Multimodal Alignment within One Unified Model (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
