Research map / Vision-language and multimodal / Multimodal LLMs
MMIFEvol: Towards Evolutionary Multimodal Instruction Following
Haoyu Wang, Sihang Jiang, Xiangru Zhu, Yuyan Chen, Xiaojun Meng, Jiansheng Wei, Yitong Wang, Yanghua Xiao
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MMIFEvol, an evolutionary framework for building multimodal instruction-following data and benchmarks with difficulty stratification, addressing shortcomings of existing datasets.
Problem
Existing multimodal instruction-following datasets and benchmarks lack difficulty stratification, among other shortcomings noted in the abstract (only the first is visible).
Approach
An evolutionary approach that progressively increases instruction difficulty to produce multimodal instruction-following data at graded difficulty levels; details beyond the abstract are not available.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| SFT on MMIFEvol single-task instructions (ablation) | LLaVA-NeXT-7B · MIA-Bench | total score: 69.4 -> 73.5 | Improved |
| SFT on MMIFEvol multi-task instructions instead of single-task (ablation) | LLaVA-NeXT-7B · MIA-Bench | total score: 73.5 -> 76.2 | Improved |
| SFT on MMIFEvol multi-task instructions (ablation) | Qwen2.5-VL-7B · MIA-Bench | total score: 83.8 -> 90.7 | Improved |
| SFT on MMIFEvol multi-task instructions (ablation) | InternVL3-8B · MIA-Bench | total score: 84.5 -> 93.6 | Improved |
| SFT on MMIFEvol multi-task instructions (ablation) | InternVL3-8B · MM-IFEval | accuracy: 44.1 -> 47.8 | Improved |
| SFT on MMIFEvol single-task instructions (ablation) | LLaVA-NeXT-7B · MM-IFEval | accuracy: 36.4 -> 36.9 | No clear effect |
| go from single-task to multi-task instructions | 7 MLLMs · MMIFEvol | preference (P): Gemini 75.3 -> 67.3; LLaVA-NeXT 26.7 -> 4.1 | Hurt |
| go from single-task to multi-task instructions | 7 MLLMs · MMIFEvol | accuracy (A): LLaVA-OneVision 68.1 -> 40.4 | Hurt |
| go from single-task to multi-task instructions | Gemini2.0-Flash and InternVL3-8B · MMIFEvol | following (F): Gemini 86.7 -> 84.7; LLaVA-OneVision 68.6 -> 40.0 | Mixed |
Closest papers by meaning
- MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talks (ICLR 2026 · similarity 0.92)
- MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMs (ICLR 2025 · similarity 0.91)
- DINGO: Towards Diverse and Fine-Grained Instruction-Following Evaluation (AAAI 2024 · similarity 0.90)
- Benchmarking Complex Instruction-Following with Multiple Constraints Composition (NeurIPS 2024 · similarity 0.90)
- Beyond task performance: evaluating and reducing the flaws of large multimodal models with in-context-learning (ICLR 2024 · similarity 0.90)
- Dynamic Mixture of Curriculum LoRA Experts for Continual Multimodal Instruction Tuning (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
