Research map / Vision-language and multimodal / Multimodal LLMs
MMR-Bench: A Comprehensive Benchmark for Multimodal LLM Routing
Hao-Xuan Ma, Guannan Lai, Han-Jia Ye
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| adaptive fusion instead of equal-weight fusion (ablation) | KMeansRouter · MMR-Bench full dataset | nAUC: +0.3403 nAUC; QNC +inf -> 1.0585 | Improved |
| adaptive fusion instead of equal-weight fusion (ablation) | MLPRouter · MMR-Bench full dataset | nAUC: +0.0033 nAUC; QNC +inf -> 0.9947 | Improved |
| adaptive fusion instead of equal-weight fusion (ablation) | LinearRouter · MMR-Bench full dataset | nAUC: +0.0124 nAUC; QNC 0.9055 -> 0.9701 | Mixed |
| adaptive fusion instead of equal-weight fusion (ablation) | KNNRouter · MMR-Bench full dataset | nAUC: -0.0074 nAUC; Ps -0.0014 | Hurt |
| use multimodal features instead of text-only or image-only for routing (ablation) | k-means router · MMR-Bench mixed workloads | accuracy at fixed cost | Improved |
| route over MLLM zoo instead of single strongest model | multimodal router · MMR-Bench | cost at matched accuracy: 33% of strongest single model cost | Improved |
| use cross-dataset router instead of best single model (OCR) | routing policy · OCR scenario | peak score Ps: 0.7062 -> 0.7234 | Improved |
| use cross-dataset router instead of best single model (VQA) | routing policy · VQA scenario | peak score Ps: 0.7936 -> 0.8012 | Improved |
| use cross-dataset router instead of best single model (Math) | routing policy · Math scenario | peak score Ps: 0.7592 -> 0.7914 | Improved |
| transfer multimodal-trained router to GSM8K text-only | routing policy · GSM8K | peak score Ps: 94.5 -> 96.7 | Improved |
| transfer multimodal-trained router to MMLU text-only | routing policy · MMLU | peak score Ps: 91.2 -> 92.4 | Improved |
| transfer multimodal-trained router to ARC text-only | routing policy · ARC | peak score Ps: 65.7 -> 66.7 | Improved |
Closest papers by meaning
- VL-RouterBench: A Benchmark for Vision-Language Model Routing (CVPR 2026 · similarity 0.91)
- Dr.LLM: Dynamic Layer Routing in LLMs (ICLR 2026 · similarity 0.91)
- RouteLLM: Learning to Route LLMs from Preference Data (ICLR 2025 · similarity 0.89)
- Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning (AAAI 2026 · similarity 0.89)
- Routing Experts: Learning to Route Dynamic Experts in Existing Multi-modal Large Language Models (ICLR 2025 · similarity 0.89)
- GraphRouter: A Graph-based Router for LLM Selections (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
