AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Demystifying Supervision Data Generalization in Multimodal LMs

Xuan Qi, Luxi He, Dan Roth, Xingyu Fu

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Studies whether intuitive similarity between supervision datasets and target benchmarks reliably predicts MLLM performance gains, and how to select training data accordingly.

Problem

Selecting supervision data for multimodal LLMs by intuitive task similarity (e.g., text-rich vs. vision-centric) may not reliably translate into better test benchmark performance.

Approach

Treats the question empirically: asks whether the effect of one supervision dataset on a test benchmark can be predicted, rather than relying on intuitive similarity between datasets and targets.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
include multimodal perplexity in the influence metric (ablation)DataProphet · data influence rankingKendall's tau: 0.487 -> 0.860Improved
include image similarity (ablation)DataProphet · data influence rankingKendall's tau: 0.625 -> 0.860Improved
include question diversity (silhouette + entropy) (ablation)DataProphet · data influence rankingKendall's tau: 0.659 -> 0.860Improved
include question similarity (ablation)DataProphet · data influence rankingKendall's tau: 0.778 -> 0.860Improved
include answer similarity (ablation)DataProphet · data influence rankingKendall's tau: 0.810 -> 0.860Improved
add answer length to the metric (ablation)DataProphet · data influence rankingKendall's tau: -0.15Hurt
encode question and answer jointly instead of separately (ablation)DataProphet similarity · data influence rankingKendall's tau: -6%Hurt
use DataProphet reweighting instead of uniform mixtureInternVL3-2B SFT · 14 benchmarks (real data)average score: 67.6 -> 71.0Improved
use DataProphet instead of ICONS (training-based) for real dataInternVL3-2B SFT · 14 benchmarks (real data)average score: 69.6 -> 71.0Improved
use DataProphet instead of Oracle reweightingInternVL3-2B SFT · 14 benchmarks (real data)average score: 70.8 -> 71.0No clear effect
use DataProphet selection on synthetic pool instead of uniformInternVL3-2B SFT · 14 benchmarks (synthetic data)average score: 55.1 -> 62.0Improved
use DataProphet allocation for RL post-training dataQwen2.5VL-3B-Instruct GRPO · 14 benchmarksaverage score: 58.3 -> 59.5 (real); 56.4 -> 57.7 (synthetic)Improved

Surprising findings

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy